Chapter 3. Configuring a VPN with IPsec

In Red Hat Enterprise Linux 8, a virtual private network (VPN) can be configured using the IPsec protocol, which is supported by the Libreswan application.

3.1. Libreswan as an IPsec VPN implementation

In Red Hat Enterprise Linux 8, a Virtual Private Network (VPN) can be configured using the IPsec protocol, which is supported by the Libreswan application. Libreswan is a continuation of the Openswan application, and many examples from the Openswan documentation are interchangeable with Libreswan.

The IPsec protocol for a VPN is configured using the Internet Key Exchange (IKE) protocol. The terms IPsec and IKE are used interchangeably. An IPsec VPN is also called an IKE VPN, IKEv2 VPN, XAUTH VPN, Cisco VPN or IKE/IPsec VPN. A variant of an IPsec VPN that also uses the Level 2 Tunneling Protocol (L2TP) is usually called an L2TP/IPsec VPN, which requires the Optional channel xl2tpd application.

Libreswan is an open-source, user-space IKE implementation. IKE v1 and v2 are implemented as a user-level daemon. The IKE protocol is also encrypted. The IPsec protocol is implemented by the Linux kernel, and Libreswan configures the kernel to add and remove VPN tunnel configurations.

The IKE protocol uses UDP port 500 and 4500. The IPsec protocol consists of two protocols:

  • Encapsulated Security Payload (ESP), which has protocol number 50.
  • Authenticated Header (AH), which has protocol number 51.

The AH protocol is not recommended for use. Users of AH are recommended to migrate to ESP with null encryption.

The IPsec protocol provides two modes of operation:

  • Tunnel Mode (the default)
  • Transport Mode.

You can configure the kernel with IPsec without IKE. This is called Manual Keying. You can also configure manual keying using the ip xfrm commands, however, this is strongly discouraged for security reasons. Libreswan interfaces with the Linux kernel using netlink. Packet encryption and decryption happen in the Linux kernel.

Libreswan uses the Network Security Services (NSS) cryptographic library. Both Libreswan and NSS are certified for use with the Federal Information Processing Standard (FIPS) Publication 140-2.

Important

IKE/IPsec VPNs, implemented by Libreswan and the Linux kernel, is the only VPN technology recommended for use in Red Hat Enterprise Linux 8. Do not use any other VPN technology without understanding the risks of doing so.

In Red Hat Enterprise Linux 8, Libreswan follows system-wide cryptographic policies by default. This ensures that Libreswan uses secure settings for current threat models including IKEv2 as a default protocol. See Using system-wide crypto policies for more information.

Libreswan does not use the terms "source" and "destination" or "server" and "client" because IKE/IPsec are peer to peer protocols. Instead, it uses the terms "left" and "right" to refer to end points (the hosts). This also allows you to use the same configuration on both end points in most cases. However, administrators usually choose to always use "left" for the local host and "right" for the remote host.

3.2. Installing Libreswan

This procedure describes the steps for installing and starting the Libreswan IPsec/IKE VPN implementation.

Prerequisites

  • The AppStream repository is enabled.

Procedure

  1. Install the libreswan packages:

    # yum install libreswan
  2. If you are re-installing Libreswan, remove its old database files:

    # systemctl stop ipsec
    # rm /etc/ipsec.d/*db
  3. Start the ipsec service, and enable the service to be started automatically on boot:

    # systemctl enable ipsec --now
  4. Configure the firewall to allow 500 and 4500/UDP ports for the IKE, ESP, and AH protocols by adding the ipsec service:

    # firewall-cmd --add-service="ipsec"
    # firewall-cmd --runtime-to-permanent

3.3. Creating a host-to-host VPN

To configure Libreswan to create a host-to-host IPsec VPN between two hosts referred to as left and right, enter the following commands on both of the hosts:

Procedure

  1. Generate an RSA key pair on each host:

    # ipsec newhostkey --output /etc/ipsec.d/hostkey.secrets
  2. The previous step returned the generated key’s ckaid. Use that ckaid with the following command on left, for example:

    # ipsec showhostkey --left --ckaid 2d3ea57b61c9419dfd6cf43a1eb6cb306c0e857d

    The output of the previous command generated the leftrsasigkey= line required for the configuration. Do the same on the second host (right):

    # ipsec showhostkey --right --ckaid a9e1f6ce9ecd3608c24e8f701318383f41798f03
  3. In the /etc/ipsec.d/ directory, create a new my_host-to-host.conf file. Write the RSA host keys from the output of the ipsec showhostkey commands in the previous step to the new file. For example:

    conn mytunnel
        leftid=@west
        left=192.1.2.23
        leftrsasigkey=0sAQOrlo+hOafUZDlCQmXFrje/oZm [...] W2n417C/4urYHQkCvuIQ==
        rightid=@east
        right=192.1.2.45
        rightrsasigkey=0sAQO3fwC6nSSGgt64DWiYZzuHbc4 [...] D/v8t5YTQ==
        authby=rsasig
  4. After importing keys, restart the ipsec service:

    # systemctl restart ipsec
  5. Start Libreswan:

    # ipsec setup start
  6. Load the connection:

    # ipsec auto --add mytunnel
  7. Establish the tunnel:

    # ipsec auto --up mytunnel
  8. To automatically start the tunnel when the ipsec service is started, add the following line to the connection definition:

    auto=start

3.4. Configuring a site-to-site VPN

To create a site-to-site IPsec VPN, by joining two networks, an IPsec tunnel between the two hosts, is created. The hosts thus act as the end points, which are configured to permit traffic from one or more subnets to pass through. Therefore you can think of the host as gateways to the remote portion of the network.

The configuration of the site-to-site VPN only differs from the host-to-host VPN in that one or more networks or subnets must be specified in the configuration file.

Prerequisites

Procedure

  1. Copy the file with the configuration of your host-to-host VPN to a new file, for example:

    # cp /etc/ipsec.d/my_host-to-host.conf /etc/ipsec.d/my_site-to-site.conf
  2. Add the subnet configuration to the file created in the previous step, for example:

    conn mysubnet
         also=mytunnel
         leftsubnet=192.0.1.0/24
         rightsubnet=192.0.2.0/24
         auto=start
    
    conn mysubnet6
         also=mytunnel
         leftsubnet=2001:db8:0:1::/64
         rightsubnet=2001:db8:0:2::/64
         auto=start
    
    # the following part of the configuration file is the same for both host-to-host and site-to-site connections:
    
    conn mytunnel
        leftid=@west
        left=192.1.2.23
        leftrsasigkey=0sAQOrlo+hOafUZDlCQmXFrje/oZm [...] W2n417C/4urYHQkCvuIQ==
        rightid=@east
        right=192.1.2.45
        rightrsasigkey=0sAQO3fwC6nSSGgt64DWiYZzuHbc4 [...] D/v8t5YTQ==
        authby=rsasig

3.5. Configuring a remote access VPN

Road warriors are traveling users with mobile clients with a dynamically assigned IP address, such as laptops. The mobile clients authenticate using certificates.

The following example shows configuration for IKEv2, and it avoids using the IKEv1 XAUTH protocol.

On the server:

conn roadwarriors
    ikev2=insist
    # Support (roaming) MOBIKE clients (RFC 4555)
    mobike=yes
    fragmentation=yes
    left=1.2.3.4
    # if access to the LAN is given, enable this, otherwise use 0.0.0.0/0
    # leftsubnet=10.10.0.0/16
    leftsubnet=0.0.0.0/0
    leftcert=gw.example.com
    leftid=%fromcert
    leftxauthserver=yes
    leftmodecfgserver=yes
    right=%any
    # trust our own Certificate Agency
    rightca=%same
    # pick an IP address pool to assign to remote users
    # 100.64.0.0/16 prevents RFC1918 clashes when remote users are behind NAT
    rightaddresspool=100.64.13.100-100.64.13.254
    # if you want remote clients to use some local DNS zones and servers
    modecfgdns="1.2.3.4, 5.6.7.8"
    modecfgdomains="internal.company.com, corp"
    rightxauthclient=yes
    rightmodecfgclient=yes
    authby=rsasig
    # optionally, run the client X.509 ID through pam to allow/deny client
    # pam-authorize=yes
    # load connection, don't initiate
    auto=add
    # kill vanished roadwarriors
    dpddelay=1m
    dpdtimeout=5m
    dpdaction=clear

On the mobile client, the road warrior’s device, use a slight variation of the previous configuration:

conn to-vpn-server
    ikev2=insist
    # pick up our dynamic IP
    left=%defaultroute
    leftsubnet=0.0.0.0/0
    leftcert=myname.example.com
    leftid=%fromcert
    leftmodecfgclient=yes
    # right can also be a DNS hostname
    right=1.2.3.4
    # if access to the remote LAN is required, enable this, otherwise use 0.0.0.0/0
    # rightsubnet=10.10.0.0/16
    rightsubnet=0.0.0.0/0
    fragmentation=yes
    # trust our own Certificate Agency
    rightca=%same
    authby=rsasig
    # allow narrowing to the server’s suggested assigned IP and remote subnet
    narrowing=yes
    # Support (roaming) MOBIKE clients (RFC 4555)
    mobike=yes
    # Initiate connection
    auto=start

3.6. Configuring a mesh VPN

A mesh VPN network, which is also known as an any-to-any VPN, is a network where all nodes communicate using IPsec. The configuration allows for exceptions for nodes that cannot use IPsec. The mesh VPN network can be configured in two ways:

  • To require IPsec.
  • To prefer IPsec but allow a fallback to clear-text communication.

Authentication between the nodes can be based on X.509 certificates or on DNS Security Extensions (DNSSEC).

The following procedure uses X.509 certificates. These certificates can be generated using any kind of Certificate Authority (CA) management system, such as the Dogtag Certificate System. Dogtag assumes that the certificates for each node are available in the PKCS #12 format (.p12 files), which contain the private key, the node certificate, and the Root CA certificate used to validate other nodes' X.509 certificates.

Each node has an identical configuration with the exception of its X.509 certificate. This allows for adding new nodes without reconfiguring any of the existing nodes in the network. The PKCS #12 files require a "friendly name", for which we use the name "node" so that the configuration files referencing the friendly name can be identical for all nodes.

Prerequisites

  • Libreswan is installed, and the ipsec service is started on each node.

Procedure

  1. On each node, import PKCS #12 files. This step requires the password used to generate the PKCS #12 files:

    # ipsec import nodeXXX.p12
  2. Create the following three connection definitions for the IPsec required (private), IPsec optional (private-or-clear), and No IPsec (clear) profiles:

    # cat /etc/ipsec.d/mesh.conf
    conn clear
    	auto=ondemand
    	type=passthrough
    	authby=never
    	left=%defaultroute
    	right=%group
    
    conn private
    	auto=ondemand
    	type=transport
    	authby=rsasig
    	failureshunt=drop
    	negotiationshunt=drop
    	# left
    	left=%defaultroute
    	leftcert=nodeXXXX
    	leftid=%fromcert
            leftrsasigkey=%cert
    	# right
    	rightrsasigkey=%cert
    	rightid=%fromcert
    	right=%opportunisticgroup
    
    conn private-or-clear
    	auto=ondemand
    	type=transport
    	authby=rsasig
    	failureshunt=passthrough
    	negotiationshunt=passthrough
    	# left
    	left=%defaultroute
    	leftcert=nodeXXXX
    	leftid=%fromcert
            leftrsasigkey=%cert
    	# right
    	rightrsasigkey=%cert
    	rightid=%fromcert
    	right=%opportunisticgroup
  3. Add the IP address of the network in the proper category. For example, if all nodes reside in the 10.15.0.0/16 network, and all nodes should mandate IPsec encryption:

    # echo "10.15.0.0/16" >> /etc/ipsec.d/policies/private
  4. To allow certain nodes, for example, 10.15.34.0/24, to work with and without IPsec, add those nodes to the private-or-clear group using:

    # echo "10.15.34.0/24" >> /etc/ipsec.d/policies/private-or-clear
  5. To define a host, for example, 10.15.1.2, that is not capable of IPsec into the clear group, use:

    # echo "10.15.1.2/32" >> /etc/ipsec.d/policies/clear

    The files in the /etc/ipsec.d/policies directory can be created from a template for each new node, or can be provisioned using Puppet or Ansible.

    Note that every node has the same list of exceptions or different traffic flow expectations. Two nodes, therefore, might not be able to communicate because one requires IPsec and the other cannot use IPsec.

  6. Restart the node to add it to the configured mesh:

    # systemctl restart ipsec
  7. Once you finish with the addition of nodes, a ping command is sufficient to open an IPsec tunnel. To see which tunnels a node has opened:

    # ipsec trafficstatus

3.7. Methods of authentication used in Libreswan

You can use the following methods for authentication of end points:

  • Pre-Shared Keys (PSK) is the simplest authentication method. PSKs should consist of random characters and have a length of at least 20 characters. In FIPS mode, PSKs need to comply to a minimum strength requirement depending on the integrity algorithm used. It is recommended not to use PSKs shorter than 64 random characters.
  • Raw RSA keys are commonly used for static host-to-host or subnet-to-subnet IPsec configurations. The hosts are manually configured with each other’s public RSA key. This method does not scale well when dozens or more hosts all need to setup IPsec tunnels to each other.
  • X.509 certificates are commonly used for large-scale deployments where there are many hosts that need to connect to a common IPsec gateway. A central certificate authority (CA) is used to sign RSA certificates for hosts or users. This central CA is responsible for relaying trust, including the revocations of individual hosts or users.
  • NULL authentication is used to gain mesh encryption without authentication. It protects against passive attacks but does not protect against active attacks. However, since IKEv2 allows asymmetrical authentication methods, NULL authentication can also be used for internet scale opportunistic IPsec, where clients authenticate the server, but servers do not authenticate the client. This model is similar to secure websites using TLS.

Protection against quantum computers

In addition to these authentication methods, you can use the Postquantum Preshared Keys (PPK) method to protect against possible attacks by quantum computers. Individual clients or groups of clients can use their own PPK by specifying a (PPKID) that corresponds to an out-of-band configured PreShared Key.

Using IKEv1 with PreShared Keys provided protection against quantum attackers. The redesign of IKEv2 does not offer this protection natively. Libreswan offers the use of Postquantum Preshared Keys (PPK) to protect IKEv2 connections against quantum attacks.

To enable optional PPK support, add ppk=yes to the connection definition. To require PPK, add ppk=insist. Then, each client can be given a PPK ID with a secret value that is communicated out-of-band (and preferably quantum safe). The PPK’s should be very strong in randomness and not be based on dictionary words. The PPK ID and PPK data itself are stored in ipsec.secrets, for example:

@west @east : PPKS "user1" "thestringismeanttobearandomstr"

The PPKS option refers to static PPKs. An experimental function uses one-time-pad based Dynamic PPKs. Upon each connection, a new part of a one-time pad is used as the PPK. When used, that part of the dynamic PPK inside the file is overwritten with zeroes to prevent re-use. If there is no more one-time-pad material left, the connection fails. See the ipsec.secrets(5) man page for more information.

Warning

The implementation of dynamic PPKs is provided as a Technology Preview, and this functionality should be used with caution.

3.8. Deploying a FIPS-compliant IPsec VPN

Use this procedure to deploy a FIPS-compliant IPsec VPN solution based on Libreswan. The following steps also enable you to identify which cryptographic algorithms are available and which are disabled for Libreswan in FIPS mode.

Prerequisites

  • The AppStream repository is enabled.

Procedure

  1. Install the libreswan packages:

    # yum install libreswan
  2. If you are re-installing Libreswan, remove its old NSS database:

    # systemctl stop ipsec
    # rm /etc/ipsec.d/*db
  3. Start the ipsec service, and enable the service to be started automatically on boot:

    # systemctl enable ipsec --now
  4. Configure the firewall to allow 500 and 4500/UDP ports for the IKE, ESP, and AH protocols by adding the ipsec service:

    # firewall-cmd --add-service="ipsec"
    # firewall-cmd --runtime-to-permanent
  5. Switch the system to FIPS mode in RHEL 8:

    # fips-mode-setup --enable
  6. Restart your system to allow the kernel to switch to FIPS mode:

    # reboot

Verification steps

  1. To confirm Libreswan is running in FIPS mode:

    # ipsec whack --fipsstatus
    000 FIPS mode enabled
  2. Alternatively, check entries for the ipsec unit in the systemd journal:

    $ journalctl -u ipsec
    ...
    Jan 22 11:26:50 localhost.localdomain pluto[3076]: FIPS Product: YES
    Jan 22 11:26:50 localhost.localdomain pluto[3076]: FIPS Kernel: YES
    Jan 22 11:26:50 localhost.localdomain pluto[3076]: FIPS Mode: YES
  3. To see the available algorithms in FIPS mode:

    # ipsec pluto --selftest 2>&1 | head -11
    FIPS Product: YES
    FIPS Kernel: YES
    FIPS Mode: YES
    NSS DB directory: sql:/etc/ipsec.d
    Initializing NSS
    Opening NSS database "sql:/etc/ipsec.d" read-only
    NSS initialized
    NSS crypto library initialized
    FIPS HMAC integrity support [enabled]
    FIPS mode enabled for pluto daemon
    NSS library is running in FIPS mode
    FIPS HMAC integrity verification self-test passed
  4. To query disabled algorithms in FIPS mode:

    # ipsec pluto --selftest 2>&1 | grep disabled
    Encryption algorithm CAMELLIA_CTR disabled; not FIPS compliant
    Encryption algorithm CAMELLIA_CBC disabled; not FIPS compliant
    Encryption algorithm SERPENT_CBC disabled; not FIPS compliant
    Encryption algorithm TWOFISH_CBC disabled; not FIPS compliant
    Encryption algorithm TWOFISH_SSH disabled; not FIPS compliant
    Encryption algorithm NULL disabled; not FIPS compliant
    Encryption algorithm CHACHA20_POLY1305 disabled; not FIPS compliant
    Hash algorithm MD5 disabled; not FIPS compliant
    PRF algorithm HMAC_MD5 disabled; not FIPS compliant
    PRF algorithm AES_XCBC disabled; not FIPS compliant
    Integrity algorithm HMAC_MD5_96 disabled; not FIPS compliant
    Integrity algorithm HMAC_SHA2_256_TRUNCBUG disabled; not FIPS compliant
    Integrity algorithm AES_XCBC_96 disabled; not FIPS compliant
    DH algorithm MODP1024 disabled; not FIPS compliant
    DH algorithm MODP1536 disabled; not FIPS compliant
    DH algorithm DH31 disabled; not FIPS compliant
  5. To list all allowed algorithms and ciphers in FIPS mode:

    # ipsec pluto --selftest 2>&1 | grep ESP | grep FIPS | sed "s/^.*FIPS//"
    {256,192,*128}  aes_ccm, aes_ccm_c
    {256,192,*128}  aes_ccm_b
    {256,192,*128}  aes_ccm_a
    [*192]  3des
    {256,192,*128}  aes_gcm, aes_gcm_c
    {256,192,*128}  aes_gcm_b
    {256,192,*128}  aes_gcm_a
    {256,192,*128}  aesctr
    {256,192,*128}  aes
    {256,192,*128}  aes_gmac
    sha, sha1, sha1_96, hmac_sha1
    sha512, sha2_512, sha2_512_256, hmac_sha2_512
    sha384, sha2_384, sha2_384_192, hmac_sha2_384
    sha2, sha256, sha2_256, sha2_256_128, hmac_sha2_256
    aes_cmac
    null
    null, dh0
    dh14
    dh15
    dh16
    dh17
    dh18
    ecp_256, ecp256
    ecp_384, ecp384
    ecp_521, ecp521

3.9. Protecting the IPsec NSS database by a password

By default, the IPsec service creates its Network Security Services (NSS) database with an empty password during the first start. Add password protection by using the following steps.

Note

In the previous releases of RHEL up to version 6.6, you had to protect the IPsec NSS database with a password to meet the FIPS 140-2 requirements because the NSS cryptographic libraries were certified for the FIPS 140-2 Level 2 standard. In RHEL 8, NIST certified NSS to Level 1 of this standard, and this status does not require password protection for the database.

Prerequisite

  • The /etc/ipsec.d directory contains NSS database files.

Procedure

  1. Enable password protection for the NSS database for Libreswan:

    # certutil -N -d sql:/etc/ipsec.d
    Enter Password or Pin for "NSS Certificate DB":
    Enter a password which will be used to encrypt your keys.
    The password should be at least 8 characters long,
    and should contain at least one non-alphabetic character.
    
    Enter new password:
  2. Create the /etc/ipsec.d/nsspassword file containing the password you have set in the previous step, for example:

    # cat /etc/ipsec.d/nsspassword
    NSS Certificate DB:MyStrongPasswordHere

    Note that the nsspassword file use the following syntax:

    token_1_name:the_password
    token_2_name:the_password

    The default NSS software token is NSS Certificate DB. If your system is running in FIPS mode, the name of the token is NSS FIPS 140-2 Certificate DB.

  3. Depending on your scenario, either start or restart the ipsec service after you finish the nsspassword file:

    # systemctl restart ipsec

Verification steps

  1. Check that the ipsec service is running after you have added a non-empty password to its NSS database:

    # systemctl status ipsec
    ● ipsec.service - Internet Key Exchange (IKE) Protocol Daemon for IPsec
       Loaded: loaded (/usr/lib/systemd/system/ipsec.service; enabled; vendor preset: disable>
       Active: active (running)...
  2. Optionally, check that the Journal log contains entries confirming a successful initialization:

    # journalctl -u ipsec
    ...
    pluto[23001]: NSS DB directory: sql:/etc/ipsec.d
    pluto[23001]: Initializing NSS
    pluto[23001]: Opening NSS database "sql:/etc/ipsec.d" read-only
    pluto[23001]: NSS Password from file "/etc/ipsec.d/nsspassword" for token "NSS Certificate DB" with length 20 passed to NSS
    pluto[23001]: NSS crypto library initialized
    ...

Additional resources

  • The certutil(1) man page.
  • For more information about certifications related to FIPS 140-2, see the Government Standards Knowledgebase article.

3.10. Configuring IPsec connections that opt out of the system-wide crypto policies

Overriding system-wide crypto-policies for a connection

The RHEL system-wide cryptographic policies create a special connection called %default. This connection contains the default values for the ikev2, esp, and ike options. However, you can override the default values by specifying the mentioned option in the connection configuration file.

For example, the following configuration allows connections that use IKEv1 with AES and SHA-1 or SHA-2, and IPsec (ESP) with either AES-GCM or AES-CBC:

conn MyExample
  ...
  ikev2=never
  ike=aes-sha2,aes-sha1;modp2048
  esp=aes_gcm,aes-sha2,aes-sha1
  ...

Note that AES-GCM is available for IPsec (ESP) and for IKEv2, but not for IKEv1.

Disabling system-wide crypto policies for all connections

To disable system-wide crypto policies for all IPsec connections, comment out the following line in the /etc/ipsec.conf file:

include /etc/crypto-policies/back-ends/libreswan.config

Then add the ikev2=never option to your connection configuration file.

Additional resources