9 June 2022, 22:26Виталий Лещенко

Building your own IKEv2 VPN server on Ubuntu 20.04

Introduction

A virtual private network, or VPN, lets you securely encrypt traffic as it travels through untrusted networks, such as those at a coffee shop, a conference or an airport. It also lets you reach things you are otherwise not allowed to see.

Internet Key Exchange v2, or IKEv2, is a protocol that allows direct IPSec tunneling between a server and a client. In IKEv2 VPN implementations, IPSec provides encryption for the network traffic. IKEv2 is natively supported on some platforms with no additional applications.

Prerequisites

To follow this tutorial you will need:

One Ubuntu 20.04 server
A bit of patience

Step 1 — Installing StrongSwan

First, we'll install StrongSwan, an open-source IPSec daemon that we'll configure as our VPN server. We'll also install the public key infrastructure (PKI) component so that we can create a certificate authority (CA) to provide credentials for our infrastructure.

Start by updating the local package cache:

sudo apt update

Then install the software by typing:

sudo apt install strongswan strongswan-pki libcharon-extra-plugins libcharon-extauth-plugins libstrongswan-extra-plugins

The additional libcharon-extauth-plugins package makes sure that various clients can authenticate to your server with a shared username and passphrase. The libstrongswan-extra-plugins package is included so that Strongswan supports elliptic curve cipher suites that use the Curve25519 cryptography suite.

An important side note here: ubuntu 20.04 does not install the md4 plugin by default, and it is essential for Windows clients. Nothing will work without it. The module is still present in 18.04, but it is gone in 20.04. Just copy /usr/lib/ipsec/plugins/libstrongswan-md4.so or build it from source (that's how it always goes in Linux - ha-ha). I built it, and here is what I got. Feel free to download it and use it.

Now that everything is installed, let's move on to creating our certificates.

Step 2 — Creating a certificate authority

An IKEv2 server needs a certificate to identify itself to clients. To help create the required certificate, the strongswan-pki package comes with a utility called pki that generates a certificate authority and server certificates.

To begin, let's create a few directories to store all the assets we'll be working on. The directory structure matches some of the directories in /etc/ipsec.d, where we will eventually move everything we create:

mkdir -p ~/pki/{cacerts,certs,private}

Then lock down the permissions so that other users can't see our private files:

chmod 700 ~/pki

Now that we have a directory structure to store everything, we can generate a root key. This will be a 4096-bit RSA key used to sign our root certificate authority.

Run these commands to generate the key:

pki --gen --type rsa --size 4096 --outform pem > ~/pki/private/ca-key.pem

After that we can move on to creating our root certificate authority, using the key we just generated to sign the root certificate:

pki --self --ca --lifetime 3650 --in ~/pki/private/ca-key.pem --type rsa --dn "CN=VPN root CA" --outform pem > ~/pki/cacerts/ca-cert.pem

The --lifetime 3650 flag makes sure that the CA's root certificate is valid for 10 years. A CA's root certificate usually doesn't change, since it would have to be redistributed to every server and client that relies on it, so 10 years is a safe default validity period.

You can change the distinguished name (DN) value to something else if you like. The common name (the CN field) here is just an indicator, so it doesn't have to match anything in your infrastructure.

Now that we have our root certificate authority, we can create a certificate for the VPN server.

Step 3 — Generating a certificate for the VPN server

Now we'll create a certificate and key for the VPN server. This certificate will allow the client to verify the server's authenticity using the CA certificate we just created.

First, create a private key for the VPN server with the following command:

pki --gen --type rsa --size 4096 --outform pem > ~/pki/private/server-key.pem

Now create and sign the VPN server certificate with the certificate authority's key you created in the previous step. Run the following command, but change the Common Name (CN) and the Subject Alternative Name (SAN) fields to your VPN server's DNS name or IP address:

pki --pub --in ~/pki/private/server-key.pem --type rsa \ 
 | pki --issue --lifetime 1825 \ 
 --cacert ~/pki/cacerts/ca-cert.pem \ 
 --cakey ~/pki/private/ca-key.pem \ 
 --dn "CN=server_domain_or_IP" --san server_domain_or_IP \ 
 --flag serverAuth --flag ikeIntermediate --outform pem \ 
 > ~/pki/certs/server-cert.pem

The --flag serverAuth option indicates that the certificate will be used explicitly for server authentication before the encrypted tunnel is established. The --flag ikeIntermediate option is used to support older macOS clients.

Now that we have generated all the TLS/SSL files StrongSwan needs, we can move them into place in the /etc/ipsec.d directory by typing:

sudo cp -r ~/pki/* /etc/ipsec.d/

In this step we created a certificate pair that will be used to secure communication between the client and the server. We also signed the certificates with the CA key, so the client will be able to verify the VPN server's authenticity using the CA certificate. With all these certificates ready, we'll move on to configuring the software.

Step 4 — Configuring StrongSwan

StrongSwan ships with a default configuration file with some examples, but we'll have to do most of the configuration ourselves. Let's back up the file for reference before starting from scratch:

sudo mv /etc/ipsec.conf{,.original}

Create and open a new blank configuration file with your preferred text editor. We'll use nano here:

sudo nano /etc/ipsec.conf

First, we'll tell StrongSwan to log daemon statuses for debugging and to allow duplicate connections. Add these lines to the /etc/ipsec.conf file:

config setup
    charondebug="ike 1, knl 1, cfg 0"
    uniqueids=no

Then we'll create a configuration section for our VPN. We'll also tell StrongSwan to create IKEv2 VPN tunnels and to load this configuration section automatically on startup. Add the following lines to the file:

. . .
conn ikev2-vpn
    auto=add
    compress=no
    type=tunnel
    keyexchange=ikev2
    fragmentation=yes
    forceencaps=yes

We'll also configure dead peer detection to clear any "dangling" connections in case a client disconnects unexpectedly. Add these lines:

. . .
conn ikev2-vpn
    . . .
    dpdaction=clear
    dpddelay=300s
    rekey=no

Next, we'll configure the IPSec parameters of the server's "left" side. Each of the following parameters makes sure the server is configured to accept connections from clients and to identify itself correctly. You'll add each of these settings to the /etc/ipsec.conf file once you're familiar with what they are and why they are used:

left=%any The %any value makes sure the server uses the network interface on which it receives incoming connections for subsequent communication with clients. For example, if you connect to the client over a private network, the server will use the private IP address on which it receives traffic for the rest of the connection.
leftid=@server_domain_or_IP This option controls the name that the server presents to clients. Combined with the next option, leftcert, the leftid option makes sure that the server's configured name and the distinguished name (DN) contained in the public certificate match.
leftcert=server-cert.pem This option is the path to the server's public certificate that you configured in Step 3. Without it, the server won't be able to authenticate itself with clients or finish negotiating the IKEv2 setup.
leftsendcert=always The always value makes sure that any client connecting to the server always receives a copy of the server's public certificate as part of the initial connection setup.
leftsubnet=0.0.0.0/0 The last "left" option you'll add tells clients about the subnets that are reachable behind the server. In this case, 0.0.0.0/0 represents the entire set of IPv4 addresses, meaning that by default the server will tell clients to send all their traffic over the VPN.

Now that you're familiar with each of the relevant "left" options, add them all to the file like this:

. . .
conn ikev2-vpn
    . . .
    left=%any
    leftid=@server_domain_or_IP
    leftcert=server-cert.pem
    leftsendcert=always
    leftsubnet=0.0.0.0/0

Next, we can configure the IPSec parameters of the client's "right" side. Each of the following parameters tells the server how to accept connections from clients, how clients should authenticate to the server, and the private IP address ranges and DNS servers that clients will use. Add each of these settings to the /etc/ipsec.conf file once you're familiar with what they are and why they are used:

right=%any The %any option for the right side of the connection tells the server to accept incoming connections from any remote client.
rightid=%any This option makes sure the server won't reject connections from clients that present an identity before the encrypted tunnel is established.
rightauth=eap-mschapv2 This option configures the authentication method that clients will use to authenticate to the server. eap-mschapv2 is used here for broad compatibility, to support clients such as Windows, macOS and Android devices.
rightsourceip=10.10.10.0/24 This option tells the server to assign private IP addresses to clients from the specified 10.10.10.0/24 pool of IPs.
rightdns=8.8.8.8,8.8.4.4 These IP addresses are Google's public DNS resolvers. They can be changed to use other public resolvers, the VPN server's resolvers, or any other resolvers the clients can reach.
rightsendcert=never This option tells the server that clients don't need to send a certificate to authenticate.

Now that you're familiar with the required "right" side options for the VPN, add the following lines to /etc/ipsec.conf:

. . .
conn ikev2-vpn
    . . .
    right=%any
    rightid=%any
    rightauth=eap-mschapv2
    rightsourceip=10.10.10.0/24
    rightdns=8.8.8.8,8.8.4.4
    rightsendcert=never

Now we'll tell StrongSwan to ask the client for user credentials when they connect:

. . .
conn ikev2-vpn
    . . .
    eap_identity=%identity

Finally, add the following lines to support Linux, Windows, macOS, iOS and Android clients. These lines specify the various key exchange, hashing, authentication and encryption algorithms (commonly called cipher suites) that StrongSwan will allow different clients to use:

. . .
conn ikev2-vpn
    . . .
    ike=chacha20poly1305-sha512-curve25519-prfsha512,aes256gcm16-sha384-prfsha384-ecp384,aes256-sha1-modp1024,aes128-sha1-modp1024,3des-sha1-modp1024!
    esp=chacha20poly1305-sha512,aes256gcm16-ecp384,aes256-sha256,aes256-sha1,3des-sha1!

Each supported cipher suite is separated from the others by a comma. For example, chacha20poly1305-sha512-curve25519-prfsha512 is one, and aes256gcm16-sha384-prfsha384-ecp384 is another. The cipher suites listed here are chosen for maximum compatibility with Windows, macOS, iOS, Android and Linux clients.

The complete configuration file should look like this:

config setup
    charondebug="ike 1, knl 1, cfg 0"
    uniqueids=no

conn ikev2-vpn
    auto=add
    compress=no
    type=tunnel
    keyexchange=ikev2
    fragmentation=yes
    forceencaps=yes
    dpdaction=clear
    dpddelay=300s
    rekey=no
    left=%any
    leftid=@server_domain_or_IP
    leftcert=server-cert.pem
    leftsendcert=always
    leftsubnet=0.0.0.0/0
    right=%any
    rightid=%any
    rightauth=eap-mschapv2
    rightsourceip=10.10.10.0/24
    rightdns=8.8.8.8,8.8.4.4
    rightsendcert=never
    eap_identity=%identity
    ike=chacha20poly1305-sha512-curve25519-prfsha512,aes256gcm16-sha384-prfsha384-ecp384,aes256-sha1-modp1024,aes128-sha1-modp1024,3des-sha1-modp1024!
    esp=chacha20poly1305-sha512,aes256gcm16-ecp384,aes256-sha256,aes256-sha1,3des-sha1!

Save and close the file once you've verified that you've added each line correctly.

Now that we've configured the VPN parameters, let's move on to creating an account so our users can connect to the server.

Step 5 — Configuring VPN authentication

Our VPN server is now configured to accept client connections, but we don't have any credentials configured yet. We need to set up a couple of things in a special configuration file called ipsec.secrets:

We need to tell StrongSwan where to find the private key for our server certificate, so the server can authenticate to clients.
We also need to set up a list of users who will be allowed to connect to the VPN.

Let's open the secrets file for editing:

sudo nano /etc/ipsec.secrets

First, we'll tell StrongSwan where to find our private key and how to parse it. /etc/ipsec.secrets

: RSA "server-key.pem"

Make sure the line begins with the : character and that there is a space after it, so that the entire line reads : RSA "server-key.pem".

Then we'll define the user credentials. You can make up any username and password combination you like:

your_username : EAP "your_password"

Save and close the file. Now that we're done with the VPN parameters, let's restart the VPN service so our configuration takes effect:

sudo systemctl restart strongswan-starter

Now that the VPN server is fully configured with both server options and user credentials, it's time to move on to configuring the most important part: the firewall.

Step 6 — Configuring the firewall and kernel IP forwarding

With the StrongSwan configuration complete, we need to configure the firewall to allow VPN traffic through and to forward it.

If you followed the prerequisite initial server setup guide, you should have a UFW firewall enabled. If you don't have UFW configured yet, start by adding a rule to allow SSH connections through the firewall so that your current session doesn't get closed when you enable UFW:

sudo ufw allow OpenSSH

Then enable the firewall by typing:

sudo ufw enable

Then add a rule to allow UDP traffic to the standard IPSec ports, 500 and 4500:

sudo ufw allow 500,4500/udp

Next, we'll open one of UFW's configuration files to add a few low-level policies for routing and forwarding IPSec packets. Before we can do that, though, we need to find out which network interface on our server is used to access the internet. Find this interface by querying the device associated with the default route:

ip route show default

Your public interface should follow the word "dev". For example, this output shows an interface named eth0, which is highlighted in the following example:

default via your_server_ip dev eth0 proto static

Once you have your public network interface, open the /etc/ufw/before.rules file in your text editor. The rules in this file are added to the firewall before the rest of the usual input and output rules. They are used to configure network address translation (NAT) so that the server can correctly route connections to and from clients and the internet.

sudo nano /etc/ufw/before.rules

At the top of the file (before the *filter line), add the following configuration block. Change each instance of eth0 in the configuration below to match the interface name you found with ip route. The *nat lines create rules so that the firewall can correctly route and manipulate traffic between the VPN clients and the internet. The *mangle line adjusts the maximum packet segment size to prevent potential issues with certain VPN clients:

*nat
-A POSTROUTING -s 10.10.10.0/24 -o eth0 -m policy --pol ipsec --dir out -j ACCEPT
-A POSTROUTING -s 10.10.10.0/24 -o eth0 -j MASQUERADE
COMMIT

*mangle
-A FORWARD --match policy --pol ipsec --dir in -s 10.10.10.0/24 -o eth0 -p tcp -m tcp --tcp-flags SYN,RST SYN -m tcpmss --mss 1361:1536 -j TCPMSS --set-mss 1360
COMMIT

*filter
:ufw-before-input - [0:0]
:ufw-before-output - [0:0]
:ufw-before-forward - [0:0]
:ufw-not-local - [0:0]
. . .

Next, after the *filter chain definition lines, add one more configuration block:

. . .
*filter
:ufw-before-input - [0:0]
:ufw-before-output - [0:0]
:ufw-before-forward - [0:0]
:ufw-not-local - [0:0]

-A ufw-before-forward --match policy --pol ipsec --dir in --proto esp -s 10.10.10.0/24 -j ACCEPT
-A ufw-before-forward --match policy --pol ipsec --dir out --proto esp -d 10.10.10.0/24 -j ACCEPT

These lines tell the firewall to forward ESP (Encapsulating Security Payload) traffic so that VPN clients can connect. ESP provides additional security for our VPN packets as they traverse untrusted networks.

Before restarting the firewall, we'll change some network kernel parameters to allow routing from one interface to another. The file that controls these settings is called /etc/ufw/sysctl.conf. We need to configure a few things in it.

First, IPv4 packet forwarding has to be turned on so that traffic can move between the VPN and the public network interfaces on the server. Next, we'll disable Path MTU discovery to prevent packet fragmentation problems. Finally, we won't accept ICMP redirects nor send ICMP redirects, to prevent man-in-the-middle attacks.

Open UFW's kernel parameters configuration file with nano or any text editor:

sudo nano /etc/ufw/sysctl.conf

Now add the following net/ipv4/ip_forward=1 setting at the end of the file to enable forwarding packets between interfaces:

. . .
net/ipv4/ip_forward=1
. . .
net/ipv4/conf/all/accept_redirects=0
net/ipv4/conf/all/send_redirects=0
. . .
net/ipv4/ip_no_pmtu_disc=1

Save the file when you're done. Now we can enable all our changes by disabling and re-enabling the firewall, since UFW applies these settings every time it restarts:

sudo ufw disable 
sudo ufw enable

You'll be prompted to confirm the process. Type Y to enable UFW again with the new settings.

Step 7 — Testing the VPN connection on Windows

Now that you have everything set up, it's time to try it out. First, you need to copy the CA certificate you created and install it on the client devices that will connect to the VPN. The easiest way to do this is to log in to your server and print the contents of the certificate file:

cat /etc/ipsec.d/cacerts/ca-cert.pem

To import the root CA certificate with PowerShell, first open a PowerShell prompt with administrator privileges.

Import-Certificate -CertStoreLocation cert:\LocalMachine\Root\ -FilePath ~\Documents\ca-cert.pem

Now, to configure the VPN with PowerShell, run the following command. Substitute your server's DNS name or IP address in the -ServerAddress line. The various flags make sure that Windows is correctly configured with the appropriate security parameters that match the options set in the /etc/ipsec.conf file.

Add-VpnConnection -Name "VPN Connection" ` 
 -ServerAddress "server_domain_or_IP" ` 
 -TunnelType "IKEv2" ` 
 -AuthenticationMethod "EAP" ` 
 -EncryptionLevel "Maximum" ` 
 -RememberCredential

Set-VpnConnectionIPsecConfiguration -Name "VPN Connection" ` 
 -AuthenticationTransformConstants GCMAES256 ` 
 -CipherTransformConstants GCMAES256 ` 
 -DHGroup ECP384 ` 
 -IntegrityCheckMethod SHA384 ` 
 -PfsGroup ECP384 ` 
 -EncryptionMethod GCMAES256

Connecting to the VPN

Once you've imported the certificate and configured the VPN using either of these methods, your new VPN connection will show up in the list of networks. Select the VPN and click "Connect". You'll be prompted for your username and password. Enter them, click OK, and you'll be connected.

Moving to swanctl

Update of 2025-12-17: since setting up strongSwan via ipsec is deprecated, I decided to write about how to move from it to swanctl - read it here.