Configuring an IPsec tunnel with Openswan and l2tpd

This document describes how to set up a VPN with Openswan combined with L2TPD. This provides for a more user-friendly experience than a standard IPSec VPN on many client operating systems. Note that for most sitesite VPN’s, you will still want straight IPSec.

If you’re not sure if IPSec is right for you, I have written a quick document about some of the various types of VPN available under Linux. It is available at: http://www.natecarlson.com/linux/linux-vpn.php. I hope this helps clear up some questions.

This page is heavily based on my basic IPSec configuration page, located at http://www.natecarlson.com/linux/ipsec-x509.php. The l2tpd configuration side is based on Jacco de Leeuw’s page, which is the definitive source for anything related to Openswan and L2TP. I’m just trying to simplify things for the average Linux geek — if you need more detailed information, or information about any clients other than Windows, check out his page. If you have any difficulties with this process, please e-mail the Openswan mailing list, or if you can’t get help from there, e-mail me at: ipsec@natecarlson.com.

All of my examples on this page are based on a Debian Sarge system, since all the packages required are readily available. Most examples are readily portable to other distributions; you will just need to get the required software for that distribution.

Setting up your Certificate Authority
I’m assuming you want to use X.509 certificates for authentication. It may be possible to get this working with pre-shared keys, but I haven’t tried it. I am also assuming that you will need your own Certificate Authority dedicated to VPN usage – if you already have access to a CA, you may just want to generate certificates from there (if that’s the case, you can just skim this section.) If you need more details that I am going into here, please read the OpenSSL documentation — it’s fairly detailed. For CA certificate management, my examples use the utilities included with OpenSSL itself – there are third-party tools out there that make this a bit simpler, but I want to keep dependencies low. Note that you do not necessarily need to use your Openswan gateway as the Certificate Authority – it can be any box with OpenSSL installed. In fact, it may be better to use a different box, so if an attacker gains access to your Openswan gateway they don’t have access to your CA, too. If you have any suggestions on how to make this process simpler, please let me know!

Now, on to the good stuff – let’s start setting up our own CA.

1) Install openssl. On Debian, ‘apt-get install openssl’ will take care of this.
2) Find your openssl.cnf file. This file has default values for OpenSSL certificate generation. Here’s a few locations for various distributions:

Debian: /etc/ssl/openssl.cnf
RedHat 7.x+: /usr/share/ssl/openssl.cnf

Open this file in your favorite editor. We will need to change the following options:

‘default_days’: This is the length of time, in days, that your certificates will be valid for, and defaults to 365 days, or 1 year. I recommend setting this to ‘3650’, as that will give you 10 years of validity on your certificates. Since this is for internal use, I am ok with the security ramifications of having a certificate valid for a long time – if you lose it or whatnot, you can revoke it without a problem.

‘[ req_distinguished_name ]’ section: You don’t really *need* to change the options below req_distinguished_name; they just set the default options (such as location, company name, etc) for certificate generation. I find it’s easier to set them here than re-type them for every certificate.

3) Create a directory to house your CA. I generally use something like /var/sslca; you can really use whatever you want. Change the permissions of the directory to 700, so that people will not be able to access the private keys who aren’t supposed to.

4) Find the command ‘CA.sh’ (some distributions rename it to just ‘CA’; don’t ask me why.) Locations on various distributions:

Debian: /usr/lib/ssl/misc/CA.sh
RedHat 7.x+: /usr/share/ssl/misc/CA

Edit this file, and change the line that says ‘DAYS=”days 365″‘ to a very high number (this sets how long the certificate authority’s certificate is valid.) Be sure that this number is higher than the number is Step 1; or else Windows may not accept your certificates. Note that if this number is too high, it can cause problems – I generally set it for 15-20 years.

5) Run the command ‘CA.sh -newca’. Follow the prompts, as below. Example input is in red, and my comments are in blue. Be sure to not use any non-alphanumeric characters, such as dashes, commas, plus signs, etc. These characters may make things more difficult for you.

nate@example:~/sslca$ /usr/lib/ssl/misc/CA.sh -newca
CA certificate filename (or enter to create) (press enter)
Making CA certificate ...
Using configuration from /usr/lib/ssl/openssl.cnf
Generating a 1024 bit RSA private key
.............................................................................+++
........................................+++
writing new private key to './demoCA/private/./cakey.pem'
Enter PEM pass phrase: (enter password -- This is the password you will need to create any other certificates.
Verifying password - Enter PEM pass phrase:(repeat password)
-----
You are about to be asked to enter information that will be incorporated
into your certificate request.
What you are about to enter is what is called a Distinguished Name or a DN.
There are quite a few fields but you can leave some blank
For some fields there will be a default value,
If you enter '.', the field will be left blank.
-----
Country Name (2 letter code) [AU]: (country -- enter your two-letter country code here
State or Province Name (full name) [Some-State]: (Enter your state/province here)
Locality Name (eg, city) []: (Enter your city here)
Organization Name (eg, company) [Internet Widgits Pty Ltd]:
Organizational Unit Name (eg, section) []: (OU, if you like. I usually leave it blank)
Common Name (eg, YOUR name) []: (The name of your Certificate Authority)
Email Address []: (E-Mail Address)
nate@example:~/sslca$

Let’s also generate a crl file, which you’ll need on your gateway boxes:

nate@example:~/sslca$ openssl ca -gencrl -out crl.pem

You’ll need to update this CRL file any time you revoke a certificate.

That’s it, you now have your own certificate authority that you can use to generate certificates.

Generating a Certificate
You will need to generate a certificate for every machine that will be making an IPSec connection. This includes the gateway host, and each of your client machines. This section details how to create the certificate, and convert it to formats needed for Windows and such.

Again, we’ll be using the CA.sh script. Except this time, instead of telling it to create a new Certificate Authority, we’re telling it to request, then sign a certificate:

nate@example:~/sslca$ /usr/lib/ssl/misc/CA.sh -newreq
Using configuration from /usr/lib/ssl/openssl.cnf
Generating a 1024 bit RSA private key
...................................+++
...............................+++
writing new private key to 'newreq.pem'
Enter PEM pass phrase: (Enter password to encrypt the new cert's private key with - you'll need this!
Verifying password - Enter PEM pass phrase: (repeat password)
-----
You are about to be asked to enter information that will be incorporated
into your certificate request.
What you are about to enter is what is called a Distinguished Name or a DN.
There are quite a few fields but you can leave some blank
For some fields there will be a default value,
If you enter '.', the field will be left blank.
-----
Country Name (2 letter code) [AU]: (Country)
State or Province Name (full name) [Some-State]: (State)
Locality Name (eg, city) []: (City)
Organization Name (eg, company) [Internet Widgits Pty Ltd]: (Company)
Organizational Unit Name (eg, section) []: (Blank)
Common Name (eg, YOUR name) []: ("Common Name" -- hostname, username, whatever)
Email Address []: (User's email address)
Please enter the following 'extra' attributes
to be sent with your certificate request
A challenge password []: (Leave blank)
An optional company name []: (Leave blank)
Request (and private key) is in newreq.pem

What we just did is generate a Certificate Request – this is the same type of request that you would send to Thawte or Verisign to get a generally-accepted SSL certificate. For our uses, however, we’ll sign it with our own CA:

nate@example:~/sslca$ /usr/lib/ssl/misc/CA.sh -sign
Using configuration from /usr/lib/ssl/openssl.cnf
Enter PEM pass phrase:(password you entered when creating the ca)
Check that the request matches the signature

That’s all that’s required for Openswan boxes – you’ll need these two files, along with the file ‘cacert.pem’ from the ‘demoCA’ directory, and the ‘crl.pem’ file you generated earlier.
If this certificate is needed for a Windows box, you’ll need to convert it to a p12 format:$ openssl pkcs12 -export -in winhost.example.com.pem -inkey winhost.example.com.key -certfile demoCA/cacert.pem -out winhost.example.com.p12

If you are running Debian, there are binary packages available in Sarge and above. For RedHat or Fedora, ATrpms provides binary packages. I can’t vouch for the quality of these packages, but I do know many people have used them with good success. See http://atrpms.net. If you want to build it from scratch, you can download it from http://www.openswan.org/code, and follow the installation directions included with the package. I recommend the most recent version in the 2.2 series, until 2.3.1 is available – 2.3.0 has some critical bugs.

You now have two options for which IPSec stack you want to install in the kernel – you can either use Openswan’s IPSec stack (KLIPS), or use the built-in IPSec stack in the 2.6 kernel (26sec). If you are running on a stock 2.4 kernel, the only option is KLIPS. You’ll need to patch NAT Traversal support into your kernel (if you intend to use it), and build the ipsec.o kernel module. Otherwise, if you are using a 2.6 kernel or a 2.4 kernel with backported 26sec support (such as the kernel Debian provides), you don’t need to touch the kernel-land at all – you can just install the Openswan user-land utilities and go. With Openswan 2.3.1, we will also have support for KLIPS on 2.6, but without NAT Traversal support (until someone gets around to fixing it!) My current recommendation (and my only tested configuration) is to use a stock kernel, patched with NAT Traversal and with KLIPS added. If you bug me, I’ll probably provide patched up Debian packages. :) I have heard stories about l2tpd not working with the kernel stack.

Once you’ve selected and set up your IPSec stack and installed the user-land programs, you’re ready to move on to configuring Openswan.

Installing the Certificate on your Gateway
This discusses how to install the certificate on your gateway machine. These same steps apply for installing the cert on Openswan clients, too. I’m assuming you’ve already created a certificate for each machine (see the “Generating a Certificate” section) – if that’s not the case, please go back and do that now.

1) Install the files in their proper locations (if installing to a remote machine, please be sure to copy the files in a secure manner):

The ‘roadwarrior-*’ lines allow roadwarriors (IE, regular IPSec clients) to connect to your IPSec gateway itself, the network behind it, and to tunnel all traffic to the ‘net at large through it. The roadwarrior-l2tp entries allow both older and newer versions of Windows to connect to an l2tpd daemon running on the same host as your Openswan gateway. Anyone will a valid certificate signed by your CA will be able to connect to your gateway. This configuration also includes NAT Traversal configuration that will allow anyone a host behind a NAT gateway using RFC1918 private addresses (defined in the ‘virtual_private’ line) to connect. The ‘auto=ignore’ lines are there to disable Opportunistic Encryption, which can cause problems if not configured properly.

1) Install l2tpd. On Debian (assuming you have ‘unstable’ in your sources.list), you can just ‘apt-get install l2tpd’; on other distributions, you can find a binary distribution, or grab the source from http://www.l2tpd.org. If building from source, you proably want to build from the CVS version.

You’ll need to change the IP range to a block of unused addresses on your internal network that you would like to hand out to L2TP clients. The ‘Local IP’ should be the local IP address of your box. The ‘pppoptfile’ specifies which options file to use.

3) Configure your PPP options. From the example above, this is located at /etc/ppp/options.l2tpd.lns.

You’ll need to change ms-dns and ms-wins to match your internal DNS and WINS servers. I’ve got the MTU set rather low so that packets won’t be fragmented – if you leave the MTU at 1500, you may find that things like SMB shares don’t work properly.

You can define multiple users with this method. If it’s not obvious, ‘username’ is the username that will be used for authentication, and ‘password’ is the password. If you’d like to give a user a static IP, you can specify it in the fourth column, ‘IP Addresses’.

That’s it for the server side! Just start l2tpd with ‘/etc/init.d/l2tpd start’, and you’re set to go on to the clients.

This section covers configuring your Windows XP client to connect to the server with L2TP over IPsec.

First of all, please ensure that Windows XP SP2, or the NAT-Traversal patches are installed. This will help your ability to connect while behind a NAT gateway and such. Also, be sure to be logged in as a user with administrator privileges.

1) The first step is to import a certificate on your Windows box. For sake of simplicity, I’ll have you import the certificate using Xelerance’s ‘certimport.exe’ tool.

I’m just covering setting up L2TP over IPSec connections on this page, but if you would like to set up Openswan or Windows IPSec clients, please see my other page at http://www.natecarlson.com/linux/ipsec-x509.php. Note that the server configuration above is alreadty set up to accept normal IPSec connections along with the L2TP connections.

I think debian openssl makes a newkey.pem instead of overwriting newreq.pem as a key, so you should try using -inkey newkey.pem in your command. You will know this is what happened if you have newkey.pem in the working directory and when you cat your winhost.example.com.key file it starts with NEW CERTIFICATE REQUEST or similar instead of BEGIN PRIVATE KEY (these are a paraphrases).

2) If you have problem with converting certificates for using in Windows with Openssl module pkcs12, try this:
You have following files in work directory: newcert.pem newreq.pem newkey.pem crl.pem and demoCA/cacert.pem.
In compliance with the article, newcert.pem renamed to host.example.com.pem and newreq.pem to host.example.com.key.
This command:
$ openssl pkcs12 -export -in winhost.example.com.pem -inkey winhost.example.com.key -certfile demoCA/cacert.pem -out winhost.example.com.p12
does not work because it uses request file in -inkey option. Try this:
$ openssl pkcs12 -export -in newcert.pem -inkey newkey.pem -certfile demoCA/cacert.pem -out winhost.p12

Same problem for me here. I’ve tried again and again and even generated a new certificate but still no luck. I’m using Debian unstable with openswan 1:2.6.38-1 and openssl 1.0.1c-4 on the client machine. The certificate was generated on a Debian squeeze box with openssl 0.9.8o-4squeeze13 in the pkcs12 format, and then imported on the client with :
openssl pkcs12 -in ~/mycert.p12 -nokeys -cacerts -out /etc/ipsec.d/cacerts/myca.pem
openssl pkcs12 -in ~/mycert.p12 -nokeys -clcerts -out /etc/ipsec.d/certs/mycert.pem
openssl pkcs12 -in ~/mycert.p12 -nocerts -out /etc/ipsec.d/private/mykey.key

As an alternative I’ve even tried to convert the key to a different format (as found on google) with “openssl pkcs8 -in mykey.key” but still no luck.

I think this is probably a recently introduced problem because it used to work just fine on this computer a few months ago, although it was with a different certificate which has now expired.

but is the problem in openssl or in openswan ? I don’t know

it might as well be in my configuration, of course, but it hasn’t changed in months.