IPsec Tunneling Between FreeBSD Hosts

IPsec is short for IP security, which is a manner of encrypting
and authenticating Internet traffic all the way down to the IP packet
level. That means that protocols such as SMTP, which are inherently
insecure, can be reliably secured. Inherent to IPv6, IPSec is somewhat
bolted onto the side of IPv4. But it works, and it works very well.

There are two key components to IPsec: Authentication Header (AH)
and Encapsulating Security Protocol (ESP). AH provides
authentication, proving the packet sender really is the sender, and
the data really is what they sent. ESP encrypts the payload of the
packets, and can also provide authentication services (for more info
on authentication via ESP, visit
the IPsec working group.)

Both ESP and AH can be used in both transport mode and tunnel mode.
Transport mode provides security between two endpoints, while tunnel mode provides VPN-like security (VPNs -- virtual private networks --allow off-site users secure access to other networks). In transport mode, packets' payloads (and, in the case of AH, immutable parts of the header, like source address) are signed and/or secured. The packet is then sent to its destination, where it is validated, decrypted, and then processed as a normal packet. Transport mode is most often used between two hosts. As the origination IP matters in transport mode, the packet cannot go through NAT'd networks.

In tunnel mode, AH or ESP (and sometimes both) secures the entire packet, and treats the result as the payload of a new packet. That packet is sent to the other side of the tunnel, where the payload is validated and decrypted. The original packet is then processed as a normal IP packet, and, if appropriate, is sent to its ultimate destination. Tunnel mode is obviously most useful between two routers or a host and a router. Since the original packet is handled in its entirety, tunnel mode ESP can be passed through NAT'd networks.

Tunnel mode ESP is an excellent way to bring security to insecure
portions of a network. For example, I'm writing this article on my
laptop in my living room, thanks to my wireless network. Since WEP is
mostly worthless, all traffic between my laptop and the Internet is
first encrypted and tunneled to my access point (a FreeBSD box). That
way, no one can peek at my traffic as it travels through the air.
This also insures that no one else can use my wireless
connection -- but that's for another time.

Setting up IPSec

IKE (Not the Former US President)

First, you must configure the two hosts to use the
Internet Key Exchange. IKE is a protocol that allows IPsec to
exchange its bulk encryption keys securely and automagically. In
FreeBSD (and NetBSD), IKE is handled by the racoon
daemon. Racoon is standard in FreeBSD 4.0 and above. To ensure
compatibility, however, it's a good idea to make sure both hosts are
running the same version -- install racoon from
/usr/ports/security/racoon.

Racoon's configuration is, by default, in /usr/local/etc/racoon/, and consists of racoon.conf and pke.txt (though you'll need to copy them from their .dist brethren on a fresh install.) There is little that needs to be done with the .conf file, though you might want to fiddle with the key lifetimes (found on the lifetime time xxx <min|sec|hour> lines). The
shorter the key lifetime, the better the connection security. However,
you may find excessively short lifetimes consume many CPU cycles and
disrupt connections as keys are renegotiated.

The most important file for this exercise is pke.txt.
This file should be chmod 600 and owned by root,
otherwise racoon will refuse to read it. It contains pre-shared keys,
a little bit of shared secret information that lets racoon
setup its initial connection.

Practical example time. We'll set up a
secure tunnel between mason (10.0.0.2), the tunnel server, and gluon
(10.0.0.77), the tunnel client. Their preshared key is
macplusbsdcool. On mason, pke.txt should
read:

Policy

Both hosts are now ready to exchange keys... But they don't know
they need to yet. The last remaining step is configuring each host's
IPsec policy -- the rule that tells the host how and when to apply
IPsec -- via the setkey command. setkey
takes input either from a file or from the command line. Since we'll
always want this tunnel, we'll put the policies into a file, and read
them at boot via the command setkey -f /etc/ipsec.rules.
(Launching racoon and setkey at boot is an exercise left to the
reader, but think /usr/local/etc/rc.d/)

For tunneling, we'll want all the client packets to be secured and
tunneled via mason. On the server, all packets to gluon must be
secured. Thus, the configuration on gluon should read:

The lines are pretty self-explanatory. spdadd says that
you're adding a new IPsec policy (as opposed to an encryption key,
for manual IPsec setup -- don't bother going there unless you must).
any says the policy applies to any protocol.
out
and in are the direction the packet is travelling.
Finally, esp/.../require; specifies that the kernel
should use IPsec in tunnel mode, and tunnel the packets from the
first IP to the second IP.

To make all this apply to IPv6, simply use IPv6 addresses.

Apply the policy on both machines by running the above
setkey command.

Testing It Out

With racoon running on both hosts, and the policies in place, ping
one machine from the other. If everything is set up correctly, there
should be a slight pause, then you'll start to see succesful pings.

To make sure your connection is using IPsec, run
tcpdump
on the client. Now pop around to a few sites outside of just the
tunnel server -- to make sure all connections use the tunnel. You
should see output like:

If you see any non-ESP traffic, something is awry. Check to make
sure your destination address is 0.0.0.0/0 on the client. If you
don't see any ESP traffic, there's obviously a problem. Make sure
your policies correspond on both the client and the server. Check
pke.txt on both machines, making sure that the keys are
identical and the IP addresses are correct. If everything checks
out, look through /var/log/racoon.log to see if there
are any problems.

One other word of warning -- if you reboot one of the hosts, and
suddenly have connectivity problems, flush the keys on both machines
by running setkey -F. It's possible for the keys to get
out of sync.

Wrapping Up

You should now have a working IPsec tunnel between your client and
server. Go outside with your wireless connection, and enjoy the
knowledge that no one can snoop your data!

Mike DeGraw-Bertsch
is a security and Unix system administration
consultant in the Boston, Mass. area. When he's not at a job, writing,
hacking with Perl, or playing with his wireless network, he can usually be
found playing goal in ice hockey.