IPSEC Protocol Overview

IPSEC Protocol Overview
IPSEC is a framework for security that operates at the Network Layer
by extending the IP packet header (using additional protocol numbers,
not options). This gives it the ability
to encrypt any higher layer protocol, including arbitrary TCP and
UDP sessions, so it offers the greatest flexibility of all the
existing TCP/IP cryptosystems. Flexibility, however, often comes at
the price of complexity, and IPSEC is not an exception. Configuring
which addresses and ports to encrypt using which IPSEC options often
begins to look like configuring packet filtering, then add in the additional
complexities of key management. While conceptually simple, setting up
IPSEC is much more complex that installing SSH, for example.

IPSEC also has the disadvantage of requiring operating system support, since
most O/S kernels don't allow direct manipulation of IP headers.
This presents legal issues, since cryptographic software is restricted
by many governments. Linux IPSEC support
(the FreeS/WAN project),
for example, isn't included in the standard kernel distribution for
this reason, and has to be applied as an add-on. Furthermore, putting
the cryptography in the kernel isolates it from the application, making
it more difficult to code crypto-aware software. Using SSL, for example,
simply requires linking a library into the application and allows the
application to easily query what certificates have been used to authenticate
a client (for example). Doing the same thing with IPSEC requires the
application to query the kernel using some kind of API.

Figure 1. IPsec Document Roadmap

IPSEC defines a "Security Association" (SA) as its primitive means of
protecting IP packets. An SA is defined by the packet's destination
IP address and a 32-bit Security Parameter Index (SPI), that functions
somewhat like a TCP or UDP port number, in that it allows multiple SAs
to a single destination address. SAs can operate in transport mode,
where the IPSEC data field begins with upper level packet headers
(usually TCP, UDP, or ICMP), or in tunnel mode, where the IPSEC data
field begins with an entirely new IP packet header, ala
RFC 2003. Furthermore,
SAs can be encapsulated within SAs, forming SA bundles, allowing layered
IPSEC protection.

For example, one SA might protect all traffic through a gateway, while
another SA would protect all traffic to a particular host.
The packets finally
routed across the network would be encapsulated in an SA
bundle consisting of both SAs. The other side of the connection
could be identical in design, consisting of a gateway implementing
a tunnel SA, followed by a host implementing a transport SA, or
the entire bundle could be terminated in a single host, which
would then implement both SAs.

Figure 2: Bundled SAs

A common use of IPSEC is the construction of a Virtual Private
Network (VPN), where multiple segments of a private network are linked
over a public network using encrypted tunnels. This allows applications
on the private network to communicate securely without any local
cryptographic support, since the VPN routers perform the encryption and
decryption. IPSEC is well suited for this environment, more so than
tunneling PPP over SSL or SSH, since it operates directly on the IP
packets and preserves a one-to-one correspondence between packets
inside and outside the network. In the case of tunneling PPP over
an encrypted TCP connection, any packet loss in the public network
would trigger a TCP retransmission, stalling the link until the packet
was delivered. In particular, running Voice Over IP (VoIP) traffic
through a TCP/PPP tunnel would largely defeat the RTP protocol used
for VoIP; IPSEC is better suited in this case.