Virtual Private Networks (VPNs) are becoming required expertise for network and security engineers, and IPSec is the most commonly used protocol when implementing VPNs. In this first article of a five-part series on the Cisco implementation of IPSec, Andrew Mason delves into the components that make up the IPSec protocol suite.

From the author of

From the author of

Internet Protocol Security (generally shortened to IPSec) is a
framework of open standards that provides data confidentiality, data integrity,
and data authentication between participating peers at the IP layer. IPSec can
be used to protect one or more data flows between IPSec peers. Documented in a
series of
Internet RFCs,
the overall IPSec implementation is guided by
"Security Architecture for the Internet Protocol,"
RFC 2401. IPSec consists of two main protocols:

Authentication Header (AH)

Encapsulating Security Payload (ESP)

IPSec also uses other existing encryption standards to make up a protocol
suite.

IPSec includes several standards that are supported by Cisco IOS and the PIX
Firewall:

IP Security Protocol

Authentication Header (AH)

Encapsulating Security Payload (ESP)

DES Algorithm

Triple DES Algorithm (2DES)

Diffie-Hellman (D-H)

Message Digest 5 (MD5)

Secure Hash Algorithm-1 (SHA-1)

Rivest, Shamir, and Adelman Signatures (RSA)

Internet Key Exchange (IKE)

Certificate authorities (CA)

These standards are described briefly in the following sections.

IP Security Protocol

The IP Security Protocol consists of the Authentication Header (AH) and the
Encapsulating Security Payload (ESP).

Authentication Header (AH)

Authentication Header (AH) provides authentication and integrity to
the datagrams passed between two systems. This is achieved by applying a keyed
one-way hash function to the datagram to create a message digest. If any part of
the datagram is changed during transit, this will be detected by the receiver
when it performs the same one-way hash function on the datagram and compares the
value of the message digest that the sender has supplied. The fact that the
one-way hash also involves the use of a secret shared between the two systems
means that authenticity can be guaranteed.

AH may also enforce anti-replay protection by requiring that a receiving host
set the replay bit in the header to indicate that the packet has been seen.
Without it, an attacker may be able to resend the same packet many times; for
example, sending a packet that withdraws $100 from account X. Figure 1 shows two
routers and confirms that the data between them is sent in cleartext.

The AH function is applied to the entire datagram except for any mutable IP
header fields that change in transit, such as Time To Live (TTL) fields that are
modified by the routers along the transmission path. AH works as follows:

The IP header and data payload is hashed.

The hash is used to build a new AH header, which is appended to the
original packet.

The new packet is transmitted to the IPSec peer router.

The peer router hashes the IP header and data payload, extracts the
transmitted hash from the AH header, and compares the two hashes. The hashes
must match exactly. If even one bit is changed in the transmitted packet, the
hash output on the received packet will change and the AH header will not
match.

ESP provides confidentiality by performing encryption at the IP packet layer.
It supports a variety of symmetric encryption algorithms. The default algorithm
for IPSec is 56-bit DES. This cipher must be implemented to guarantee
interoperability among IPSec products. Cisco products also support use of 3DES
for strong encryption. Confidentiality may be selected independent of all other
services.

NOTE

Deciding whether to use AH or ESP in a given situation may seem complex, yet
can be simplified to a few rules. When you want to make sure that data from an
authenticated source gets transferred with integrity and doesn't need
confidentiality, use the AH protocol. If you need to keep data private
(confidentiality), then you must use ESP. ESP will encrypt the upper-layer
protocols in transport mode and the entire original IP datagram in tunnel mode
so that neither is readable from the wire. However, ESP can also provide
authentication for the packets.