Introduction

With the recent iterations of the CCNA exam blueprint, GRE tunnels have become one of the most commonly discussed topics here on CLN. With this blog, I hope to explain and demonstrate the basic configuration required to form GRE tunnels with specific use cases in mind. I will also address certain pitfalls encountered while implementing GRE tunnels, beginning with a basic overview of tunneling.

Tunneling Overview

Tunnels provide a way to transport protocols that the underlying network does not support. There are several reasons why this may be:

The network infrastructure doesn’t support the traffic type (multicast or broadcast)

The most common use case for tunnels is to connect remote, geographically separated sites over an existing network, most notably routing over a public infrastructure (such as the Internet). When used in this manner, tunnels create VPN overlay networks between remote sites. Packets destined to remote private networks are encapsulated within a new IP header that is used to traverse the public internet.

Tunnels accomplish this by creating a virtual network (overlay network) on top of a physical underlying infrastructure (underlay network), providing a logical interface that emulates a direct physical link connecting the two sites. The tunnel interface encapsulates the original protocol traffic, the passenger protocol, using a carrier protocol. The carrier protocol is then encapsulated inside a transport protocol, which is used to route over the underlying infrastructure.

Traffic that enters the tunnel is forwarded, hidden from the underlying infrastructure. It is received on the other end to be unwrapped and further processed. The devices in the transit network do not examine the original packet's IP header or the payload.

There are many varieties of carrier protocols that can be used to form tunnels, such as L2TP, IP-in-IP, and GRE to name a few. As mentioned earlier, the focus of this blog is on point-to-point GRE tunnels.

GRE Tunnels

GRE tunnels provide an interface the device can use to forward data. The “data” in this sense is the passenger protocol itself, such as IPv6 or IPv4. These tunnels are comprised of three main components:

Delivery Header (Transport Protocol)

GRE Header (Carrier Protocol)

Payload Packet (Passenger Protocol)

The passenger protocol is sent to the GRE tunnel interface where it is encapsulated by the GRE header. The resultant packet is encapsulated by the transport protocol, which adds the delivery header used to route across the underlying network. For example, if IPv6 is to be tunneled across an IPv4 infrastructure, the device forwards the IPv6 packet to the GRE tunnel interface. The GRE tunnel interface adds its header and uses the IPv4 protocol stack to encapsulate and deliver across the underlying IPv4 network infrastructure.

GRE can be used with many different combinations of passenger and transport protocols. However, IPv4 and IPv6 are the most common transport protocols for GRE. For example:

GRE can use IPv4 as the transport protocol to tunnel an IPv4 packet across the underlying network infrastructure.

GRE can use IPv4 as the transport protocol to tunnel an IPv6 packet across the underlying network infrastructure.

GRE can use IPv6 as the transport protocol to tunnel an IPv4 packet across the underlying network infrastructure.

GRE can use IPv6 as the transport protocol to tunnel an IPv6 packet across the underlying network infrastructure.

Why Use GRE Tunnels?

GRE’s support for multiple protocols and packet types makes it ideal for solving many of the problems faced when trying to form VPNs across the Internet. The most obvious issue is that private addressing used in the enterprise cannot be routed across the public Internet. GRE solves this by encapsulating the IP header with private addressing using an outer IP header that uses public addressing.

When it comes to routing, as we know hello messages used by IGPs to discover neighbors are sent as multicast, and the IGP adjacencies are limited to directly connected neighbors.

When an IGP tries to discover neighbors, by default, it will multicast hello messages out to all of the interfaces on which it is enabled. However, in certain situations multicast transmission or routing multicast traffic is unsupported, such as over the public Internet or when using IPsec VPN tunnels. Additionally, even if this limitation were surmounted, IGPs only form adjacencies with directly attached neighbors.

GRE can be used to solve both of these problems:

GRE supports multicast traffic allowing hello messages generated by an IGP to be transported through the GRE tunnel across the underlying infrastructure as a unicast packet. IPsec can then be used to encrypt all traffic flowing through the GRE tunnel.

GRE configuration creates a logical direct connection between two sites over the underlying infrastructure. This means the control plane of the IGP believes it is directly connected to the neighbor with which it is exchanging hellos and therefore can form the adjacency.

GRE Configuration

To illustrate the utility of GRE tunnels, the following example topology has been created. Below there are two sites that need connectivity to each other by the means of a GRE tunnel over a public infrastructure:

This topology includes all of the issues discussed in the previous section:

The two sites require direct connectivity over another underlying infrastructure.

The two sites utilize multiple protocols (IPv4 and IPv6).

The two sites utilize IPv4 private addressing.

The two sites need to exchange routing information in order to establish reachability.

A GRE tunnel capable of tunneling both IPv4 and IPv6 simultaneously has been configured with 102.1.1.0/24 as the overlay network. The 12.1.1.0/24 and 23.1.1.0/24 networks form the transport (underlay) connecting R1 and R2 to the public internet. The tunnel endpoints are the loopback interfaces 1.1.1.1/32 and 2.2.2.2/32 on R1 and R2 respectively. Connectivity between the loopbacks has been established using static default routes.

The following steps list the basic GRE tunnel configuration for this scenario:

Step 1:

The first step is to create the logical interface for the tunnel. In Cisco IOS, tunnels are formed using tunnel interfaces. Tunnel interfaces are virtual interfaces that act as if they are directly connected interfaces and are created using the interface tunnel [tunnel interface number] global configuration command.

The tunnel interface number can be any locally significant number and is arbitrary. However, it is good practice to choose a tunnel interface number starting at 10 or above. This is because certain features, such as IP multicast, also require tunnels and may select lower numbers by default.

Step 2:

Once the tunnel interface is created, the carrier protocol needs to be defined. GRE over IPv4 is the default carrier protocol for tunnel interfaces on Cisco IOS and is automatically enabled whenever you create a tunnel interface. Thetunnel mode gre ip can be used to manually change the transport protocol to IPv4, and the tunnel mode gre ipv6 command can be used to change the transport protocol to IPv6.

Step 3:

Once the carrier protocol has been defined (GRE by default), the passenger protocol must be identified. This is accomplished by configuring the appropriate addresses under the tunnel interface. For the example topology, IPv4 and IPv6 is required to be carried inside the IPv4 tunnel.

Simply assigning an IPv4 address using the ip address x.x.x.x x.x.x.x command designates the passenger protocol as IPv4. Similarly, assigning an IPv6 address to the tunnel interface designates an IPv6 packet as the passenger protocol. The addresses assigned to the tunnel interface directly are referred to as overlay, tunnel, or VPN addresses.

Configuring the tunnel address not only identifies the passenger protocol for the tunnel but it also enables that protocol stack to run over the tunnel. This allows the tunnel interface to source control plane traffic using the tunnel address. This is how routing information is exchanged between tunnel endpoints. The next-hop for all routes through the tunnel is the remote tunnel’s IP address.

Step 4:

In the final step, the source/destination address of the transport header needs to be identified. For the tunnel to work, the tunnel endpoints must have reachability to these addresses. The term “tunnel endpoint” identifies the two devices that create the beginning and end of the tunnel. For example, in the topology, R1 and R2 are the tunnel endpoints.

The source address of the tunnel is configured using the tunnel source command. The source can be any interface on the local router or any valid IP address. The tunnel destination is the address of the remote tunnel endpoint and is set using the tunnel destination command. In order for the tunnel to be operational, the tunnel destination must be reachable by the local router.

The source/destination is mirrored between the two tunnel endpoints. The source of one endpoint is the destination of the other. The IP addresses configured as the source/destination should also match the configured tunnel mode. For instance, if the tunnel mode is gre ipv6 the source/destination addresses should be IPv6 addresses.

The tunnel source can be a loopback or a physical interface. Depending on the network topology, using loopbacks can provide the most high-availability, because loopback addresses are not tied to a physical interface. If a physical interface goes down, the routing protocol of the underlying network can route around the failure.

Optionally, a tunnel key can be configured under the tunnel interface. The tunnel key helps the receiving router associate the received tunnel packet with the proper tunnel interface. This is used in case multiple tunnel interfaces are associated with the same source and destination. In the example topology, since only a single tunnel interface has been created between R1 and R2, the configuration of the tunnel key is not required.

At the end of the above configurations, we should have an operational logical point-to-point link between R1 and R2 using the GRE tunnel interface.

R1#show ip route

[--omitted--] 102.0.0.0/8 is variably subnetted, 2 subnets, 2 masks

C 102.1.1.0/24 is directly connected, Tunnel100

R1#show int tunnel 100

Tunnel100 is up, line protocol is up

[-- omitted --]

Tunnel source 1.1.1.1, destination 2.2.2.2

Tunnel protocol/transport GRE/IP

Pings between the tunnel interfaces on R1 and R2 succeed, verifying communication between the two endpoints.

R1#ping 102.1.1.2 source 102.1.1.1

Type escape sequence to abort.

Sending 5, 100-byte ICMP Echos to 102.1.1.2, timeout is 2 seconds:

Packet sent with a source address of 102.1.1.1

!!!!!

Notice the outer IP header (transport protocol) with source/destination 1.1.1.1 and 2.2.2.2, followed by the GRE header (carrier protocol), and then the original IP packet (passenger protocol) with source/destination 102.1.1.1 and 102.1.1.2, followed by the payload, which in this case is the ping.

Routing Over GRE Tunnels

In order to provide reachability through the tunnel for the networks behind each tunnel endpoint, an overlay routing protocol must be enabled to advertise the prefixes through the tunnel. In the example, OSPF is used as the overlay routing protocol. The commands ip ospf 1 area 0 and ipv6 ospf 1 area 0 are issued on the tunnel interface. This causes R1 and R2 to form an IPv4/IPv6 OSPF adjacency over the tunnel with each other and exchange routes through the tunnel.

R1:

interface Tunnel100

ip address 102.1.1.1 255.255.255.0

ip ospf 1 area 0

ipv6 enable

ipv6 ospf 1 area 0

tunnel source Loopback0

tunnel destination 2.2.2.2

Since the tunnel interfaces are transit links, traffic will never be destined for the tunnel interface itself, negating the need for an IPv6 global unicast address. IPv6 IGPs use the link-local address when forming adjacencies. A simple trick in this scenario is to enable IPv6 on the tunnel interface with the ipv6 enable command. This causes the router to generate only a link-local address and use it to form the IGP adjacencies.

The end result is the routes learned from R2 by R1 are installed in R1’s routing table with the tunnel interface as the exit interface:

!Notice the Next-Hop is the IPv6 Link-Local address of R2’s tunnel interface

Any route in the routing table that uses the tunnel interface as the exit interface will be tunneled across to R2 using the GRE tunnel. A ping is performed from PC1’s IPv4/IPv6 addresses to PC2’s IPv4/IPv6 addresses to verify reachability.

PC1> ping 2001:20::2

2001:20::2 icmp6_seq=1 ttl=60 time=217.214 ms

PC1> ping 192.168.20.2

84 bytes from 192.168.20.2 icmp_seq=1 ttl=62 time=59.990 ms

The captures below confirm the tunnel is fully operational. The IPv4 and IPv6 echo requests are being sent encapsulated in a GRE header which is then encapsulated with a transport header as it is sent across the tunnel.

GRE Tunnel Interface State and Keepalives

One of the shortcomings of GRE tunnels is the inability of the remote tunnel endpoints to signal to each other their line protocol status.

In order to understand this better let’s consider the topology again. R1 uses Loopback0 (1.1.1.1/32) as the tunnel source and has a default static route for the tunnel destination 2.2.2.2/32. In addition, there is also a static route on R1 to the network 192.168.20.0/24 behind R2.

R1#show running-config | in ip route

ip route 0.0.0.0 0.0.0.0 12.1.1.2

ip route 192.168.20.0 255.255.255.0 Tunnel100

R1#show ip route[--omitted--]

S* 0.0.0.0/0 [1/0] via 12.1.1.2

S 192.168.20.0/24 is directly connected, Tunnel100Next, we disable the tunnel interface 100 on R2 by shutting it down.

We can see above that bringing down the Tunnel interface on R2 has no impact on the line protocol status of the connected GRE tunnel interface on R1. This is the stateless nature of GRE tunnels by default. The tunnel endpoints are not aware of the state of the remote endpoint or the network conditions between them.

As a result of this limitation, the routing table still shows routes that use the tunnel interface as the exit. Both of the static routes, to the tunnel destination and to the network 192.168.20.0/24, persist in the routing table with the exit interface as the tunnel, even though the remote tunnel endpoint has been brought down. Traffic routed through this interface will never reach their destination.

R1#show ip route

[--omitted--]S* 0.0.0.0/0 [1/0] via 12.1.1.2

S 192.168.20.0/24 is directly connected, Tunnel100

So in order for point-to-point GRE tunnel interface status to be in up/up state, the following must be true on the local router:

The tunnel source must be a valid IP address or up/up interface with a valid IP address assigned.

The tunnel destination must be routable, not necessarily reachable.

For the tunnel source to be considered valid, if the source is an interface, this interface must be in the up/up state and have a valid IP address assigned. This interface can be a physical interface or a loopback interface. If the interface is not being used as the source, then there must be a valid IP address configured as the tunnel source. This point requires more emphasis and explanation leading to the point that a tunnel source can be configured in two ways:

Specify the interface as the keyword in the tunnel source command (tunnel source loopback0)

When using theinterface keyword as the tunnel source, the tunnel interface status is tied to the status of that interface. For eg, in the below, the tunnel source was configured using the loopback0 keyword (tunnel source Loopback0). When the loopback interface is shut down the tunnel interface goes down as well. This is confirmed in the output below where the tunnel line protocol status has changed to up/down.

R1#show running-config interface tunnel 100

interface Tunnel100

tunnel source Loopback0

R1#show running-config interface loopback 0

interface Loopback0

ip address 1.1.1.1 255.255.255.255

shutdown

!Loopback has been shutdown

R1#show ip int brief

Interface IP-Address OK? Method Status Protocol

Loopback0 1.1.1.1 YES manual administratively down down

Tunnel100 102.1.1.1 YES manual up down!Notice line Protocol status is down

However, if we use an IP address as the tunnel source (tunnel source 1.1.1.1), the tunnel interface will stay up regardless of the loopback interface’s administrative status, as shown below:

R1#show running-config interface tunnel 100

interface Tunnel100

tunnel source 1.1.1.1

R1#show running-config interface loopback 0

interface Loopback0

ip address 1.1.1.1 255.255.255.255

shutdown

!Loopback has been shutdown

R1#show ip int brief

Interface IP-Address OK? Method Status Protocol

Loopback0 1.1.1.1 YES manual administratively down down

Tunnel100 102.1.1.1 YES manual up up

!Line protocol status still remains up

In both examples above, there is no reachability between the tunnel endpoints. When the Loopback0 interface on R1 is shutdown, R1 loses its connected route to 1.1.1.1/32 in its RIB. When R2 tunnels a packet to R1, the destination of the transport header will be 1.1.1.1/32. R1 receives this packet and the routing fails because R1 does not have a route for 1.1.1.1/32 in its RIB. This can be verified using debug ip packet and pinging between the tunnel interfaces of R1 and R2:

Another factor that is tied to the Tunnel interface status is a valid tunnel destination address. The tunnel destination must be routable by the local router. In other words, the local router must have a route to the destination in its routing table. This can be accomplished with a static route as we saw above. However, this address doesn’t have to be reachable by the local router, meaning the local router will not verify end-to-end reachability of the destination address.

As long as a valid tunnel source address or interface and a valid tunnel destination address (routable) is configured, the tunnel interface stays up on the local router even if the other side of the tunnel is not operational.

As we saw, this raises a problem. If a problem in the underlying network prevents successful delivery of the tunnel packets, the traffic flowing through the tunnel will be black holed. This also keeps the static routes intact in the routing table, preventing an alternate route from being installed.

The GRE keepalive function can be used to verify the end-to-end reachability and bring down the line protocol status of the tunnel. The keepalive function is disabled on the tunnel interface by default and can be enabled using the keepalive [seconds [retries]] keyword.

interface Tunnel100

ip address 102.1.1.1 255.255.255.0

ip ospf 1 area 0

ipv6 enable

ipv6 ospf 1 area 0

keepalive 10 3

tunnel source Loopback0

tunnel destination 2.2.2.2!The above configures the router to send keepalives every 10 seconds. If a keepalive is not received after 3 tries the router will bring down the line protocol status of the Tunnel interface

When the keepalives are enabled on R1’s tunnel interface 100, the following happens:

R1 constructs a GRE header with no payload and protocol type 0. The protocol type 0 indicates that it is a keepalive packet. This becomes the passenger protocol.

R1 adds an outer IP header to the above GRE packet with the source set to the tunnel destination, 2.2.2.2/32, and the destination of the packet set to the local router’s tunnel source, 1.1.1.1/32.

The above outer IP header is then encapsulated with another GRE header with the protocol type set to 0x0800 (IP). This becomes the carrier protocol.

Next, a new outer IP header is added to the GRE header in order to transport it over the underlying infrastructure. This becomes the transport protocol that carries the keepalive packet as the payload.

R2 receives the above packet, removes the first GRE header to reveal the IP packet with the source set to itself and forwards the packet back to the destination 1.1.1.1/32.

R1 receives the above GRE packet with the protocol type of 0 which signifies that this is a keepalive packet.

So the innermost GRE packet was effectively bounced back to R1 by R2, allowing the GRE keepalive mechanism to work without any configuration on R2.

The capture below shows the GRE keepalive packet generated by R1 and destined to R2. In essence, R1 has tricked R2 into echoing a packet back to it. Using this method, R1 can verify that the remote tunnel endpoint R2 is up. If this packet isn’t received within a specific time period, R1 brings down the tunnel interface. With the use of keepalives, the tunnel status is based on the actual reachability to the destination.

The GRE keepalives should be enabled at both tunnel endpoints in order to avoid routing black holes.

Recursive Routing

The last section of this blog is about another case that can cause the tunnel to be in an up/down state. This could happen when the tunnel destination is routed and learned through the tunnel itself, resulting in recursive routing.

A possible error message that indicates a recursive routing issue while working with GRE Tunnels is the following:

The error above indicates that the router experienced a problem when trying to forward traffic through the tunnel interface because of an internal logical loop that caused it to temporarily disable the tunnel. This loop is caused by recursive routing, where the tunnel destination address is matched by a route in the RIB with the tunnel interface itself as the exit interface.

A few common scenarios that can lead to recursive routing are:

The route to reach the tunnel destination through the tunnel is a more specific route than through the underlying path

The tunnel interface provides a better metric to reach the tunnel destination. This could happen in case of using a single IGP as the underlay and the overlay, such as RIP. In this case, the hop count to reach the tunnel destination is 1 (because the tunnel appears as a directly connected interface), which could be lower than the hop count through the underlying infrastructure.

Expounding on scenario 1, consider In the example topology 1.1.1.1/32 and 2.2.2.2/32 are the endpoints for the GRE tunnel. A static default route on R1 provides reachability to the tunnel destination 2.2.2.2/32, exiting the physical interface:

R1#show ip route static

[--omitted--]

Gateway of last resort is 12.1.1.2 to network 0.0.0.0

S* 0.0.0.0/0 [1/0] via 12.1.1.2

R1#show ip cef 2.2.2.2

2.2.2.2/32

nexthop 12.1.1.2 FastEthernet2/0

Next, on enabling OSPF on the loopback interface on R2, R1 learns the tunnel destination 2.2.2.2/32 through its tunnel interface.

R2(config)#int lo0

R2(config-if)#ip ospf 1 area 0

R1#show ip route

[--omitted--]

Gateway of last resort is 12.1.1.2 to network 0.0.0.0

S* 0.0.0.0/0 [1/0] via 12.1.1.2

2.0.0.0/32 is subnetted, 1 subnets

O 2.2.2.2 [110/1001] via 102.1.1.2, 00:00:02, Tunnel100

!Static default routes are least specific.

!Tunnel destination is reachable via the tunnel itself due to a more specific route learned via OSPF

The route to the tunnel destination 2.2.2.2/32 on R1 through the tunnel is a more specific route than the static default route. In effect, R1 installs the route through the tunnel destination. This leads R1 to believe that to reach the tunnel endpoint it should use the tunnel interface itself.

This causes a loop in the logic on R1.

When R1 encapsulates a packet going through the GRE tunnel, it adds destination address 2.2.2.2/32 to the transport header.

R1 routes the new packet using the RIB.

The route in the RIB to 2.2.2.2/32 points to tunnel interface 100.

R1 forwards the packet to the tunnel interface and tries to encapsulate the packet again. This causes it to route the packet back to the tunnel interface.

R1 detects this problem and breaks the loop by shutting down the tunnel interface.

The routing adjacency is torn down with R2 and clears the error state.

A few seconds later the tunnel will come back up due to the static default route

The routing adjacency between R1 and R2 is re-established, and R2 once again advertises its endpoint through the tunnel.

R1 re-learns 2.2.2.2/32 through its tunnel interface, taking it back to step 1.

The process will continue over and over, hence the term recursive.

In order to prevent recursive routing with tunnels, the key point we need to take away is to make sure that the route to the tunnel destination in the RIB does not point to the tunnel interface itself as the exit interface. For this we need to make sure that the tunnel destination is not learned via the tunnel. Depending on the implementation, a few possible solutions include:

Separate IGPs can be used for the underlay and the overlay while ensuring the tunnel destination is never advertised in the overlay protocol.

In case a single IGP is being utilized:

The metric of the route learned through the tunnel should be higher than the path through the underlying network. For routing protocols like OSPF and EIGRP, this is taken care of automatically because the tunnel interface bandwidth and delay are set artificially high, resulting in routes learned through the tunnel having a higher metric by default.

Route filtering could be used to filter the tunnel endpoints from being learned over the tunnel. This is useful for a routing protocol like RIP that uses hop count as a metric.

For the example topology, the simplest way to prevent recursive routing is by using a specific static route to reach the tunnel destination. In this way, due to the order of preference, the static route will always be preferred over the dynamically learned route to the tunnel destination.

Conclusion

This blog has emphasized the purpose and common use cases for point-to-point (P2P) GRE tunnels using the most basic configuration steps. The last two sections detailed GRE tunnel interface state and the recursive routing issues encountered while implementing GRE tunnels. The configurations and solutions were engineered to be as simple as possible for demonstration purposes and may not follow exact best-practice standards.