Junos: L2Circuit over GRE with fragmentation configuration

8032011

A customer has asked me to work out how to get layer-2 traffic from a Juniper SRX, across the public internet and back into another SRX so that they can conserve IP address space. I worked the followign out in the lab, using a pair of SRX240s, and a pair of Cisco routers to simulate the public internet. Crucially, fragmentation of large packets is important, since MTU sizes across the internet could be variable. GRE tunnels don’t fragment by default, but this configuration permits that.

There might be scalability issues with the configuration as shown below, so it needs more work doing to it, but it at least proves that this is possible.

What we’re doing here is running MPLS over a GRE tunnel across the Internet, and putting a layer-2 circuit (Circuit Cross-Connect as Junos calls it) over that.

GRE does not fragment packets by default, but in this scenario, packet fragmentation is required. Originally the intention was to use IPSec to do this, but it seems that the tested Junos version (10.0R3.1), GRE is able to fragment packets.

An example configuration from Juniper’s Application Note entitled “Branch SRX Series and J Series Selective Packet Services” was used to achieve this. It should be noted that the IDP configuration is not included below – the solution was tested with and without the IDP, and it seemed to work in both situations. Also in the Application Note, the “policy-options policy-statement export-ldp” is defined, but appears to be unused.

What appears key to this working is the use of a firewall filters that turn off flow-based processing of packets received in the master routing-instance. This is required so that MPLS can run. There is then a logical tunnel (lt) interface between it and a virtual router, which is operating in flow mode. The GRE tunnel is built from an IP address on the logical tunnel. Conceivably, an IPSec tunnel could be built from the flow-mode virtual router, or security policy could be applied without affecting the operation of the GRE tunnel.

Note that for the purposes of this lab test, no security measures are in place – all host-inbound-traffic is being permitted. As such, this is not a configuration recommended for use on the public internet – it is posted here to show a working example. Further testing is advisable.

The test-bed network diagram looks as follows:

Packets can be pinged between the client devices shown in green at MTUs higher than 1500 bytes (tested up to 9000 bytes) with no issues. The only discernible difference is the round-trip time achieved – with a standard 100 byte ping, the average RTT is 2ms. With 9000 byte pings, this increases to 20ms – presumably because of the fragmentation overhead involved.

SRX1 Configuration

This configuration has been colour-coded to indicate which interfaces/policies/protocols etc. apply to which virtual router

Related

Actions

Information

12 responses

You see, that pesky flow mode is just a pain. I really do wish they would let you tuen it off and still have everything work (like IPsec !)

I have this working on some MX80s at the moment but there are some issues with reassembly. Juniper recommend you turn on tunnel keys it seems.

Some other vendors implement over-size-MTU tunnels using TCP so they see a stream they can easily reassemble without holding fragments in buffers. Of course this is really inefficient and subject to stalling when packets are dropped.

Nice writeup, I was just going to lab something similar up myself, although for a different reason: being able to use packet-mode routing with SRXs using IPsec and possibly having asymmetric routing. (MPLS is in the back of my mind as a future enhancement (path separation) for our setup).

I had just read the same paper as you, and had in the margin made notes:
– Why use IDP, doesn’t GRE fragment/reassemble in flow-mode?
Now with that verified, the next margin note is:
– What use is the Flow-VR in this setup, it is only connecting ge-3/0/1.0 with lt-0/0/0.1? (In your setup: ge-0/0/15.0 with lt-0/0/0.1)

By the way, the RTT values with large ping, were they the same w/wo IDP?
Was there a sharp increase when fragmentation kicked in?

Well to be honest this was a few years ago now – I think there may have been an issue with MTU but nothing that couldn’t be sorted out. As it happened this was the last thing I did for a previous employer, so I don’t know if the customer went ahead.

But actually, OpenContrail works in a similar way – creating GRE tunnels with MPLS running over them. Quite cool.

srx does fragment the packet mode , when you have the fe interface in the packet mode . keep the mtu for the IP packet more than 1500 and mpls mtu need to be set to default (1460) . MX fragment he packet in 13.3 onwards …

I am runniing mpls between srx 110 and mx5 / 80 on gre and does fragment the packets .