Category Archives: Cisco

This post follows on from part 4, but this time we’ll be configuring a Layer-2 Ethernet to Ethernet MPLS VPN between the 2 CEs.

MPLS L2VPN

I’m going to configure a Martini Layer 2 VPN. Martini uses LDP to signal and setup the VPN across the MPLS network.

With MPLS L2 VPNs there will also be a minimum of two labels. The top label being the transport label and the bottom label being the VC label. The transport label will be swapped hop by hop through the MPLS network. The VC label is used by the egress PE router to identify the Virtual Cirtcuit that the incoming packet relates to.

On an Ethernet segment, the frame would look like this for a payload of 1500 bytes.

L2 Header – 14 bytes. This is the Ethernet header for the frame. 14 bytes, or 18 bytes if the interface is an 802.1q trunk.

MPLS Label – 4 bytes. The transport label.

VC Label – 4 bytes. The VC ID.

Control Word – 4 bytes. This is an optional field only required when transporting FR or ATM, carrying additional L2 protocol information.

L2 Data – 1514 – 1518 bytes. The encapsulated Layer-2 frame that is being transmitted across the MPLS network. In this case 1500 bytes of data, plus the L2 header (an additional 4 bytes would be added if the VPN interface is a trunk).

Well this means that we will be putting a minimum of 1540 bytes (14 +4+4 + 4+1514) of data on the wire if there was 1500 bytes of customer data in the encapsulated L2 frame. If the customer interface is a trunk and the SP interface is a trunk, then we are up to 1548 bytes on the wire across the P network.

Lab

For this lab, I’ll be using the topology below. The base configurations are using OSPF as the routing protocol and LDP to exchange transport labels.

Software revisions are as follows

CE1, CE2, P, PE1: IOS (Cisco 7200 12.4(24)T)

PE2: Junos (Firefly 12.1X46)

The base configs are similar to part 3, using OSPF as the IGP and LDP to signal transport labels, so I’ll jump straight in to the Martini VPN config.

PE1

The configuration could not be simpler. The VCID must match at both sides and is set to 12.

The obvious difference compared to L3 MPLS VPN is that the provider network has no involvement in customer routing.

What’s the maximum ping we can get from CE1 to CE2? The provider core is set to 1500 bytes on the PE-P-PE interfaces. It should be 1474 right? 1500 – 14 bytes of L2 headers – 2 labels – 1 control word. And it is.

Now how about I make the CE1-CE2 link dot1q. I’ve not changed anything on the PE routers, just enabled a 802.1q tagged interface using on each CE. As there will be an extra 4 bytes of overhead, the maximum ping size will drop to 1470.

Share this:

I had only intended to do 3 parts to this series, but I can’t really stop at part 3 (MPLS) without posting about how MPLS Layer 3 VPN affects MTU. In this post we’ll build a simple MPLS VPN network and our goal is to transmit 1500 bytes of IP data between the CEs.

MPLS VPN

With MPLS L3 VPNs there will be two labels. The top label being the transport label and the bottom label being the VPN label. The transport label will be swapped hop by hop through the MPLS network. The VPN label is used by the egress PE router to identify the VPN/VRF that the incoming packet relates to. Assuming PHP is being used, the transport label will be removed by the router before the egress PE (in this lab the “P” router), leaving only the VPN label in the stack.

Lab

For this lab, I’ll be using the topology below. The base configurations are using OSPF as the routing protocol and LDP to exchange transport labels. A full mesh of MP-BGP VPNv4 sessions will be configured between the PE routers to exchange VPN labels. Software revisions are as follows

CE1, CE2, CE3, PE3: IOS (Cisco 7200 12.4(24)T)

PE1: IOS-XR (IOS-XRv 5.1.1)

PE2: Junos (Firefly 12.1X46)

As with previous posts in this series I’m going to show what needs to be done to enable the required MTU. I’ll give some commentary on the MPLS config but will save the detailed analysis of MPLS control/data plane for another time.

PE1

PE1 is running IOS-XR. The relevant parts of the base config are as below:

The routing instance defines the VRF, the route-distinguisher and route-targets to import. All the OSPF configuration takes place here as well. I have an export policy configured so that the BGP VRF routes are exported in to OSPF and distributed onward to CE2.

OK, we are getting an ICMP unreachable back from PE1 exactly as expected.

We know that MPLS VPNs adds 2 labels to the stack, and therefore adds an extra 8 bytes of overhead, so all we need to do is increase the interface MTU to allow this extra data. From previous posts in this series we know how the different software does things.

On PE1, the IOS-XR router, the interface MTU will be increased to 1522 (1514 + 8), and the MPLS MTU will be set to 1508.

On P, the interface MTU will be increased to 1508 and the MPLS MTU will be set to 1508. The same config will be required on PE3 as both are running IOS.

On PE2, the Junos router, the interface MTU will be increased to 1522 (1514 + 8), and the MPLS MTU will be set to 1508.

Share this:

This week I took the CCDE written to recertify my CCIEs. That chalks up 8 years now of being a CCIE.

Firstly, why did I recert? Ethan Banks recently wrote a great post “Is the CCIE Certification Losing value?” in which Ethan talks about his upcoming re-certification deadline, and if he’ll take the exam or not.

I decided to recertify for a few simple reasons:

It took a shed load of time and effort to become a dual CCIE. So much went in to achieving the R&S and SP, I’m not ready to let them go just yet.

In comparison to the actual lab, the written is a fairly simple way to keep the certs alive. I don’t necessarily agree with being able to recertify any CCIE by taking a single expert level exam, but as I don’t have much spare time, it sure makes life easier!

Having the CCIE is a meaningful and simple way for clients to gauge my skill level, to a point of course. As with many CCIEs, my experience extends way beyond working on the CLI (which is one of the reasons I originally became interested in the CCDE).

In my opinion, the CCIE is valuable and isn’t going away any time soon. Whilst it has a place I’ll aim to keep the badges.

I like learning.

One thing for sure though – if you are working on the CCIE right now, great, but when you’ve finished I think it’s important to learn more than just networking. How about Python? Do you know virtualisation? What do you know about SDN? etc

352-001

Now on to the exam itself. I took the CCDE v1 written exam at the last recert, I thought I might take the CCDE practical. I still might, who knows, but this time I wanted to check out the v2 exam.

The 1st thing to note about the exam blueprint is that it’s huge. You can be tested on pretty much anything – networking, wireless, datacentre, standards, process, etc. There is no formal training requirement for this exam, just a load of books to read. Check out the list. Over the years I’ve read most of the books on this list already cover to cover. Some more than once! Having a Safari books subscription is a great investment. If you don’t have access, I’d recommend it – particularly whilst you are working up to taking an exam. This time around I re-read my notes from last time, picked out the sections from the blueprint that I wanted to improve on, and went to town on the book list, CCO etc.

The exam itself is quite challenging – it is very different to other Cisco exams that I’ve taken in the past. Whilst it might not always go in to the depth and nuts and bolts that you’d see right the way though for example, the CCIE SP written, there’s a lot to the questions and you need to be prepared to take time to think clearly through several different designs, technologies & concepts before finding the best answer and clicking that next button.

You can go from R&S topics, to SP, to Datacentre! Everything is in there. But, in my opinion, it is a fair exam with a good distribution of the topics. Most of the questions were quite clear, but as can be expected there were several that I left comments on – some of the questions were just crazy, but that’s exams for you…

Overall I enjoyed the exam, and that’s the CCIE re-certification done now for another 2 years. Next time around it will be 10 years! That’s flown by.

If you are taking the CCIE or CCDE exams right now, or any vendor expert level certification for that matter, best of luck with the journey. It’s hard work, but most importantly make sure you enjoy the ride!

No rest for me yet though, now I’m working through the Juniper certs…

Share this:

Part 1 of this series focussed on the interface MTU configuration, looking how different vendors implement the setting. Some include the layer 2 headers, some don’t. Part 2 looked at Jumbo Frames, IP MTU and OSPF. In this post we’ll build a simple 4 node MPLS network and check out how the default MTU settings affect the transmission of data. We’ll also enable a couple of 802.1q tagged interfaces. The configuration will be adjusted accordingly to enable the transmission of 2000 bytes of IP payload across the MPLS core.

802.1q Frame

First of all a quick refresher on what an Ethernet frame looks like if it contains an 802.1q tag in the header. 802.1q does not encapsulate the Ethernet frame, simply a new 4 byte header is inserted between the source MAC address and the Ethertype. Now with 1500 bytes payload, the frame size has increased to 1522 bytes (1518 not counting the FCS). The 802.1q tag consists of the following fields:

TPID: Tag Protocol ID. A 16-bit field with a value set to 0x8100 to identify the frame as IEEE 802.1q tagged

Priority. A 3 bit field which refers to the IEEE 802.1p priority. i.e. class of service

DEI: Drop eligible indicator. Used to indicated if frames can be dropped.

VID: VLAN ID. A 12-bit field specifying the VLAN.

Multiple 802.1q tags can be present in the frame. This is known as IEEE 802.1ad Q-inQ. More on this another time.

MPLS Frame

Also let’s take a look at an Ethernet Frame containing an MPLS header. The 32-bit MPLS label is inserted between the L2 header and the protocol data. This is why it’s sometimes called a shim header. The ethertype is set to 0x8847 to identify the payload as MPLS. The MPLS label contains the following fields:

Label. The label itself – 20 bits.

EXP. A 3-bit field used for QoS markings.

S. 1-bit to represent if a label is the last in the stack.

TTL. An 8-bit time to live field.

As the S bit implies, there can be a stack of labels. More on this another time.

Lab

For this lab, I’ll be using the topology below. We’ll start with a base IP/OSPF configuration and add MPLS. MTUs will be adjusted to enable a 2000 byte MPLS payload to be transmitted across the topology. Software revisions are as follows

On the tagged interface to VLAN23, Junos has also increased the MTU to 1518 to accommodate the VLAN tag and enable 1500 bytes of protocol data. Note, if you configure the interface MTU manually or XR or Junos you’d still need to allow for the dot1q tags.

At this point the biggest ping that we’ll be able to get from R4 to R1’s loopback address 1.1.1.1 is going to be 1472, representing 1500 byes of IP data and 1514 on the wire (1472 data + 8 ICMP header + 20 IP header + 14 Ethernet).

MPLS

MPLS and will now be enabled across the topology. LSPs will be RSVP signalled and I’ll demonstrate how the MTU settings need to be updated. As this post is focussing on the approach different vendors take to MTU, I’ll skip the MPLS detail and save it for another time, but the relevant MPLS config is below.

On R1 I’ve enabled a Tunnel interface and statically routed IP traffic to R4’s Loopback via this MPLS TE tunnel. LDP has not been enabled and doesn’t need to be.

R4. I’ve created a dynamic label switched path to R1. This path gets installed in table inet.3 but as inet.3 is only used for BGP next hops by default, my traffic test from R4 to 1.1.1.1 won’t be labelled. I want to keep this lab simple, so I’ve used traffic-engineering mpls-forwarding to ensure that the LSP to 1.1.1.1 is placed in to table inet.0

No, that maximum that could be transmitted in both directions was 1488 bytes of IP data. Don’t forget for Junos, 28 bytes of headers are added to the ping size.

By default Junos derives the MPLS MTU from the Interface settings and makes allowance for 3 labels in the stack, so with a 1500 byte interface MTU, the MPLS MTU would be set to 1488. This is why we can only ping with 1488 bytes of protocol data.

On both of the Junos routers I’m going to go ahead and set the Interface MTU to 9192, but since my goal is to test an MPLS switched payload of 2000 bytes IP data, I’ll set the MPLS MTU and leave the IP MTU at 1500 bytes. Note if the MPLS MTU is user configured then the configured value includes the labels. I’ll set the MPLS MTU to 2004 to allow the 2000 bytes of data and for this test I only need to make allowance for 1 label.

I love how Junos allows a regexp to be used pretty much however I feel like using it.

At this point the maximum ping size from both sides of the network is currently 1496 bytes. This is as expected because we have 1 label in the stack and the Cisco routers are still set to 1500 bytes on the Interface MTU.

Let’s have a look what’s going on with the interface MTU parameters on the Ciscos.

We’re being told that the packet is too big at R2. Why’s this? Well remember that MPLS will pop the label at the hop before the destination. This is known as penultimate hop popping. So the data between R2 is and R1 is transmitted as IP, not MPLS. As I left the MTU at 1500 for IP data, the echo request cannot be transmitted.

I actually did this on purpose by changing the config on R1 so that the LSPs were implicit null signalled. If I go ahead and change the signalling to explicit null, the LSP will then be labelled end to end and the ping will work.

Excellent! The capture below was taken on R2s interface to R1, and you can see the 2018 bytes on the wire (1972 data + 8 ICMP + 20 IP + 4 MPLS + 14 Ethernet). Notice Label 0 is used for Explicit Null on IPv4.

Summary

In this post we focussed on MPLS MTU and how to use MPLS MTU to enable a payload >1500 bytes. I’ve demonstrated the the Interface MTU, MPLS MTU and IP MTU can be set to totally different values.

Why does any of this matter? In a provider core you need to be able to transport your customers full size frames across your network. You can guarantee that customer data will be at least 1500 bytes, but depending on the service specification that you want to sell, could be as large as 9000.

I’ve introduced quite a lot of MPLS terms in this lab. In a later post I will go through this topology again in detail, and talk through the MPLS specifics.

Share this:

Part 1 of this series focussed on the interface MTU configuration, looking how different vendors implement the setting. Some include the layer 2 headers, some don’t. Part 3 will add MPLS MTU to the mix.

This post looks at Jumbo Frames, IP MTU and we’ll also introduce OSPF to the mix. OSPF exchanges IP MTU information in DBD packets when forming a neighbor adjacency and will detect any MTU mismatch, so if the MTU settings are wrong or mismatched, we won’t fully establish the adjacency. Of course, this feature can be turned off, but following the OSPF Version 2 specification (RFC 2328):

If the Interface MTU field in the Database Description packet
indicates an IP datagram size that is larger than the router can
accept on the receiving interface without fragmentation, the
Database Description packet is rejected.

Jumbo Frames

Why would anyone actually want to increase the MTU size beyond 1500? Well, back in the day, larger packets were desirable because they resulted in less overhead on the server – fewer CPU interrupts, fewer CPU cycles wasted etc. Today with Large Segment Offload (LSO) etc the performance increase might not be what you’d expect. Take a look here for more info. As always – implement something that is suitable for your environment, and test before you do. Anyway, you are not here to debate the use of Jumbo frames or otherwise, so let’s crack on.

IP MTU

As with Ethernet frames, the protocol MTU can be changed for IP packets. The accepted “standard” payload for a Jumbo Frame is 9000 bytes (i.e. an IP MTU set to 9000).

In this post, the interface MTU will be increased to the maximum supported by the interface hardware, but for the purposes of this post and to demonstrate that the interface MTU and IP MTU can be different, we will set the IP MTU to a consistent value of 2000 bytes.

Topology

The same virtual topology will be used as Part 1.

Software revisions are as follows

IOS (Cisco 7200 12.4(24)T)

IOS-XE (CSR1000V 15.4(1)S)

IOS-XR (IOS-XRv 5.1.1)

Junos (12.3R5.7)

Junos (Firefly 12.1X46)

IOS/IOS-XE

Our Interface config is below, the interface MTU has been changed to 9216, and the protocol MTU to 2000. From our tests earlier, we know what this means that the maximum IP payload is 2000 bytes, which would result in 2014 bytes being put on the wire including the L2 headers.

Pretty clear that there is an MTU problem. Below I’ve set the Interface MTU to 9000 – remember IOS-XR includes the L2 headers in the Interface MTU, so the maximum encapsulated data on this wire would be 8986.

UPDATE 23/12/15: I’m setting the interface MTU to a different value to the IOS router’s MTU setting, to show that it’s the IP MTU that is the important setting and must match for two OSPF routers to establish an adjacency.

Now for the ping, remember that the Junos ping size excludes the ICMP (8 bytes) and IP (20 bytes) headers , so we’ll be expecting the maximum working ping size to be 1972 bytes, for 2000 bytes of protocol data and 2014 bytes on the wire.

NX-OS

I don’t have a Nexus in my lab, but for completeness below is the config for updating the MTU on NX-OS.

On the 7k the system will be enabled for Jumbo by default, if you need to change this value it’s done like this

switch(config)#system jumbomtu 9216

Make sure you check the Interfaces and Vlan interfaces have the correct MTU. That’s done with the interface command “mtu X”. For Layer 2 interfaces, configure either the default MTU size (1500 bytes) or up to the system jumbo MTU size.

On the 5k it’s done a bit differently – in a QoS policy map! For NX-OS >4.1