There’s no 1-tier network… Do you have a problem with that?

I’m not a car person and I don’t worry too much about what’s under the hood. That means that I’m just a car user, I only want to turn the ignition key and drive. In the Data Center world, the server team is typically a user of the network. Server guys don’t want to know how the network is implemented. They just want their VLANs to extend to the whole network so that they can connect their devices with no constraint, without having to worry about high availability, risk containment, link provisioning… network stuff. That’s precisely what FabricPath is designed to offer them: a network that looks like a single switch, the simplest networking entity. This “Fabric” offers efficient any-to-any connectivity with high bandwidth and low latency, all without having to understand how it works.

Figure 1

Of course, this user perspective is an abstraction. The following Figure 2 represents an example of the physical topology of the network, a Clos fabric, typical in Data Center environments. Note that this could just as well be a ring, a star, or even a network distributed across two sites. FabricPath turns an arbitrary topology into a Fabric and does not lock you into a particular model.

Figure 2

The network view is critical to the network admin. How could you manage this Fabric without exposing those links? How would you troubleshoot errors on a port? How would you enforce a security policy that would constraint traffic to a subset of the links? How would you provision bandwidth and prioritize traffic? However nice the dashboard might be, the mechanic cannot maintain a car without looking under the hood. Based on an open standard (TRILL), FabricPath provides full access to all the networking elements.

Finally, let’s get our hands dirty examining how traffic is switched in a network. In Figure 3, a host on switch 2 sends a frame to a peer on switch 4. When this frame reaches switch 1, a switching decision must be taken. This decision *has* to be made based on some information in the frame, there is no way around this process. Switch 1 extracts some bits from the frame, does a lookup on an internal table, and the result determines the exit port.

Figure 3: Classical Ethernet switching

Now, what are those bits, where are they coming from? In a typical Layer 2 network, switches do a lookup on the destination mac address of the frame. Of course, some switches can perform more intelligent things and are capable of looking even further into the frame. However, if this deeper lookup does not really impact latency, it requires additional hardware resources and complex provisioning, making the solution more expensive.

Figure 4: TRILL switching in FabricPath

Figure 4 illustrates how TRILL (and thus FabricPath) operates. When the frame enters the Fabric on switch 2, a full mac address lookup is performed. The result is “4”, a value that identifies a destination switch across the Fabric. This result is added to the frame when it is forwarded by switch 2 to switch 1. Instead of doing a full mac address lookup, switch 1 now only needs to parse this smaller switch ID (here 4) and do a lookup in an smaller internal routing table in order to identify the exit link (here L3). So the full processing is only done once at the edge, the devices inside the Fabric like switch 1 don’t even implement a mac address table.

So let’s summarize some of important properties of FabricPath described here:

FabricPath does not perform any mac address lookup inside the Fabric. I could then pretend that FabricPath is performing a single switching decision at the edge.

With the above and because FabricPath looks like a single switch from the perspective of its users, I could say that FabricPath is a single switch, and that the result is a 1-tier network.

So if this Fabric is in fact a switch, performing a single switching decision, I could finally suggest that with FabricPath, every port is “directly” connected to all the others.

The messaging about 1-tier networks circulating in the industry is based on those above claims that could be applied to FabricPath. Do you believe in 1-tier networks now you see where they’re coming from? As a network engineer, I consider it’s just marketing buzz. Hopefully, having an opinion on this is not important. The technical aspects described here are real, and FabricPath’s performance and availability is no hype. That’s all that matters;-)

8 Comments.

Hi Francois, what you described is how TRILL is supposed to work, right? Within the TRILL network, forwarding is determined at the edge, and the destination is the peer edge switch which then decapsulates the original frame and forwards to the next hop.
What does FabricPath do above and beyond that?

Hi Mike,
Yes, everything described here is also applicable to TRILL. Maybe I should clarify that. The post was not to contrast FabricPath with TRILL, but rather compare FabricPath/TRILL to other claims for 1-tier Fabric spread in the industry. The technical fundation for those claims is basically the same as what I've exposed here. Only the marketing is very different;-)
Thanks,
Francois

Hi Francois,
At this level this is also the same way that Brocade Ethernet Fabric's work as it is also TRILL based.
Of course this is very different from Juniper's Ethernet Fabric. Was this the point you are making?
Thanks
Michael.

Hi Michael,
I'm just using classical Ethernet and TRILL as generic example here. The box in the position of switch 1 has to extract some bits from the frame in order to do a switching decision. For me, that's a switching tier. That's going to be the same for any platform, past, present or future like Juniper's upcoming fabric.
My claims were just showing how you could put a marketing curtain to hide this switching tier. It's not because you don't see it that it's not there;-)
Regards,
Francois

Well, not so much, I've basically worked on pure Cisco all my career. But Juniper's upcoming Qfabric, fabric.. blah marketing term seems more comparable to the Nexus5k/7k and it's extender model. Single point of management, the difference being that the edge devices in the Juniper case can actually do distributed forwarding.. which still baffles me why Cisco didn't include this in their 2k extender. But that's besides the point and topic for different conversation. TRILL/Fabricpath doesn't seem like the right comparison model here.

The FEX is really a passive device. That's why we can consider it as a part of the upstream switch. Anything that is received on the host ports is sent unconditionally on the uplinks. No parsing, no lookup there: there's no switching decison. If the FEX was taking a switching decision locally, that would be a two-tier design (even if configuration is centralized) and that would be fine too;-)

All this two-tier, fabric, marketing speak really doesn't mean anything to engineers. FEX should be able to do local-switching, it's a line-card right? even the age old 6500 has DFCs in it's linecards. The whole two-tier, one-tier system is that "marketing" terms, the words fabric, and so forth keep being thrown in there too. I'm not arguing that, all I'm saying is the right comparison should've been made to what you were trying to get at, and frankly TRILL and Fabricpath was not it. :)

Some of the individuals posting to this site, including the moderators, work for Cisco Systems. Opinions expressed here and in any corresponding comments are the personal opinions of the original authors, not of Cisco. The content is provided for informational purposes only and is not meant to be an endorsement or representation by Cisco or any other party. This site is available to the public. No information you consider confidential should be posted to this site. By posting you agree to be solely responsible for the content of all information you contribute, link to, or otherwise upload to the Website and release Cisco from any liability related to your use of the Website. You also grant to Cisco a worldwide, perpetual, irrevocable, royalty-free and fully-paid, transferable (including rights to sublicense) right to exercise all copyright, publicity, and moral rights with respect to any original content you provide. The comments are moderated. Comments will appear as soon as they are approved by the moderator.