But with a few exceptions like your blog (thank you!) nobody, especially not the vendors, focus on the big problem: the edge, where things get a bit out of control.

No wonder: dealing with real-life problems is not sexy, often doesn’t sell, and usually turns out to be a huge morass.

As you mentioned, it is not particular hard to implement reactive fixes: run an active protocol to detect the loop; you mention STP but vendors should have defined a separate L2 probe, the job is important enough to have a separate protocol. Send a probe, detect a packet with a fabric-owned src-MAC on another fabric-edge port, shutdown the particular port(s).

Could we “solve” the problem by adding another STP-like protocol (like VMware beacon frames)? I’m positive there are vendors out there doing something along these lines with proprietary protocols (would appreciate pointers in the comments).

However, I have a nasty feeling that trying to create a standard protocol to solve that challenge would quickly bring us into “turtles all the way down” scenario - who would define what MAC address should be used for that protocol, and what happens when you connect two fabrics together?

The only real solution I see is to:

Admit that data-link layer connects adjacent nodes;

Stop pretending that we can use a technology that was designed to connect nodes to a shared cable when implementing multi-site networking;

Use routing regardless of whether it uses IP prefixes, IP addresses or even MAC addresses to build robust and stable networks.

I eventually will address most of the challenges described in this blog post in How Networking Really Works webinar. I hope to have the first part ready in June 2019.

Related posts by categories

10 comments:

At the ingress node it’s always routing even if the ingress node is a web proxy server (the approach used by large content providers). Whether you do hop-by-hop routing decision or insert path information (label) into the packet is another story.

Well Fabricpath did have a form of summerization -(conversation learning in the core switches)Had Cisco went a little further with the Fabricpath packet construct and use of end node id field they could have been on to something.

But hey, for those 80s protocols isn't this a testament to the hardiness of these 40 year old protocols? They work no matter how they are twisted.

Unfortunately, every single cache-based forwarding scheme I’ve seen eventually falls apart when faced with cache trashing or cache overflow. Conversational learning is no exception, as people who tried to use SVIs on F2 linecards quickly figured out.

As for “they work no matter how they are twisted”, I’m positive anyone experiencing a meltdown after a bridging loop would tend to disagree.

Apologies on my lack of context “they work no matter how they are twisted”. I wasn't referring to meltdowns from twisted use I meant the fact that these protocols have lasted so long, still in use working as intended predictably as designed, still in our toolbox, for we haven't developed(sadly by now) the "be all protocol" over any medium to date that fixes everything we encountered using our old tools to manage the applications above them. But then again I recall some great posts by you and the gang here about topics covering fixing the applications too.

Regarding manufacturer-specific loop detection mechanisms, Nortel/Avaya/Extreme had a protocol called Simple Loop Prevention Protocol (SLPP) that was moderately tunable. I'm pretty far removed from that world now and have no idea if it's still in production or not, but it's at least available on older Nortel and Avaya ERS switches. Michael F. McNamara wrote a blog post about it here:

There are also other loop detection/prevention mechanisms available depending on what topology you have, such as Cisco's Resilient Ethernet Protocol (REP), HP's Rapid Ring Protection Protocol (RRPP), and other vendors' similar offerings built for ring topologies.

The overly simplified explanation of REP (and I believe RRPP functions similarly) is that you specify a link to keep in a blocking state, then all the other switches send eachother keepalives on their ring ports. When a break is detected, an emergency message is sent around the ring to bring that blocked link from before into a forwarding state.

The author

Ivan Pepelnjak (CCIE#1354 Emeritus), Independent Network Architect at ipSpace.net, has been designing and implementing large-scale data communications networks as well as teaching and writing books about advanced internetworking technologies since 1990.