Playing chicken: ISP depeering a high-school lovers’ quarrel

The peering dispute between large ISPs Cogent and Telia has disrupted Internet …

On March 14, the first reports appeared about loss of connectivity between Cogent and Telia. Cogent is a large "backbone" ISP in the US and Telia is a large European ISP with roots in Scandinavia. Random outages happen all the time on the Internet, but this outage was anything but random. In a message to its customers, Telia unequivocally placed the blame with Cogent. "Cogent has decided not to exchange traffic directly with TeliaSonera's AS 1299 or indirectly with AS 1299 through a third-party provider," Telia told its customers. "As a result, Cogent has partitioned the Internet and disrupted the flow of traffic between Cogent and TeliaSonera customers." In global Internet routing, Telia's network is identified by Autonomous System number 1299, while Cogent has 174.

But David Schaeffer, CEO of Cogent Communications, told the AP that Telia continuously breached the terms of the peering contract that allowed the two companies to exchange traffic directly on a settlement-free basis. Depeering because of differences of opinion about who benefits most from exchanging traffic for free are fairly common, but usually don't result in completely broken connectivity as happened here. There are only two handfuls of very large "tier one" ISPs that exclusively depend on peering to exchange traffic between them. The lower-tier ISPs buy wholesale transit capacity from a bigger ISP to reach destinations for which they don't have peering arrangements.

The peering between Cogent and Telia was disconnected on March 13, but traffic kept flowing between the two networks through Telia's transit ISPs. A day later, this route, too, was cut off, something of a scorched earth strategy on the part of the perpetrator. In this game of "he said, she said," the burden of proof must be on Cogent, which has a long history of peering disagreements. These scuffles are often with European networks, but a high-profile dust-up in 2005 involved tier-1 network Level3.

In the telephony world, there are regulations about interconnection, but depending on jurisdiction, it can be unclear to what degree such regulations apply to Internet peering. Typically, one party will blink before the other heads to the courts for redress. Typically, significant depeering without rerouting of some kind doesn't last long.

One commentator compares depeering with a break up, but let me extend the analogy a bit further. Peering is a lot like dating. The small ISPs who operate in a small region are very eager to court the big ones with extensive networks, because then the big network carries the traffic most of the way. The big networks, on the other hand, know how pretty they are, and they tend to sport the "I'm so hot" attitude. As a rule, big networks won't enter into peering agreements with (much) smaller networks. Instead, all they care about is hooking up with one another—but only under the right conditions.

Unless you go out of your way to make things happen differently, Internet routing follows the early exit or "hot potato" model: when traffic is destined for another network, it gets handed over to that network as quickly as possible by sending it to the closest interconnect location. Late exit or "cold potato" routing is more difficult, because the traffic may inadvertently be carried to the wrong place before it's handed off to the other network. And it doesn't make much sense to do late exit when everyone does early exit—when you do, you have to carry traffic most of the way in both directions.

The consequence of early exit routing is that the receiving network must carry inbound traffic most of the way. A network that has a lot of content customers has more outbound than inbound traffic, so it will benefit more from a peering arrangement. Content-rich ISPs are big players in the peering world. To avoid giving away the milk for free, peering contracts tend to have a clause that requires a balance in inbound and outbound traffic. A network may thus be tempted to take away its affections from a long-time smaller peer to create a balanced mix of incoming and outgoing traffic for an a prospective new peer.

Another common requirement is that traffic is exchanged in a certain number of locations, and that the connections are of a certain bandwidth. However, it's not unheard of for one peer to be unwilling to upgrade a congested peering link. DSL and especially cable networks often have limited bandwidth in the last mile. If a peering link with another consumer access ISP is saturated because of peer-to-peer traffic, it's really not in the ISP's interest to remove that bottleneck so more traffic can hit that last mile infrastructure, drowning out more valuable traffic in the process. Why let your significant other spend lots of money on that big HDTV set when it only means that he'll have his friends over to watch sports more often?

As with any break up, it's the kids who suffer. But unlike kids of divorced parents, the (larger) customers of depeered ISPs can take charge of their own destiny by adopting additional parents. By having connections to two or more ISPs, end-users insulate themselves from these spats for the most part. If traffic won't flow to Cogent over Telia, simply send it over Abovenet or Global Crossing. This "multihoming" works extremely well, but it does require that customers have an address block of their own and run the BGP protocol that ISPs use to exchange routing information themselves. This puts multihoming well outside the reach of a typical broadband user, as it requires expensive routers, ARIN fees, and a monthly bandwidth bill that makes it worth the engineering resources to set all of this up for the ISPs in question. This means that the average consumer or small business user is left with little recourse other than mustering up the requisite amount of patience... or they can switch ISPs.

Iljitsch van Beijnum / Iljitsch is a contributing writer at Ars Technica, where he contributes articles about network protocols as well as Apple topics. He is currently finishing his Ph.D work at the telematics department at Universidad Carlos III de Madrid (UC3M) in Spain.