by Geoff Huston, APNIC

Over the past year or so we have seen a heightened level of interest in Internet routing and addressing. Speculation regarding the future role of the Internet raises the possibility of the Internet supporting as many as hundreds of billions of chattering devices. What does such a future imply in terms of the core technologies of the Internet? Consideration of this topic has prompted a critical examination of the architecture of the Internet, including the scaling properties of routing systems, the forms of interdependence between addressing plans and routing, and the roles of addresses within the architecture.

The March 2007 meeting of the IETF, IETF68, saw some further steps in analysing these topics, and many sessions addressed aspects of routing and addressing. This article reports on these sessions, and includes some conjecture as to what lies ahead.

Plenary ROAP – The Plenary Session on Routing and Addressing

The plenary session presented an overview of the topic, looking at the previous initiatives in routing and addressing, as well as providing some perspectives on the current status of work in this area. There are concerns that the technology platform cannot scale by further orders of magnitude without some changes. Also of concern are the scalability of routing, the "transparency" of the network, renumbering questions, provider-based addressing, and service and traffic engineering and routing capabilities—and these concerns are potentially even more relevant and challenging for tomorrow's Internet.

Our routing technology does not localize the external effects of local configuration choices. Far from being a protocol that damps instability, the Border Gateway Protocol (BGP) is a highly effective amplifier of noise components of routing events. So although it is a remarkably useful information-dissemination protocol, the properties of BGP in an ever-more connected world with ever-finer granularity of information raise some questions about its scaling properties. Will the imposed "noise" of the behaviour of the protocol completely swamp the underlying information content? Will we need to deploy disproportionately larger routers to support a larger network? The prospect here is that routing may become far less efficient because as we simultaneously increase the degree of interconnection and the information load, the inability to effectively localize information creates a far greater load on network routing.

In addition to these observations about routing, there is the continuing suspicion that the semantic load of addresses in the Internet architecture, where an address simultaneously conveys the concepts of "who," "where," and "how," contributes to routing load.

To what extent the semantic intent of endpoint identity (or "id") can be separated from the semantic intent of network location and forwarding lookup token (or "loc") is a question of considerable interest. Although the current IP address semantics removes the need to support an explicit mapping operation between identity and location, the cost lies in the inability to support an address plan that is cleanly aligned to network topology, and the inability to cleanly support functions associated with device or network mobility. In the end it is the routing system that carries the consequent load. The questions in this area include an evaluation of the extent to which identity can be separated from location, and the effect of such a measure on the operation of applications. How much of today's Internet architecture would be affected by such a change, and what would be the resultant benefits if this measure were deployed? Are we necessarily looking at a single model of such an id/loc split, or should we think about this scenario in a more general manner with numerous potential id/loc splits?

Obviously this study of routing and addressing, and the related aspects of name space attributes and mapping and binding properties, has a very broad scope. The larger question posed here is whether we can defer resolution of this problem to a comfortably distant future, or whether its effect on the present network is imminent. Are we accelerating toward some form of near-term technical limit that will cause a significant disruptive event within the deployed Internet, and will volume-based networks economics hold or will bigger networks start to experience disproportionate cost bloat—or worse? Is it time to be alarmed?

The unallocated IPv4 address pool will certainly be exhausted in the coming years, but this sense of alarm over routing and addressing is more about whether there are real limits in the near future in the capability to continue to route the Internet within the deployed platform, using the current technologies, and working within current cost-performance relationships irrespective of whether the addresses in the packet headers are 32 or 128 bits in size. There was a strong sense of "Don't panic!" in the plenary presentation, with the relatively confident expectation that BGP will be able to carry the routing load of the Internet over the next 3 to 5 years without the need for major protocol "surgery," and that Moore's Law will continue to ensure that the capacity and speed of hardware will track the anticipated growth rates. Expectations are that the current technologies and cost-performance parameters will continue to prevail in this time frame.

The Internet Engineering Steering Group (IESG) has followed the
Internet Architecture Board's (IAB's) initiative and has begun working with a focus group, the Routing and Addressing Problem Directorate (ROAP), to refine the broad space into many more specific work areas, and has assumed a role of coordination and communication across the related IETF activities.

In addition, because a relatively significant research agenda is posed by such long-term questions, the Routing Research Group of the Internet Research Task Force (IRTF) has been rechartered and, judging by the participation at its most recent meeting, effectively reinvigorated to investigate various approaches to routing that take us well beyond tweaking the existing routing toolset.

Internet ROAP – The Internet Area Meeting

The Internet Area meeting concentrated on aspects of this approach of supporting an identifier/locator split within the architecture of the Internet, and gathering some understanding as to whether this approach would assist with routing scaling. One of the important considerations in this area is working through what could be called boundary conditions of the study. For example, is this matter purely one for protocol stacks within an endpoint, or should distributed approaches that have active elements within the network also be considered? To what extent should a study consider mobility, traffic engineering, Network Address Translation (NAT), and Maximum Transmission Unit (MTU) behaviour? What appears to be clear at the outset is that this network is not a "clean-slate" network, and any approach should be deployable on the existing infrastructure, should use capability negotiation to trigger behaviours so that deployment can be incremental and piecemeal, should allow existing applications and their identity referential models to operate with no changes, and, hopefully, should have a direct benefit to those parties who decide to deploy the technology.

From the routing perspective, the overall desire is to reduce the growth rates of the interdomain routing space. The desired intent is to reduce the amount of information associated with locators so that locators reflect primarily network topology in such a way that the locators can be efficiently aggregated within the routing system that attempts to maintain a highly stable view of the network topology.

More detailed consideration of the implications of disambiguating aspects of identity from those of network location involves many dimensions—including the structure of the spaces—the mapping functions, and the practicalities of any form of deployment of such a technology.

A critical topic appears to be how an identity-mapping function relates to the forwarding-mapping function. Assuming that the existing name spaces remain unaltered, then the resultant framework appears to require distinct "name-to-identifier" and "identifier-to-locator" mappings and a "locator-to-forwarding" mapping. Where these mapping functions should be performed, who should perform them, when they should be performed, the duration of the validity of the outcomes, whether the mapping function outcomes are relative or universal, the scope and level of granularity in time and space of the map elements, the security of these mapping functions, and whether there is a simple operation in each mapping function or multiple operations all remain undefined at this point.

Other questions include whether the mapping is explicit or implicit, what evidence of a previous mapping operation is held in a packet in a visible manner, and what is occluded from further inspection after the mapping operation has been performed. In addition, what level of state is required in each host, and is there true end-to-end transparency—and at what level?

It is likely, at least at this stage of the study, that such a split can have a variety of approaches, both in the intended roles of identifier and location tokens and in their binding. The expectation at this stage of the study is that further ideas will surface, and such ideas will be helpful rather than distracting. It is unclear if a single solution can emerge from this activity, or whether different actors have a sufficiently different set of relative priorities that multiple approaches—each of which expresses different prioritization of functions—are viable longer-term outcomes.

The critical consideration here is that it is unlikely that scaling routing over the longer term to a much larger network is simply a matter of just changing the operation of the routing system itself. Real improvement in this area appears to also require an understanding of the meaning of the objects, or "addresses," that are being passed within the routing system. The motivation for opening up the identifier or locator space within the Internet area appears to be strongly tied to the notion that if you can unburden some of the roles of the addresses used in routing, and treat these routed tokens as unadorned network locality tokens, then you can gain some additional capability in routing.

Routing ROAP – The Routing Area Meeting

The first part of the Routing ROAP session looked at the trends in the routing system over 2005 and 2006. The overall trend appears to be a system that is increasingly densely interconnected, carrying more information elements, each of which expresses finer levels of granularity in reachability. There appears to be two forms of dynamic BGP load: the BGP "supernova" that burst with an intense BGP update load over some weeks and then disappear, and "background radiation" generators that appear to be unstable at a steady update rate for months or even the entire year.

In looking at scaling the BGP routing environment, one response is that of behavioural changes in local instances of BGP that reduce the potential for unnecessary updates to be propagated beyond a "need-to-know-now" radius. Another response is to consider changes to BGP in terms of additional attributes to BGP updates—such as a "withdrawal-at-origin" flag, or selective advertisement of "next best path"—both of which are intended to limit the span of advertised intermediate transitions while the BGP distance vector algorithm converges to a stable state.

It appears that we could improve our understanding of the operational profile of the routing space, looking particularly at the various forms of pathological routing behaviours and comparing these behaviours against the observations of known control points. Such a study may also lead to some more effective models of projections of the size of the routing space in the near- and medium-term future, and allow some level of quantification as to what "scaling of the routing space" actually implies.

The second part of the Routing ROAP session considered the current status of the routing world, updating some of the observations made at the IAB Routing Workshop and outlining some further perspectives on this space. One critical perspective on BGP is the behaviour of BGP under load. It was noted that most BGP implementations use adaptive responses to peer load, so that BGP attempts to ensure that its peer receives only the most current state information when the peer signals that it is not keeping pace with the update rate.

Another critical factor is the nature of "convergence" in BGP. The claim was made that this problem was the biggest, yet least important, problem with BGP. Convergence delays can be mitigated by Graceful Restart, Nonstop Routing, and Fast Reroute. One of the measures that exacerbates convergence is the use of Route Reflectors. The model of information hiding or Route Reflectors is intended to reduce the number of BGP peer sessions and the update load, but the benefits they do achieve are at the cost of slower convergence with a higher message rate during the intermediate-state transitions. Perhaps it is appropriate to consider small-scale changes to BGP behaviour to mitigate the transient BGP update bursts caused by path hunting, including those already mentioned of "withdrawal-at-origin" notification and propagation of backup paths.

The approach advocated here is based on the perspective that BGP is not in danger of imminent collapse, and there is still considerable "headroom" for BGP operation in today's Internet.

More ROAP?

The routing space is a classic example of the commons, where each party can use routing to solve a multitude of business problems.This includes, for example, using routing to perform load balancing of traffic over a set of transit providers, using a "spot market" in Internet transit services, creating differentiated transit offerings using more specific routes and selective advertisements. The ultimate cost of these local efforts in optimising local business outcomes lies in the increasing bloat in the routing system and the consequent escalation in costs across the entire network in supporting the routing system. There is no way to impose administrative controls on the global routing system, nor have we been able to devise an economic model of routing where the incremental costs of local routing decisions are visible to the originator as true economic costs for the business, and the benefit of a conservative and prudent use of the routing system reaps economic dividends in terms of relatively lower costs for the business.

Like the commons, there are no effective feedback mechanisms to impose constraint on actors in the routing space. Also, like the commons, there is the distinct risk that the cumulative effect of local actions in routing creates a situation that pushes the routing system, either as a whole or in various locales, into a nonfunctioning state.

Whether it needs a sense of urgency to motivate the work, or a sense that there can and should be a better way to plan a future than crude crisis management, the underlying observation is that the routing and address world is fundamental to tomorrow's Internet. Unless we make a concerted effort to understand the various interdependencies and feedback systems that exist in the current environment, and understand the interdependences that exist between network behaviours and routing and addressing models, then I'm afraid that the true potential of the Internet will always lie within our vision—but frustratingly just beyond our grasp.

Further Reading

Following are references to further material on this topic, as presented at IETF68:

GEOFF HUSTON holds a B.Sc. and a M.Sc. from the Australian National University. He has been closely involved with the development of the Internet for many years, particularly within Australia, where he was responsible for the initial build of the Internet within the Australian academic and research sector. The author of numerous Internet-related books, he is currently the Chief Scientist at APNIC, the Regional Internet Registry serving the Asia Pacific region. He was a member of the Internet Architecture Board from 1999 until 2005, and served on the Board of the Internet Society from 1992 until 2001. E-mail: gih@apnic.net