Abstractions? Yes. But to what end?

Abstractions? Yes. But to what end?2013-07-102013-07-10https://www.plexxi.com/wp-content/uploads/2018/04/plexxi-logo-rgb.pngPlexxihttps://www.plexxi.com/wp-content/uploads/2018/04/plexxi-logo-rgb.png200px200px

Amidst all of the SDN talk, offshoot industry dialogues have sprouted up around a couple of topics I want to cover today: abstractions and network programmability. And depending on the audience, the reactions can be somewhat visceral for each of these.

Most of the talk about abstraction has a uniquely networking-centric view. The dialogue typically follows a set of familiar talking points:

The network has grown over the past few decades through layering of protocol after protocol.

This has created an environment that is entirely too complex to effectively architect and manage.

The burden of this complexity falls to operational staff.

And with uptime as a critical requirement for all networks, it means change comes at a snail's pace.

The network guys aren't making changes fast enough and even when they do, they make too many mistakes.

The answer is to abstract out the complexity.

I actually agree with the need for abstraction, but I think the starting point for the abstractions needs to be carefully considered. When I talk to folks about abstractions, I frequently get questions like "Oh, you mean policy?" Everyone knows there is an edge policy problem. Much of the management complexity is due to how we manage and distribute policies, particularly in heterogeneous, multi-vendor environments.

But when all is said and done, if all we have done is create a new way of specifying VLANs and ACLs, we will have missed horribly as an industry. The issue is not that the semantics are too complex; it is that a provisioning model based on fine-grained control of device behavior is prone to human error because of the sheer volume of knobs operators must concurrently manage. And when behavior requires such fine-grained tuning, getting functional equivalence out of different devices can be difficult to impossible.

The DevOps tools have certainly been a move in the right direction. Tools like Chef, Ansible, Salt, and Puppet, provide vendor-agnostic network abstractions in a way that simplifies provisioning from beyond the network wall. These are great tools, particularly for infrastructure-savvy people. But I think that we have collectively limited the advances we can make… and we have done so somewhat artificially.

The starting objective seems to be to make provisioning simpler. The goal is to remove pain if you will. But is the best that we can hope for a pain-free experience? Is there not something more we can achieve if we get these abstractions correct?

If you believe that the end goal is fixing a problem, then I think having vendor and device-agnostic ways of specifying VLANs and the like is actually the right approach. But fundamentally, I think the long-term objective ought to be an exchange of information between the network and the applications and infrastructure using the network.

And this gets to the second aspect I wanted to cover: network programmability. When I have discussions about network programmability with typical IT shops (read: not huge software shops), the reaction is fairly universal: I don't want application wonks touching my network! Does network programmability create a means through which non-networking folks can impact the network? What are the change control implications? What are the security considerations? What happens when someone who doesn't know networking gets their hands into the guts of the network? This all sounds scary.

The goals of network programmability are clear: use information from outside the network to improve network performance in support of application requirements. This is absolutely the right objective. Let me say that again: THIS IS ABSOLUTELY THE RIGHT OBJECTIVE. Indeed, if the network understood better what applications required to deliver the experience users expect, the network should be able to better deliver that experience. Understanding which applications are bandwidth-heavy or latency-sensitive allows the network to make more intelligent pathing decisions.

But what is the right way for information to be exchanged?

Network programmability – the notion that applications have deep, programmatic access to the networking gear – might not be a necessary or even helpful approach for the majority of users. Application guys know applications. Asking them to express their requirements in networking terms via the wrong abstractions, or asking them to program the network via deep access forces them to work in an area that they are by design not experts in.

What we want instead is a way for applications to specify their requirements in application terms, a way to express their intent without needing to translate that intent into the language of a different infrastructure. The application guys might not care how their requirement of PCI compliance translates into VLAN provisioning, path isolation, or any number of other networking constructs. They don't want or need programmatic access to the device, nor do they want to use a vernacular with which they are less familiar. It ought to be as easy as tagging an application with PCI.

Ultimately, the application teams don't want to manage the network any more than the network teams want them mucking with the network. But if the abstractions are just a rethinking of policy or if getting network performance requires deep programmatic access, I suspect both camps will wind up disappointed.

Where we end up will obviously depend on where we start. And my fear is that we are aiming too low. If we continue to view the abstraction dialogue and the SDN trend more broadly as a means to alleviate pain, we will miss out on some of the more impactful things that might otherwise be delivered.

Mike, I cannot agree with you more. It’s like you are describing the vision of our company and product. The thing you got so right is that by abstraction you can build and operate more complex networks and services. Operators can manage those with easy tasks and application guys can kick of predefined tasks through a northbound API. Network programmability and abstractions are not the goal, they are merely the means to a goal. To deliver faster, more reliable services/applications through the network.

Let me challenge you a bit (but mostly for fun). What would you say to renaming the northbound API to something else? The fact that the most notable thing about that API is the direction of the arrow in PPT presentations is unsettling for me. And that we see it as Northbound reinforces our view that the network is the center of the universe. What if that API was named based on what role it served? Is that API really more about applications expressing intent? Maybe it’s an API for networks to share information?

To be honest with you, we didn’t call it the northbound API until the SDN hype. Before that we used to call it the exchange-interface. Not the sexiest name, very functional, just like plain API. To come up with a name based on the role is not as easy as it sounds. Actually, I even want to push this a bit further. An API is based on a one on one or multiple to one relationship. Applications like these are, together with OSS/BSS, CMDB, monitoring etc., part of a bigger service architecture. That’s why I think it is better to connect these applications to a service bus. Based on the type of information exchanged it is either north- or southbound, but the most important thing is that certain information exists only once. In retrospect exchange interface is still not far off.

This actually is a good conversation to have. Particularly since how I think you are meaning services is different from how most people on the network vendor side mean them. OSS/BSS integration is painful in part because vendors don’t create services – they create service building blocks that have to be tied together (typically in some order) to create an actual billable service. That service is then applied to various traffic (could be defined by application, user, destination, whatever). Ideally, this mythical unnamed interface (maybe exchange interface) ought to simply tag network conversations with the desired services (or a container that includes some set of services if another level of indirection is really required). Services is probably the most abused term in networking (aside from maybe virtualization).

Anyway, great commentary. Has me thinking, which is always dangerous as I am in marketing now. Product management, beware. Nobody tell Mat Mathews.

This post is spot on. Why should a developer have to worry about VLANs, IP
addresses, firewall choke points, load balancer pools before they can even
get started? And why should an infrastructure admin have to rack and stack
more boxes or configure more ports to meet end user demands? You have attack the problem from both ends – end user consumption and network
programmability. –Gargi Mitra Keeling, VMware, Inc.