Thoughts on the “Maturing APIs” panel at Gluecon 2011

The original topic of the session was pretty open but we’d decided before hand to try to focus on how APIs were maturing as a part of Web Infrastructure and how far they still had to go. In the three short presentations we touched upon a whole bunch of interesting issues and got some great audience questions afterwards. I won’t manage to do it all justice but a few points stood out for me:

William focused on the key issues of trust and reliability – APIs are critical parts of other people’s applications and so engineering them correctly is critical. Beyond this however there is the question of meeting SLAs both from a operational perspective (yep it’s up) and functioning correctly (yep, it’s doing the right thing) – which is where trust and reliability issues really begin to kick in. As the number of APIs grows and the space matures, it will get increasingly hard to figure out which APIs actually deliver and what tasks they can be trusted for. OpusGrid’s new TheRightAPI.com is designed to help in this respect – monitoring public APIs and providing performance and trust data – something that will be increasingly critical as APIs become central to the Web.

Jeff went deeper and picked out some of the emerging pathologies in API usage – especially as usage scales up. Architectures that seem simple (keep it simple stupid right?) quickly begin to break at scale. In particular setups which require APIs to be polled incessantly on a per account, per user basis for status change. This might work fine for small volumes but generates huge volumes of calls when applied across many accounts. Patterns like webhooks could replace a lot of polling in many cases and should find wider adoption (although they are also not a silver buller). This also lead to good debate on change management of APIs – there is an innate tension between API provider’s desire to innovate and differentiate themselves from the competition and the need to keep interfaces as stable as possible. Even worse, innovating often means trying to push APIs into areas not covered by comparable (competing) APIs – great for differentiation, nightmarish for developers.

I tried to give an overview of how we see the Web as a whole changing as APIs become more prevalent (see slides on slideshare). In particular the idea that APIs are beginning to enable a strong separation of concerns between Data, Control and Audience – allowing companies to focus on their absolute core value. The analogy for us was something similar to the“Model View Controller” (MVC) software design pattern. MVC has seen great success in the last few years in powering many web application stacks. In this context though, the thought is that MVC can increasingly be thought to apply to the web as a whole. In other words, rather than companies being required to engineer the whole stack (data, logic, interface) to deliver value, through APIs they can focus on one element and partner up for the others. An example would be a weather data provider like weather.com being able to focus on the data while leaving it to others to correlate that data with other services (news, travel) and yet others to package it up for the end user via applications.

There were some great audience thoughts also – such as how Terry Jones‘ architecture of “shared memory for the web” (check out fluidinfo!) would affect the emergence of APIs: will they become redundant, become more abstract or even unnecessary? It seems to me that the notion of shared memory available is a big missing piece of the web today and it’s likely to enable APIs and be enabled by APIs just as much as it might replace some of their function. Data flows are just as much part of a computing system as shared memory and given the distributed nature of the web we can likely expect a lot of both point to point data sharing / control and shared addressable storage.

Another hot topic was on standards and when / how they might take hold, with agreement that most areas where APIs are in use are way to young to have effectively “settled” to a standard interface. Further, the largest players are rarely incentivized to slow down their pace of interfaced innovation. This doesn’t bode well for developers in the short and mid term, who likely face rolling change across a lot of the APIs they use. On the other hand the hope is that consolidation will eventually occur on core features of different types of APIs and that differentiation will shift to areas other than simply interface changes.

Hard to do the session justice and no doubt a lot got forgotten here – make sure you book your tickets for Gluecon 2012. 2011 was an awesome event all around and pleasure to be on this panel with Jeff, William and Ben!