APIs aren’t apps: Make them as thin and light as possible

For fast and scalable APIs, forget the heavy frameworks and go lean

A few weeks ago I talked about how Bash is more powerful than you might think. My point was that sometimes choosing the best tool for the job means skipping over the newer, sexier, but more dependency-ridden language in favor of the simpler and ubiquitous alternative. It’s all about the end goal, not the means of getting there.

I discussed this with respect to a simple script that could be used to manage a remote system, but similar examples are everywhere -- even a few places you might not expect, like Web APIs. Of course, I’m not really suggesting Bash for CGI work -- that’s the true home of the brave. However, choosing the right language and tool set to develop an API shouldn’t be based on what the hot and sexy languages might offer in terms of ease of coding and frameworks, but rather on what the API is designed to do, expected scalability needs, modularity, and extensibility.

Web APIs generally fall into three categories: read-only, write-only, and read-write. A read-only API might be an information service that receives structured requests from an app or client and delivers data back in JSON, XML, or HTML. A write-only API receives data from a client and returns a result code, nothing more. A read-write API does both: It might respond to queries with data, but can also ingest report data from a client. All of these are very different scenarios and all should be weighed on their own merits during design.

If we look at the read-only API, we should see the need for heavy database interaction, as much caching as possible, and light code. All this API is doing is receiving a query, validating the query and possibly performing authorization, restructuring the query into a suitable database query, returning the resulting data in the proper formatting, and caching the result based on the query to lighten the load on the database server and speed up results for the next client asking for the same data. The database should be structured for heavy reads, Memcached or even Redis should be used for caching, and the front-end code should be extremely light.

While it may be tempting to write this code in a heavier language that affords possibly simpler development at the expense of a heftier application, that’s probably not the best idea. This type of API would fly if written in PHP with Nginx/PHP-FPM (without a PHP Web framework), for instance. Nginx is very good at thin, lithe apps like this, in addition to static content delivery, which may or may not be part of the API. This isn’t the place to do anything with Java or Tomcat.

On the other hand, we have the polar opposite API: write-only. This isn’t a perfect mirror of the read-only API, primarily due to the lack of caching, but it’s quite similar. It too does basic authentication/authorization, error handling, and data format inspection, then structures the submitted data to insert into a database. The database thus needs to be tuned for heavy writes, and as with all APIs, request throttling should be used to prevent clients from getting out of hand. This is another place where thin and lithe rules the day, and that can quickly translate into both higher performance and lower operational costs.

The combination of the two is the read-write API, which is the category that most APIs fall into. The read-write ratio might be 50/50, 60/40, or 90/10, but most APIs will accept and return data of some form (though anything exceeding 90/10 is basically a read-only API). This is where it can get sticky; if the API accompanies a website or an app, it might be prudent to leverage the same code for both.

A prime example is the Ruby on Rails framework, which allows you to easily bolt on a RESTful API, removing the need to develop a separate codebase for the API. This is a wonderful concept for many applications because you get to have your cake and eat it too -- but it’s going to burn you in the scalability department compared to code written specifically as an API in a lighter, raw language. For the purposes of your particular app, this might not be a concern. Not every Web app is written for Web scale, after all.

The fact of the matter is that there's no such thing as a free lunch, and not everything is a nail. If scalability and performance are key for a Web API, think as thin as possible, and leave the heavy frameworks behind.