Blog

The world of APIs is changing. While the early API landscape was largely dominated by monoliths, or an application which provides all of its services through a giant, singular application, the modern API development architecture has been steadily moving towards microservices. Unlike monolithic systems, Microservices deliver their functionality split between multiple, separate applications and processes, and provide their functionality across multiple servers when needed.

As Microservice development has grown, so has the need for more diverse and extensible architecture designs. In this article, we’re going to take a look at two of the juggernauts of the API world — REST and SOAP — as well as an up-and-coming contender for the throne, Apache THRIFT.

Why It’s Important

When developing an API, one of the most important considerations in the entire development cycle is the architecture upon which the system will be built. The choices made in this design phase can make or break your API, turning a useful concept and smart application into a useless chunk of code. Why is this? What makes architecture so important? Simply put, the type of architecture behind an API alters the way it’s used and the reaction to that use.

The Machine Perspective

Firstly, consider the use of your API from a machine perspective. As the internet has grown by leaps and bounds, so has the scope of the technology it requires — while services initially had to serve a few hundred highly specific requests daily, modern services may service thousands of queries and requests in an almost unthinkably large number of permutations every single hour.

Because of this, the modern API has to be designed to interface with the right systems, at the right time, using the right languages. This is done first in the architecture — after all, a skyscraper is built using a far stronger foundation than an ordinary house, and accordingly, your API must be built with a different foundation depending on the weight of your client, the needs of the system, and the overall requirements of functionality.

The Human Perspective

Even when the technical requirements of your API are considered, there’s still a gulf of potential pitfalls and roadblocks. The user experience falls directly into this gulf. No matter how well your API interfaces with data and systems, the delivery of this data to the user is what is going to be publicly seen and judged — your user base and usage rate will hinge directly upon it. Designing your API to first deliver good functionality, usability, and a good user experience will breed success, adoption, and user retention.

This is largely dealt with in the foundation of your API, just as with designing for machines. While it’s important to consider the basic structure of your API, considering the user experience can help inform our choice away from a machine, service centric model, and more towards a holistic approach.

Defining SOAP, REST, and Apache THRIFT

Now that we know how important it is to create a strong, functional base, what choices do we have in architectural design? While the world of APIs is flooded with proprietary systems, experimental structures, and archaic design platforms, most architectures fall broadly into two categories — SOAP or REST.

REST, or Representational State Transfer, was created in 2000 by Roy Fielding in UC, Irvine; later versions of this architecture were created in collaboration with the W3C Technical Architecture Group (TAG). While SOAP aimed to be a complete system, REST was designed to be more lightweight for building the following right into the design;

Scalability by using cached data from the client and intermediate nodes built into HTTP to self-define;

Portability by tying the transfer of data to the program code during transfer;

Extensibility by allowing individual elements of the greater network to develop independent of one another, using uniform interfaces.

For a time, the two preceding architectures were dominant in the API market. Common thought amongst designers specified that you were either RESTful or SOAP in orientation; this mindset changed with one significant development. In 2007, social media juggernaut Facebook released a technical paper detailing an internal architecture system upon which the bulk of their system operated in.

The service, called Thrift, was soon released as an open source project under the Apache Software Foundation label. The architecture, designed specifically to compete with SOAP and REST while delivering data quickly in a number of formats, includes a complete stack for the manufacture, maintenance, and expansion of clients and servers. The system was designed specifically to include:

Simplicity by eschewing the frameworks of REST and the XML of SOAP in favor of a simple library;

Speed by utilizing binary serialization to handle data;

Evolution by allowing for soft versioning, supporting third party teams to develop RPC calls as needed;

Drawbacks and Benefits With Microservices

Given the bevy of protocol options offered these days, what then is the best choice for a microservice architecture? Unfortunately, that depends largely on the specifics of your particular system, application, or service; subtle minutia that might seem unimportant in the grand scheme of things could change the architecture of choice dramatically.

SOAP is a wonderful architecture of choice when the needs of the system leave no room for misinterpretation. Invoice billing, management of resources, and even city management can utilize the effective strict contract methodologies of SOAP. Additionally, time consuming processes, such as processing a large amount of media data or calculating the most efficient routing paths, are best served by SOAPs asynchronous design. Because SOAP is by definition stateful, services that require the states of both the client and the server can use SOAP far more effectively than REST. Bank transfers, car reservations, and even revision tracking with updating systems utilize this stateful system to its maximum potential.

REST is best used by systems that do not require this additional information — in addition to its increased speed through eschewing XML and heavy system functionality, REST scales extremely quickly, and is more extensible in a dynamic system than SOAP. When viewed within the framework of a web application, REST is a natural candidate due to the fact that it inherits a great deal of its operations from the HTTP stack; furthermore, due to the fact that REST is less restrictive than SOAP, REST is often seen as supporting a larger amount of experimental and future functionality.

Thrift, on the other hand, straddles the best of both worlds. While SOAP is very extensible, it is extremely verbose; Thrift moves away from this, opting for smaller file sizes and simpler documentation. While both REST and SOAP are wonderful architectures, they are limited in the languages they speak; Thrift functions in binary and has support for common languages such as C, C++, C#, PHP, Haskell, Go, and more. Thrift is open source, meaning it is monitored, changed, and tracked by many groups, making it more secure and up to date.

The biggest drawback of Thrift is the fact that it is relatively new, and thus is fighting a crowded field to build a solid user base — SOAP had a 9 year head start on Thrift, one that REST nearly matched with its own 7 year head start. Thrift started behind the curve in terms of adoption — it has steadily grown in use however, with giants such as Tumblr and Facebook pushing adoption through its unique and innovative applications.

Part of the appeal of Thrift is the forward-thinking nature of the architecture — because of its adoption of soft versioning, RPC calls can be freely developed and implemented with a central library or repository functioning as a standard codebase. As future technologies emerge requiring more complex, experimental, and forward-thinking features than REST or SOAP can provide, Thrift will become a more and more attractive proposition.

Here’s a visual comparison of the features for each web service, using the averages of scores collected from the Nordic APIs team:

REST

SOAP

Thrift

Extensibility

☆☆☆☆☆

☆☆☆

☆

Neutrality

☆☆

☆☆☆☆

☆☆☆

Independence

☆

☆☆☆

☆☆☆☆

Large Data Handling

☆

☆☆☆☆☆

☆☆☆

Scalability

☆☆☆☆

☆☆

☆☆☆☆☆

Portability

☆☆☆

☆

☆☆☆☆

Simplicity

☆☆☆

☆☆

☆☆☆☆☆

Speed

☆☆☆

☆

☆☆☆☆☆

Evolution

☆☆

☆

☆☆☆☆☆

Use Case Code Examples

Let’s look at the three standards using a common scenario. In our examples, we have a host defined as “www.example.net”, we have built our architecture on the SOAP standard, and we utilize SOAP requests to get the data of a specific user registered on our platform. In our examples, we’re going to show the most basic request possible in each system.

If using SOAP, our request would look like this:

1

2

3

4

5

6

7

8

9

10

<?xml version="1.0"?>

<soap:Envelope

xmlns:soap="www.example.net/resources/soapenvelope"

soap:encodingStyle="www.example.net/resources/soapencoding">

<soap:body pb="www.example.net/resources/userdatabase">

<pb:GetUserInfo>

<pb:UserID>555</pb:UserID>

</pb:GetUserInfo>

</soap:Body>

</soap:Envelope>

As you can see, the SOAP request used is rather verbose; while certain information detailed in the SOAP message protocol can be left empty, such as the Header and Fault elements, a good deal of information has to be given to the server directly wrapped in the message itself.

This is part of the drawback of SOAP; unless the data being handled is not needed in real-time (i.e. in the case of media processing, large data collation, etc.), SOAP can be considerably slower than other architectures, and is quite verbose, often to the point that the sheer amount of data provided is too verbose.

As you can see, the REST request is far simpler — even the nature of the “method” of each call is far simpler, with the REST call dropping SOAP’s GetUserInfo for a simple URL method of UserInfo. While the SOAP method will deliver the data according to the way the systems are constructed and according to the way requested, the REST method will deliver the data in a raw form.

With Thrift, the request is very different.

1

2

3

serviceFindUserID555{

results find(1:stringname)

}

The trick here is that the bulk of Thrift’s request is handled within the server, and is defined by the API developer. Whereas SOAP and REST can alter their requests to modify their data output (relatively speaking — REST will always deliver its data in a raw format through HTTP unless using a third party service to interpret the data), Thrift’s requests are defined and expanded upon internally.

There is no need for a fault methodology, for a max length delineation, or for a specific location search, as these are all defined in the service “FindUserID” in the server itself. When SOAP or REST receives a request, they need to consult their internal architecture, additional files, or the request itself for information on how to process it. When Thrift receives a request, this handling format is already known, because it is part of the system itself.

Understanding the Services: Snail Mail Metaphor

Complex topics are easier to handle when they’re broken down into a recognizable story. So, for those who may be unaware of these subtleties, we’ll now aim to compare SOAP, REST, and Thrift to sending a letter.

Say you want to send a letter to your neighbor telling them to stop playing the drums at three in the morning. How would you go about this?

Well, if you’re using SOAP, you first write a long letter, then place it into an envelope, printing the address, the return address, and attaching postage. You’re wrapping the message in such a way that the mail system understands who, how, and where to deliver it.

REST, on the other hand, is like a pre-paid postcard. You’ve forgone the envelope in favor of a lightweight, succinct delivery device that is easy to interpret and handle, with your message quickly scribbled on the back. The postcard uses less material (bandwidth), is typically shorter in content, and might actually get to your neighbor faster because there’s less bulk to move around.

Thrift is like a megaphone. When you point the megaphone at your neighbor, he implicitly knows you’re upset about something — and that you’re going to say it specifically to him. He knows what to do with the information, because the reaction a human has to a megaphone is pretty universal (assuming, of course, that your megaphone is louder than his drums). The message is sent in a lightweight but forceful manner that depends entirely on the receiver to properly process it, in this case your neighbor.

Conclusion

It would be easy and convenient to say that any of these architectures were “better” than the other — having a standard architecture from which to work universally in would make development easier, communication between APIs more seamless, and adoption of new technologies far easier. Unfortunately, this is not the case; the API world is so diverse that any number of unique requirements, caveats, and issues could drive adoption of any of the three above mentioned architectures.

When designing your API, however, hark back to the metaphor above. Instead of sending POST requests via the web, think of how the exchange would be processed if you were using actual paper to request information.

If you were requesting album data from a record company, for example, how would you go about it? Would you send a postcard? Probably not, as that wouldn’t allow you to explain specifically why you need the data and in what format you want it, at least not as well as a letter would. Would you use a megaphone? Well, no, because you need a specific service the way you request it, not the way whoever gets your letter might think you need it. So you would decide upon a letter, a simple letter with simple requests. In this case, you may choose SOAP over REST or Thrift, because your needs are specific, and SOAP meets the criteria of your detailed request.

Considering the specific requirements of your Microservice can help you make a more informed choice; whether the verbose flexibility of SOAP beats out the caching and extensibility of REST, or whether the scalability of REST outstrips the feature-rich forward thinking of Thrift, your particular situation will determine your best course of actions.

About Kristopher Sandoval

Kristopher Sandoval is a web developer, author, and artist based out of Northern California with over six years of experience in Information Technology and Network Administration. He writes for Nordic APIs and blogs on his site, A New Sincerity.

I think your missing a big part of REST when you leave out the HATEOAS requirement of the REST architecture.

With HATEOAS you would most probably access the http://www.example.net/resources/userdatabase/UserInfo/555 from a link part of a response from a previous call to another resource (part of call chain originating in the “discovery API”, the API entry point). If the architecture is RESTful the link object would include both rel and type where type could e.g would be com.nordicapis.user+json which would specify the format the response from that resource (link). If I understand what you mean with ‘raw’ correctly it’s not true for a system that has truly implemented the REST architecture where HATEOAS is a requirement.

Kristopher Sandoval

Good point – I mostly avoided HATEOAS due to the fact that this is meant to be a simple introduction. I’ll look into better clarifying this at a later date. Thanks for the comment!

Magnus

I made a comment yesterday regarding the HATEOAS constraint of REST and how that unfortunately is left out in this article. How long before comments become visible?

Interesting comparison… can you elaborate on what you mean by Soft Versioning and thrift (forward thinking)? From the link I thought you meant the following within that paper:

5.3 Case Analysis
There are four cases in which version mismatches may occur.
1. Added field, old client, new server. In this case, the old client does not send the new field. The new server recognizes that the field is not set, and implements default behavior for out-of-date requests.
2. Removed field, old client, new server. In this case, the old client sends the removed field. The new server simply ignores it.
3. Added field, new client, old server. The new client sends a field that the old server does not recognize. The old server simply ignores it and processes as normal.
4. Removed field, new client, old server. This is the most dangerous case, as the old server is unlikely to have suitable default behavior implemented for the missing field. It is recommended that in this situation the new server be rolled out prior to the new clients.

I do not see something new here? REST has the option to make a field as optional and boils down to the same solutions and final concern for case 4 apply as well…

Also I cannot quite follow your reasoning on the remark that thrift is supported by many languages. REST, being an HTTP based protocol is supported by any language that has HTTP Client/Server libraries and some JSON processing.

Last but not least, some elaboration on the number of stars within the comparison, would be very useful for people that need to make a decision on what to choose.