Stay ahead with the world's most comprehensive technology and business learning platform.

With Safari, you learn the way you learn best. Get unlimited access to videos, live online training,
learning paths, books, tutorials, and more.

Introduction

“Most software systems are created with the implicit assumption that
the entire system is under the control of one entity, or at least that
all entities participating within a system are acting towards a common
goal and not at cross-purposes. Such an assumption cannot be safely
made when the system runs openly on the Internet.”

I’m going to show you a better way to do distributed computing, using
the ideas underlying the most successful distributed system in
history: the World Wide Web. I hope you’ll read this book if you’ve
decided (or your manager has decided) that your company needs to
publish a web API. It doesn’t matter whether you’re planning a public
API, a purely internal API, or an API accessible by trusted partners—they can all benefit from the philosophy of REST.

This is not necessarily the book for you if you want to learn how to
write API clients. That’s because most existing API designs are
based on assumptions that are several years old, assumptions that I’d
like to destroy.

Most of today’s APIs have a big problem: once deployed, they can’t
change. There are big-name APIs that stay static for years at a time,
as the industry changes around them, because changing them would be
too difficult.

But RESTful architectures are designed for managing change. The World
Wide Web is made of millions of websites, running atop thousands of
different server implementations, and undergoing periodic
redesigns. Websites are accessed by billions of users who are using
hundreds of different client implementations on dozens of hardware
platforms. Your deployment won’t look like this howling mess, but the
closer you come to web scale, the more familiar this picture will
look.

A very simple system is always easy to change. At small scales, a
RESTful system has a larger up-front design cost than a push-button
solution. But as your API matures and starts to change, you’ll really need some way—like REST—of adapting to change.

An API that’s commercially successful will stay available for years
on end. Some APIs have hundreds or even thousands of users. Even if the
problem domain only changes occasionally, the cumulative effect on
clients can be huge.

Some APIs change all the time, with new data elements and business
rules constantly being added.

In some APIs, each client can change the workflow to suit its
needs. Even if the API itself never changes, each client will
experience it differently.

The people who write the API clients usually don’t work on the same
team as the people who write the servers. All APIs that are open to
the public fall under this category. If you don’t know what kind of
clients are out there, you need to be very careful about making
changes—or you need to have a design that can change without breaking
all the clients.

If you copy existing designs for your API, you will probably only
repeat the mistakes of the past. Unfortunately, most of the
improvements are happening below the surface, in experiments and
through slow-moving standards processes. I’ll cover dozens of specific
technologies in this book, including many that are still under
development. But my main goal is to teach you the underlying
principles of REST. Learn those, and you’ll be able to exploit
whichever experiments pan out and whichever standards are approved.

There are two specific problems I’m trying to solve with this book: duplication of effort and avoidance of hypermedia. Let’s take a look at them.

Duplication of Effort

An API released today will be named after the company that hosts it. We
talk about the “Twitter API,” the “Facebook API,” and the “Google+
API.” These three APIs do similar things. They all have some notion
of user accounts and (among other things) they all let users post a
little bit of text to their accounts. But each API has a completely different
design. Learning one API doesn’t help you learn the next one.

Of course, Twitter, Facebook, and Google are big companies that compete
with each other. They don’t want to make it easy for you to learn
their competitors’ APIs. But small companies and nonprofits do the
same thing. They design their APIs as though nobody else had ever had
a similar idea. This interferes with their goal of getting people to
actually use their APIs.

Let me show you just one example. The website ProgrammableWeb has a directory of over
8,000 APIs. As I write this, it knows about 57
microblogging APIs—APIs whose main purpose is posting a little bit of
text to a user account.[1] It’s great that
there are 57 companies publishing APIs in this field, but do
we really need 57 different designs? We’re not talking
about something complicated here, like insurance policies or
regulatory compliance. We’re talking about posting a little bit of
text to a user account. Do you want to be the one who designs the
58th microblogging API?

The obvious solution would be to create a standard for microblogging
APIs. But there already is a standard that would work just fine: the
Atom Publishing Protocol. It was published in 2005, and almost nobody
uses it. There’s something about APIs that makes everyone want to
design their own from scratch, even when that makes no sense from a
business perspective.

I don’t think I can single-handledly stop this wasted effort, but I do
think I can break down the problem into parts that make sense, and
present some ways for a new API to reuse work that’s already been
done.

Hypermedia Is Hard

Back in 2007, Leonard Richardson and Sam Ruby wrote the predecessor to
this book, RESTful Web Services (O’Reilly). That book also tried to address two
big problems. One of the problems has been solved; the other is
nowhere close to being solved.[2]

The first problem: in 2007, the REST school of API design was engaged
in a standoff against a rival school that used heavyweight
technologies based on SOAP and questioned the very legitimacy of
the REST school. RESTful Web Services was a salvo in this standoff,
a defense of RESTful design principles against the attacks of the SOAP
school.

Well, the standoff is over, and REST won. SOAP APIs are still used,
but only within the big companies that were backing the SOAP school in
the first place. Pretty much all new public-facing APIs pay lip
service to RESTful principles.[3]

Which brings me to the second problem: REST isn’t just a technical
term—it’s also a marketing buzzword. For a long time, REST was a
slogan that signified nothing beyond opposition to the SOAP school. Any
API that didn’t use SOAP was marketed as REST, even if its design
made no sense or betrayed the technical principles of REST. This was
inaccurate, confusing, and it gave REST—i.e., REST as a technical term—a
bad name.

This situation has improved a lot since 2007. When I look at new APIs,
I see the work of developers who understand the concepts I’ll be
explaining in the first few chapters of this book. Most developers who
fly the REST flag today understand resources and representations, how
to name resources with URLs, and how to properly use HTTP methods. The
first three chapters of this book don’t do much but get new developers
up to speed.

But there’s one aspect of REST that most developers still don’t
understand: hypermedia. We all understand hypermedia in the context of
the Web. It’s just a fancy word for links. Web pages link to each
other, and the result is the World Wide Web, driven by hypermedia. But
it seems we’ve got a mental block when it comes to hypermedia in web
APIs. This is a big problem, because hypermedia is the feature that
makes a web API capable of handling changes gracefully.

Starting in Chapter 4, my overriding goal for RESTful Web APIs will
be to teach you how hypermedia works. If you’ve never heard of this
term, I’ll teach it to you along with the other important REST concepts. If
you’ve heard of hypermedia but the concept intimidates you, I’ll do
what I can to build up your courage. If you just haven’t been able to
wrap your head around hypermedia, I’ll show it to you in every way I
can think of, until you get it.

RESTful Web Services covered hypermedia, but it wasn’t central to
the book. It was possible to skip the hypermedia parts of the book
and still design a functioning API. By contrast, RESTful Web APIs is
effectively a book about hypermedia.

I did it this way because hypermedia is the single most important
aspect of REST, and the least understood. Until we all understand
hypermedia, REST will continue to be viewed as a marketing buzzword rather
than a serious attempt to handle the complexity of distributed
computing.

What’s in This Book?

The first four chapters introduce the concepts behind REST, as it
applies to web APIs.

Hypermedia is the missing ingredient that
ties representations together into a coherent API. This chapter shows
what hypermedia is capable of, mostly using a hypermedia data format
you’re already familiar with: HTML.

The next four chapters describe different strategies for designing a
hypermedia API:

When the collection pattern
doesn’t fit your requirements, you can convey any representation you
want using a general-purpose hypermedia format. This chapter shows how
it works using three general hypermedia formats (HTML, HAL, and Siren)
as examples. This chapter also introduces HTML microformats and
microdata, which lead in to the next chapter.

A profile fills in the gaps between a data
format (which can be used by many different APIs) and a specific API
implementation. The profile format I recommend is ALPS, but I also
cover XMDP and JSON-LD.

In this chapter, my advice begins to outstrip the state of the art at the
time this book was written. I had to develop the ALPS format for this
book, because nothing else would do the job. If you’re already
familiar with hypermedia-based designs, you might be able to skip up
to Chapter 8, but I don’t think you should skip past it.

Chapters 9 through 13 cover practical topics like choosing the right hypermedia format and getting the most out of the HTTP protocol.

Linked Data is
the Semantic Web community’s approach to REST. JSON-LD is arguably the most
important Linked Data standard. It’s covered briefly in
Chapter 8, and I revisit it here. This chapter also covers the
RDF data model, and some RDF-based hypermedia formats that I
didn’t get to in Chapter 10.

Similar to Appendix A, this appendix is also an extension of Chapter 11. It provides a detailed outline of the
46 request and response headers defined in the HTTP
specification, as well as a few extensions.

The glossary contains definitions to terms you’ll frequently encounter when working with RESTful web APIs. It’s a good place
to turn for familiarizing yourself with basic concepts or if you need a quick,
at-a-glance reminder of a particular concept’s definition.

What’s Not in This Book

RESTful Web Services was the first book-length treatment of REST,
and it had to cover a lot of ground. Fortunately, there are now over a
dozen books on various aspects of REST, and that frees up RESTful Web
APIs to focus on the core concepts.

To keep this book focused, I’ve removed a few topics that you might
have been expecting me to cover. I want to tell you what is not in
this book, so that you don’t buy it and then feel disappointed:

Client programming is not covered here. Writing a client to consume a hypermedia-based
API is a new kind of challenge. Right now, the closest thing we have
to a generic API client is a library that sends HTTP requests. This
was true in 2007, and it’s still true. The problem is on the server
side.

When you write a client for an existing API, you’re at the mercy of
the API designer. I can’t give you any general advice, because right
now there’s no consistency across APIs. That’s why, in this book,
I’m trying to drum up enthusiasm for a little server-side
consistency. When APIs become more similar to each other, we’ll be
able to write more sophisticated client-side tools.

Chapter 5 contains some sample client implementations and tries to
classify different types of clients, but if you want a whole book on
API clients, this is not your book. I don’t think the book you want
exists right now.

The most widely deployed API client in the world is JavaScript’s
XMLHttpRequest library. There’s a copy in every web browser, and
most websites today are built atop APIs designed for consumption by
XMLHttpRequest. But that’s far too big a field to cover properly in
this book. There are whole books written about individual JavaScript
libraries.

I spend quite a bit of time on the mechanics of HTTP (Chapter 11,
Appendix A, and Appendix B), but I don’t cover any given HTTP topic in a
lot of depth, and there are some topics—notably HTTP intermediaries
like caches and proxies—which I barely cover at all.

RESTful Web Services focused heavily on breaking down your
business requirements into a set of interlinked resources. My
experience since 2007 has convinced me that thinking of API design as
resource design is a very effective way to avoid thinking about
hypermedia. This book takes a different approach, focusing on
representations and state transitions rather than resources.

That said, the resource design approach is certainly valid. For advice on moving in that direction, I
recommend RESTful Web Services Cookbook by Subbu Allamaraju (O’Reilly).

Administrative Notes

This book has two authors (Leonard and Mike), but for the duration of
this book we’ve merged our identities into a single
authorial “I.”

Nothing in this book is tied to any particular programming
language. All of the code takes the form of messages (usually JSON
or XML documents) sent over a network protocol (usually HTTP). I will
be assuming that you’re familiar with common programming
concepts like antipatterns and breadth-first search, and that you have
a basic understanding of how the World Wide Web works.

I chose Node because it lets me use the same programming language for
client and server code. You won’t need to mentally switch back and
forth between programming langauges to understand both sides of a
client-server transaction. Node is open source and available on
Windows, Mac, and Linux systems. It is easy to install on these
operating systems, and you shouldn’t have much trouble getting the
examples up and running.

I’m hosting the code on GitHub because that will make it easy to
update the implementations over time. This also makes it possible for
readers to contribute ports of the example clients and servers to
other programming languages.

Understanding Standards

The World Wide Web isn’t an objective thing that’s out there to be
studied scientifically. It’s a social construct—a set of agreements
to do things a certain way. Fortunately, unlike other social
constructs (like etiquette), the agreements underlying the Web are generally agreed upon. The core agreements underlying
the human web are RFC 2616 (the HTTP standard), the W3C’s
specification for HTML 4, and ECMA-262 (the standard that underlies
JavaScript, also known as ECMAScript). Each standard does a different
job, and over the course of this book, I’ll discuss dozens of other
standards designed specifically for use in APIs.

The great thing about these standards is the solid baseline they give
you. You can use them to build a completely new kind of website or
API, something that no one has ever tried before. Instead of having to
explain your entire system to all your users, you’ll only have to
explain the part that’s new.

The bad news is that these agreements are often borderline unreadable:
long walls of ASCII text written in tooth-achingly precise English in
which everyday words like “should” have technical meanings and are
capitalized “SHOULD.”[4] A lot of technical books are bought by people who are
hoping to avoid having to read a standards document.

Well, I can’t make any guarantees. If one of these standards looks
like something you can use in your work, you need to be willing to
dive into its spec and really understand it (or buy a book that
covers it in more detail). I don’t have space to give more than a
basic overview of standards like Siren, CoAP, and Hydra. Not to
mention that giving a lot of detail would bore all the readers who
don’t need those particular standards to do their work.

When navigating the forest of standards, it’s useful to keep in mind
that not all standards have equal force. Some are extremely
well established, used by everyone, and if you go against them you’re
causing a lot of trouble for yoursef. Other standards are just one
person’s opinion, and that opinion might be no better than yours.

I find it helpful to divide standards into four categories: fiat standards, personal standards, corporate standards, and open standards. I’ll be
using these terms throughout the book, so let me explain each one in a bit more depth before we move on.

Fiat Standards

Fiat standards aren’t really standards; they’re behaviors. No one
agreed to them. They’re just a description of the way somebody does
things. The behavior may be documented, but the core assumption of a
standard—that other people ought to do things the same way—is
missing.

Pretty much every API today is a fiat standard, a one-off design
associated with a specific company. That’s why we talk about the
“Twitter API,” the “Facebook API,” and the “Google+ API.” You may need
to understand these designs to do your job and you may write your own
clients for these designs, but unless you work for the company in
question, there’s no expectation that you should use this design for
your API. If you reuse a fiat standard, we don’t say your API
conforms to a standard; we say it’s a clone.

The main problem I’m trying to solve in this book is that hundreds of
person-years of design work is locked up in fiat standards where it
can’t be reused. This needs to stop. Designing a new API today means
reinventing a long series of wheels. Once your API is finished, your
client developers have to reinvent corresponding wheels on the client
side.

Even under ideal circumstances, your API will be a fiat standard, since
your business requirements will be slightly different from everyone
else’s. But ideally a fiat standard would be just a light gloss over a
number of other standards.

When I describe a fiat standard, I’ll link to its human-readable
documentation.

Personal Standards

Personal standards are standards—you’re invited to read the documents
and implement the standards yourself—but they’re just one person’s
opinion. The Maze+XML standard I describe in Chapter 5 is a good
example. There’s no expectation that Maze+XML is the standard way to
implement a maze game API, but if it works for you, you might as well
use it. Someone else has done the design work for you.

Personal standards generally use less formal language than other kinds
of standards. Many open standards start off as personal standards—as side projects that are formalized after a lot of
experimentation. Siren, which I cover in Chapter 7, is a good
example.

When I describe a personal standard, I’ll link to its specification.

Corporate Standards

Corporate standards are created by a consortium of companies trying to
solve a problem that plagues them all, or by a single company trying
to solve a recurring problem on behalf of its customers. Corporate
standards tend to be better defined and to use more formal language
than personal standards, but they have no more force than personal
standards. They’re just one company’s (or a group of companies')
opinion.

Corporate standards include Activity Streams and schema.org’s
microdata schemas, both of which are covered in Chapter 10. Many industry
standards start off as corporate standards. OData (also discussed in Chapter 10)
started as a Microsoft project, but it was submitted to OASIS in 2012
and will eventually become an OASIS standard.

When I describe a corporate standard, I’ll link to its specification.

Open Standards

An open standard has gone through a process of design by committee,
or at least had an open comment period during which a lot of people
read the specification, complained about it, and made
suggestions for improvement. At the end of this process, the
specification was blessed by some kind of recognized standards body.

This process gives an open standard a certain amount of moral
force. If there’s an open standard that does more or less what you
want, you really should use it instead of making up your own fiat
standard. The design process and the comment period probably turned up
a lot of issues that you won’t encounter until it’s too late.

In general, open standards come with some kind of agreement that
promises you can implement them without getting hit with a patent infringement lawsuit from a company that was involved in the
standards process. By contrast, implementing someone else’s fiat
standard may incite them to file a patent infringement
lawsuit against you.

A few open standards mentioned in this book came out of the
big-name standards bodies: ANSI, ECMA, ISO, OASIS, and especially the
W3C. I can’t say what it’s like to sit on one of these standards
bodies, because I’ve never done it. But the most important standards
body[5] is one anyone can
contribute to: the IETF, the group that manages the all-important
RFCs.

Requests for Comments (RFCs) and Internet-Drafts

Most RFCs are created through a process called the Standards
Track. Throughout this book, I’ll be referencing documents that are in
different places on the Standards Track. I’d like to briefly discuss
how the track works, so that you’ll know how seriously to take my
recommendations.

An RFC begins life as an Internet-Draft. This is a document that
looks like a standards document, but you’re not supposed to build
implementations based on it. You’re supposed to find problems with the
specification and give feedback.

An Internet-Draft has a built-in lifetime of six months. Six months
after it is published, a draft must be approved as an RFC or replaced
with an updated draft. If neither of those things happens, then the
draft expires and should not be used for anything. On the other hand,
if the draft is approved, it expires immediately and is replaced by an
RFC.

Because of the built-in expiration date, and because an Internet-Draft
isn’t technically any kind of standard, it’s tricky business
mentioning them in a book. At the same time, API design is a field
that’s changing rapidly, and an Internet-Draft is better than
nothing. I will be mentioning many Internet-Drafts in this book under
the assumption that they’ll become RFCs without major changes. That
assumption has held up pretty well; several Internet-Drafts that I mention here became RFCs while I was writing the book. If a
particular draft doesn’t pan out, all I can do is apologize in
advance.

RFCs and Internet-Drafts are given code names. When I describe one of
these, I won’t link to its specification. I’ll just refer to it by
its code and let you look it up. For example, I’ll refer to the HTTP/
1.1 specification as RFC 2616. I’ll refer to an Internet-Draft by
its name. For example, I’ll use “draft-snell-link-method” to refer to
the proposal to add LINK and UNLINK methods to HTTP.

Whenever you see one of these code names, you can do a web search and
find the latest version of the RFC or Internet-Draft. If an
Internet-Draft becomes an RFC after this book is published, the final
version of the Internet-Draft will link to the RFC.

When I describe a W3C or OASIS standard, I’ll link to the
specification, because those standards aren’t given code names.

Conventions Used in This Book

Used for program listings, as well as within paragraphs to refer to program elements such as variable or function names, databases, data types, environment variables, statements, and keywords.

Constant width bold

Shows commands or other text that should be typed literally by the user.

Constant width italic

Shows text that should be replaced with user-supplied values or by values determined by context.

Tip

This icon signifies a tip, suggestion, or general note.

Warning

This icon indicates a warning or caution.

Using Code Examples

This book is here to help you get your job done. In general, if this book includes code examples, you may use the code in this book in your programs and documentation. You do not need to contact us for permission unless you’re reproducing a significant portion of the code. For example, writing a program that uses several chunks of code from this book does not require permission. Selling or distributing a CD-ROM of examples from O’Reilly books does require permission. Answering a question by citing this book and quoting example code does not require permission. Incorporating a significant amount of example code from this book into your product’s documentation does require permission.

We appreciate, but do not require, attribution. An attribution usually includes the title, author, publisher, and ISBN. For example: “RESTful Web APIs by Leonard Richardson and Mike Amundsen (O’Reilly). Copyright 2013 Leonard Richardson and amundsen.com, Inc., and Sam Ruby. 978-1-449-35806-8.”

If you feel your use of code examples falls outside fair use or the permission given here, feel free to contact us at permissions@oreilly.com.

Safari® Books Online

Note

Safari Books Online is an on-demand digital library that delivers expert content in both book and video form from the world’s leading authors in technology and business.

Acknowledgements

We owe a debt of thanks to Glenn Block who spent untold hours listening to ideas and working through real code to test those ideas. To Benjamin Young and all the folks at RESTFest who agreed to be part of our experiments, and who gave great feedback and advice even when we didn’t want to hear it. To Mike’s colleagues at Layer 7 Technologies, including Dimitri Sirota and Matt McLarty, who supported and encouraged his work on this project. To Sam Ruby and Mike Loukides, who were essential to RESTful Web Services, this book’s predecessor. To Sumana Harihareswara, Leonard’s supportive wife. To the social communities that create an excellent place to collaborate and converse on REST and APIs; especially Yahoo’s REST-Discuss, Google Groups’ API-Craft, and the Hypermedia group at LibreList.

[1] The full list of ProgrammableWeb APIs tagged with microblogging provides information about each of these APIs.

[2] RESTful Web Services is now freely available as part of O’Reilly’s Open Books Project. You can download
a PDF copy of the book from the book’s page.

[3] If you’re wondering, this is
why we changed the title. The term “web services” became so tightly
coupled with SOAP that when SOAP went down, it took “web services”
with it. These days, everyone talks about APIs instead.