Erik Wilde on Services and APIs

Wednesday, May 20, 2009

REST Programming Toolbox Requirements

REST is all the rage, and everybody claims they're doing REST. REST
is a style of designing web services, and the question is how this
style can be best supported by a programming environment. there are two
possible perspectives of this problem. one is the more high-level view
of how to best structure code that is built around the assumption of a
uniform interface. a popular example for this is JAX-RS, released as JSR-311 last year, and implemented by the Restlet framework. this is not what i want to discuss here.

a different perspective is that of basic tools that might be useful
for implementing RESTful services (in that case interpreting REST in
its narrower sense of web architecture and its use of HTTP and URIs).
in a recent blog post, marc hadley reports on Integrating Jersey and Abdera.
the interesting thing is how his approach combines a high-level
framework (JAX-RS), with more low-level tools for handling HTTP and
content (in that case, feeds).

so looking at the low-level
support one would like to see when programming RESTful services, what
is required or useful? i came up with this list, which is probably
incomplete, so any feedback about what else should go in here would be
highly welcome!

HTTP support: HTTP must be fully accessible, which means
that all methods, headers, and status codes must be accessible to the
application. if required, certain behavior might be enabled on request
(such as following redirects), but this should be fully under the
control of the application.

HTTP encoding: since content in HTTP can be transmitted
using chunked encoding or some transfer-encoding, it must be possible
to either explicitly deal with these encodings, or to have this handled
by the HTTP API.

HTTP authentication: in many cases, HTTP authentication is
required for gaining authenticated access to access-controlled
resources. the HTTP API should support the predefined HTTP
authentication methods (basic/digest), and should also have a plug-in
mechanism so that other authentication methods can be used as well.

caching: caching is one of the core features of how REST can
be optimized. caching can be based on modification dates and/or on
ETags. client-side caches can be shared or private caches. there must
be built-in support for caching, API controls for handling shared and
private caches, and there should be integration with a client-side
shared cache for the client as a whole (imagine how useful it would be
if all browsers on your computer were smart enough to share the same
cache).

URI parser: URI parsing is not trivial, and URIs are used
everywhere. there should be URI prcessing support for parsing and
resolving URIs. and depending on where URI templates are going (they are still under construction), support for them might be very useful as well.

language tag processing: since language tags are part of
HTTP and can be used in HTTP mechanisms such as content negotiation,
handling language tags is a integral part of HTTP. since language tags have a surprisingly complex logic behind them, support for parsing and matching languages is very useful.

MIME type processing: similarly to language tags, MIME types (officially called media types)
have a structure, are used in content negotiation, and often should be
compared or matched. support for these operations should be part of a
toolbox as well.

XML tools: XML is the most widely used syntax for
machine-readable data, so there should be a parser, validation support
for maybe various schema dialects (DTD, XSD RELAX NG), selection via
XPath, hopefully XPath 2.0 as the more powerful language, and finally
more advanced XML processing might be supported by XML-centric
languages such as XSLT or XQuery.

JSON tools: many RESTful services support JSON, and
sometimes only JSON. support for parsing JSON therefore also should be
part of the toolbox.

Atom tools: one of the most widely used formats for
machine-readable data is Atom, and thus Atom support should be part of
the toolbox, also supporting additional features which are part of
the Atom landscape such as feed paging.

AtomPub: AtomPub is one way of implementing a RESTful
service providing write access to Atom-oriented services, and thus
support for AtomPub (at least for the client side) might be useful as
well.

fault-tolerant parsing: many RESTful services access
resources from a variety of sources. support for fault-tolerant parsing
of HTML and feeds (in any of the RSS variants and in Atom) makes it
possible to implement more robust applications.

while i am sure that this list is incomplete, i am really interested
to better figure out what it really takes to provide a useful REST
programming toolbox. currently, i am mostly interested to figure out
what would be of general interest to most/many RESTful service
developers.

@scott @cesare: it seems to me that WADL was a nice thought experiment but was largely rejected by the community. which of course doesn't mean it couldn't be supported, but it might not be something many people would use.

wrt tool: yes, tools are good. testing is something i'd see a bit outside of the actual language toolbox for development, but it definitely should be part of each developer's toolbox. but what kind of "APIs" would you like to document, scott? according to REST, they all have a uniform interface, so most of the "documentation" is in the media types and in the interactions. but yes, there should be some support for systematically documenting the grid of HTTP methods and "URI templates".

HTTPS: definitely something i should have mentioned, HTTPS should be supported as part of the toolbox, and it should support the largest possible set of encryption methods. authentication (which i did mention) is useful and often required, but without HTTPS, authentication often i only one part of security. another thing one might add to HTTPS is the support of client-side certificates, which are optional in HTTPS, but very useful in scenarios where all participants can agree on a set of certification authorities they are going to trust.