Playing Together Nicely: Getting REST and SOAP to Share Each Other's Toys

It's tremendously difficult to argue a RESTful approach to a service-oriented architecture (SOA), when the corporate mindshare is SOAP--where project stakeholders tout the SOA buzzword, nod their heads sagely when you say SOAP, nod their heads again when you say XML-RPC, and then look blankly when you mention REST. At an official level, it seems that for the IBMs, Suns, Microsofts, and Oracles (et al) of this world, REST isn't even on the radar; perhaps more because they would find it difficult to build a commercial strategy around something that is based on simplicity and standards (like HTTP) that have been around for years, than from a true lack of visibility at the coalface. So when the push from on high is for SOAP web services and associated technologies, and your business partners and colleagues have been drinking the Sun/MS/IBM/etc. Kool-Aid, you're generally fighting a losing battle if you're promoting alternatives.

While you might usually end up stuck in a buzzword-compliance nightmare, with packets of WSDLs, BPELs, and SOAPs flying around left, right, and center, there are occasions where it may be possible to push through a REST-style, resource-centric approach; where there is no official strategic direction for SOA already in place, where there is a reasonable amount of flexibility and imagination on the part of the project owners, and perhaps with a bit of technical enlightening on the part of the technical lead(s).

But it does still come back to the fact that a service-oriented architecture, in particular, is not a simple, disconnected, independent application, and that many of the organizations wishing to interact with the system will be expecting the (in some cases, de facto) standard mechanisms for interaction, and won't be happy unless your software is emitting buzzwords and acronyms like an engine with a split gasket spits oil.

The best method I've come up with so far to get around this issue is to deliver both. Design the core of your system with REST resources, and subsequently plug in some kind of interfacing or adapter to the REST components, in order to deliver SOAP messaging for those who require it. This article discusses one approach to a SOAP interface for REST services. However, one thing we aren't going to do is resolve issues of context between REST's coarser-grained idea of a resource, and SOAP's fine-grained RPC methods. There is guaranteed to be a significant chunk of the REST crowd who think, "Why bother? A REST resource doesn't map to SOAP calls and it shouldn't, anyway." That's fine, and agreed, to a point--however, this article is aimed at those fighting the good fight in corporate IT, wanting to design REST systems, but stuck delivering SOAP messages because it's the standard, and what everyone expects you to deliver.

A side note: there are, of course, SOAP web methods, which provide an HTTP verb-style interface to a standard SOAP web service--which would be a more direct mapping to a REST resource. But if you're using a form of orchestration, pulling a number of services and processes together with something like BPEL, you are potentially going to be constrained (at the very least by convention, but perhaps also by technical considerations of the orchestration and choreography tools) to using GET and (more commonly) POST for your SOAP methods.

The REST Resource Definition

The place I'll start is with the REST services, or resources (in REST terminology). For the purposes of this discussion, I've (RAD) prototyped all of the code in Jython, which gives me access to all of the Java libraries I need to use, and also allows me to throw components together without too much infrastructure (and without too much in the way of recompilation and preparation). Jython also has the advantage of being able to (generally) do more in less space than Java, so from the point of view of explanation, it's quite a useful tool. That said, it's worthwhile to note that in a production setting, I would re-implement certain segments of the code, where performance would be an issue (the interfacing specifically), in "native" Java, due to Jython's performance limitations.

To get the directory structure out of the way first, I've set up a directory in my servlet container's webapps directory with the basic layout in Figure 1.

Figure 1. Directory layout

Working servlet files (in this case, Jython scripts) will be placed in the root test folder, while source code obviously goes in the source directory and is compiled to classes--pretty much as you might expect. I'll come to the generated directory shortly.

To test the services for yourself, extract the contents of the sample code rest+soap.zip file, found in the Resources section, into the webapps directory of your servlet container (tested on Jetty and Tomcat 5). The various .jar files required for WEB-INF/lib are detailed in the readme.txt file located in that directory. Source code can be compiled using Ant and the build.xml file located in source.

For this example, my REST service will be a "shopping basket" of stocks, supporting GET (retrieve an existing shopping basket), PUT (create or update a shopping basket), and DELETE (remove a basket). The URI for accessing the resource will be http://localhost:8080/test/basket/<basket_name>, so it could be viewed from a browser with something like http://localhost:8080/test/basket/mybasket. In addition, I'm going to use XML schemas to define the messages passed to and from my resource, and JAXB to simplify the process of conversion from flat XML to object. So the first step in this process will actually be defining the XML schema for the basket resource. In my (perhaps less than) infinite wisdom, I've made the decision that the definitions for my resource will go in schema file Basket.xsd, while any commonalities (i.e., definitions that could be used by other resources) will go in Common.xsd. For example, the definition for the BasketRQ element is show below:

BasketRQ is used for creating a basket of stocks--a basket can contain a reference, and then a number of items (each of which contain a string code and an integer quantity). Note the emphasis on can contain a reference--in the case of my REST resources, the reference won't feature in the XML message, but will be important when we get to the SOAP interface later. Including the Reference element in this schema (despite the fact it isn't required at all by REST resources) is necessary to reduce the complexity of maintaining an alternate XML schema specially for the matching SOAP services.

The next schema excerpt shows the basket response (BasketRS), which has two additional elements: the stock price, and total price after multiplying stock price by the quantity:

These two elements are used in the GET and PUT methods of the basket resource: a PUT expects a BasketRQ, while a GET returns the BasketRS. The elements (along with all other elements defined in these schema files) are generated as JavaBeans using a JAXB Ant task (which can be found in build.xml, again located in the source directory).

Code for the Resource

The straightforward code for the GET method is shown below--the basic process is to retrieve the basket name (or reference) from the servlet request path info, and then loop through the matching basket object (if found), updating the price and quantity fields. The object is then marshalled to XML using JAXB, to be written back to the client.

def doGet(self, request, response):
response.setContentType('text/xml')
w = response.getWriter()
basketref = request.getPathInfo()[1:]
if not baskets.has_key(basketref):
response.sendError(HttpServletResponse.SC_NOT_FOUND,
'basket reference "%s" was not found' % basketref)
return
# loop through the items in the basket, and update the price and total fields
# with the price retrieved from the NZ stock exchange
itr = baskets[basketref].getItem().iterator()
while itr.hasNext():
item = itr.next()
item.setPrice(BigDecimal(get_stock_price(item.getCode())))
item.setTotal(BigDecimal(item.getPrice().doubleValue() *
item.getQuantity().intValue()))
# write back the xml for the basket, using jaxb to marshal the object
w.write(pyutils.marshal('testbeans', baskets[basketref]))
w.close()

The code for the doPut method is also fairly simple, with the following process:

Get the basket name from the servlet path info.

Unmarshal the XML in the body of the PUT request into a BasketRQ Java object using JAXB.

Create a BasketRS Java object and copy the contents of the BasketRQ into it (necessary so we don't need to do this for every GET request).

Save the BasketRS to the "datastore," which for the purposes of this example is just a map of objects in local memory.

There is an additional, and rather important, HTTP method the REST resource will implement: doOptions. The code for this method is shown below:

represent a request for information about the communication options available on the request/response chain identified by the Request-URI. This method allows the client to determine the options and/or requirements associated with a resource, or the capabilities of a server, without implying a resource action or initiating a resource retrieval.

So in the example above, I use an Allow header to specify which HTTP methods are supported, and then custom headers (prefixed with X-*) to present more detail about the schema used by the resource, and which nodes are used for the requests and responses to particular methods. All of this allows a client process to interrogate the resource to find out what its requirements are, which will become important shortly.

SOAP Interface #1: Discovering the REST Resources

The SOAP interface is basically a servlet pretending to be any number of SOAP services. The interface must first figure out:

What resources it needs to mimic.

What HTTP methods each resource implements.

What translations need to be performed in order for a SOAP client to talk to the SOAP interface, and then for the SOAP interface to talk to the REST resource.

REST doesn't necessarily constrain you to any particular form of web service contract or the publication of service definitions to a centralized directory--there is plenty of discussion and debate on the Web, from both the REST and SOAP camps (and within those camps as well) over the benefits of either approach from a discovery point of view. Without getting into any debate over discovery methods, the REST resource in this example is published in a rather obvious and simple place: http://localhost:8080/test/discovery.html:

SOAP Interface #2: Missing Schemas

In creating the SOAP "mapping," we need to take into account some fundamental characteristics that will be generally true across most REST resources:

A GET request will not contain a body. Therefore, we need to create an XML schema for the body content of the associated SOAP request (of course, we don't need to create the schema for the response, because we will use the response from the GET).

A PUT request will contain a body, and may return a response, in which case default schemas would be required for neither requests nor responses.

A DELETE request will not contain a body, but may return a response (the contents of the item deleted), so will require a separate schema for the SOAP requests.

For the purposes of this article, neither my DELETE nor PUT methods will return a response--hence, default schemas will be required.

That aside, the main difficulty we have mapping a SOAP call to a REST resource method is that of data identification (or naming). A REST resource is identified by its URI: /order/123A, /stock/ORCL. A SOAP web service can also be identified by a URI (for example, /GetLocalReservations?code=ABCD) but may just as likely be identified within the SOAP message itself. Hence, the URI will have no identifying characteristics other than that of distinguishing the service method itself. Hopefully, the reason for including a Reference element in both the BasketRQ and BasketRS definitions has now become clear. For a REST resource, the Reference isn't used and will be ignored. For a SOAP service, it is a necessary part of identifying the data element in question.

Consequently, the definitions of GenericRQ and GenericRS, which are used where a REST resource does not require content for a request or for responses (but a SOAP request does need this data), also include the Reference element, although in these cases it is no longer optional:

A GenericRQ identifies data by its Reference, while the response adds the HTTP status code and message. A Reference is, in turn, made up of a name and any number of Parameter elements--required in order to reproduce the information that is included in a typical REST resource URI (where parameters may be used to filter the data returned, uniquely identify subsets of data, and so on).

The doPut method of the SOAP interface (found in the Jython script soap.py), is used for all SOAP requests to the system, and is (excerpted) below:

Get the reference from the object and then turn this into a URI for calling the REST resource.

Call the resource, and in the event of a successful result with content, write this back to the client.

In the case of an unsuccessful result, or no content, write a GenericRS back to the client including the status and message from the REST resource.

What is perhaps not immediately obvious from this code is the fact that JAXB is unmarshalling the SOAP envelope (soapobj = pyutils.unmarshal('testbeans', s)), while the body content (retrieved via obj = soapobj.getBody().getAny().get(0)) is then remarshalled to send on to the REST resource. This is accomplished by way of the XML schema for SOAP envelopes (envelope.xsd), large chunks of which have been hacked out in order to get it working successfully with the version of JAXB I'm using (you will also find this somewhat-mangled version in the source code for this article).

SOAP Interface #3: Buzzword Compliance

The doGet method of soap.py is the final piece of the interfacing puzzle. It is used to deliver a WSDL definition for each of the services mapped by the system. For example, Figure 2 shows the results of browsing to http://localhost:8080/test/soap/wsdl/GetBasket.wsdl.

Because both my definitions and the responses from a REST request are generally quite limited, this WSDL file is not as complicated as it could be. Although the rather pedantic nature of validating WSDLs (missing "/" anyone?) means slight changes can have significant impacts on validation, and I'm only guessing (because I have yet to fully test this final feature) as to its ultimate usability.

Figure 2. Browsing the WSDL

Running/Testing the Services

In the resource file for this article, I've included a number of Jython scripts for testing the various services, which can be found in the source directory.

The scripts test-put.py, test-get.py, test-options.py, and test-delete.py can all be used for calling the REST resource directly, while soaptest-PutBasket.py, soaptest-GetBasket.py, soaptest-GetBasket-invalid.py, and soaptest-DeleteBasket.py can be used for the SOAP services. To run a test, from a command prompt change to the WEB-INF/source directory for the project and execute jython test-put.py (for example). Figure 3 shows the results of executing GetBasket, after PutBasket has been run.

Figure 3. Testing GetBasket

Conclusion

In this article, I've presented one method of creating a SOAP service that can communicate with REST resources, for client applications that are unable (or unwilling) to talk to those resources directly. However, this is really only a starting point: it didn't discuss security (authentication, authorization, securing transmissions, and so on), which is a major component of any web services application, but is in this case, of course, very dependent upon what approach you take when developing the RESTful components. This functionality needs to be addressed for the SOAP interface to be a valid solution. My initial thoughts are that, at least for the authentication functionality, a pluggable structure will be required for the SOAP interface to handle different authentication protocols (at least where this is driven by the client/caller of the service).

Other features (how many billion WS-* specifications are there now?) may also be client-driven, or dictated by the business, and affect the extent to which you can successfully design and implement your SOA as a useful and flexible collection of REST resources. In the end, a RESTful design differs significantly enough from SOAP that these requirements may force you down the SOAP avenue, whether you intend it or not. For those with a choice, hopefully this provides a starting point.

Resources

The code for this article can be found here. The following resources are also required in order to compile and run the web application: