Featured in AI, ML & Data Engineering

In this article, author shows how to use big data query and processing language U-SQL on Azure Data Lake Analytics platform. U-SQL combines the concepts and constructs both of SQL and C#. It combines the simplicity and declarative nature of SQL with the programmatic power of C# including rich types and expressions.

Featured in Culture & Methods

The book Agile Leadership in Practice - Applying Management 3.0 by Dominik Maximini is an experience report of the agile transformation journey of NovaTec. Maximini shares his experiences from applying principles and practices from Management 3.0, success stories, failure stories, and learnings from experiments.

Featured in DevOps

Yuri Shkuro presents a methodology that uses data mining to learn the typical behavior of the system from massive amounts of distributed traces, compares it with pathological behavior during outages, and uses complexity reduction and intuitive visualizations to guide the user towards actionable insights about the root cause of the outages.

Give it a REST: Mark Baker on Web Services

Mark Baker is well-known in the SOA and Web services community because of his continuous efforts to promote an architectural style called REST (REpresentational State Transfer), criticizing many of the standards and specifications as being ignorant of what made and continues to make the Web successful. InfoQ's Stefan Tilkov had the chance to talk to Mark about REST principles, its benefits, and the relationship to Web services.

InfoQ: Can you tell us a little bit about yourself and your current work?

MB: My background is in distributed systems, more recently Internet scale distributed systems. I started my career developing software on a proprietary socket based infrastructure, then moved to DCE and CORBA, and have now settled on the Web as my infrastructure of choice.

Related Sponsor

MB: The first thing to understand about REST is that it's an architectural style, like "client/server", "pipe and filter", "publish/ subscribe"; an abstract description of an architecture not tied to any particular technology ... though it certainly shapes how one would use different technologies when used as a guide for designing or evaluating architectures. REST was defined by Roy Fielding as the architectural style he used to craft the early development of the foundational Web specifications, in particular HTTP and URIs.

InfoQ: What's the relationship between REST and HTTP?

MB: HTTP (1.1) is a standardized data transfer protocol. As mentioned, its development was guided in large part by Roy and therefore the REST style. For example, REST requires that components expose (via connectors) the same, uniform application interface, and HTTP accomplishes that by defining uniform methods such as GET and POST. That's just one example though, and there are many more aspects of HTTP which can be traced to REST. Roy's dissertation talks about some of them.

InfoQ: So a key distinction is that the interface is fixed when using REST, and application-specific when using something like CORBA or Web services?

MB: Absolutely. It's arguably *the* distinction.

InfoQ: What's the benefit?

MB: There's many benefits; simplicity, scalability, reliability. These are all architectural properties which REST has more of than SOA, and it's due primarily to the uniform interface.

InfoQ: But doesn't this just move the problem to a different layer?

MB: No, the problem is simplified because the number of moving parts has been reduced.

InfoQ: Roy's dissertation lists 4 key constraints: identification of resources; manipulation of resources through representations; self- descriptive messages; and, hypermedia as the engine of application state. The first one is handled by URIs; the second one by HTTP's uniform interface (GET, PUT, POST, DELETE). Can you elaborate on the third and especially the fourth constraint?

MB: Self-descriptive messaging is the single most important constraint of REST because it implies other important constraints, such as the uniform interface. What it means is just that every message needs to include all the information necessary in order to understand the message. Some of the implications of that aren't widely appreciated though. For example, it requires standardized methods, which is why an interface constraint isn't required. It also requires standardized media types.

"Hypermedia as the engine of application state" is simply a long-winded way of saying that REST clients make progress via links embedded in the data they retrieve. That sounds like an almost information-free statement, but it's not difficult to violate this constraint, and it can be quite costly to do so as well. Consider what would happen if a server decided that appending "/french" to some of its URLs would identify French-language equivalents of the identified resource. If it didn't include actual links to those resources in the representations it returned of other resources, and instead relied on the clients to know that rule, then that would violate this constraint. One obvious cost would be that search engines will not know that rule, and so those French versions of resources would not be indexed.

InfoQ: Why should anyone care about this?

MB: Care about REST? Because it's an architectural style which possesses many important architectural properties that improves reliability, scalability, and simplicity, as well as easing integration. These are all properties which Web services & SOA don't possess.

InfoQ: Why do *you* care so much about it? You seem to be on some sort of mission :-)

MB: Since I first started using the Internet 20 years ago, I always felt that it was inevitable that the industry would deploy a pervasive distributed computing platform that would enable the integration of business (and other) systems over the Internet. And I always felt that this platform would be distributed object or component based, which is why I first got involved with CORBA, because I thought it might be that platform. Then, after hooking up with some folks working on the Web in 1996, I began to realize that there was more to it than initially met my eye. This culminated in a 1998 discussion with Roy where I realized that the Web was, in fact, this platform that I'd been looking for all that time.

My "mission", if you can call it that, is to try to show others what I've seen; how the Web is actually an *improvement* on architectures like that of CORBA, DCOM, or even Web services. Obviously it's taking longer than I thought it would.

InfoQ: But how can the Web be an improvement upon Web services, when Web service build on the Web? Or don't they?

MB: Web services build upon HTTP, but they don't build upon the Web. The Web uses HTTP as an application contract which enables the loosely coupled exchange of documents between applications, while Web services uses HTTP as a bit pipe - as a transport protocol. Doing the latter instead of former means that you're starting from scratch, and not taking advantage of the existing network effects in the Web.

InfoQ: REST seems to have been reverse-engineered from HTTP. Does this really justify the claim that the interface uniformity constraint was really intentional? Isn't the HTTP interface specific to the hypertext domain, and the generalization just an afterthought?

MB: Insofar as REST incorporates some constraints which were already implicit in HTTP - like the uniform interface - I suppose that's true. At the time Roy began working with HTTP though, at least as I understand it, there were many problems with HTTP which he helped fix by using REST, so it wasn't entirely reverse-engineered.

InfoQ: Many claim there are many things HTTP is not sufficient for, e.g. reliability or transactions. Is REST intended for the simple use cases, and SOAP/WS-* for the complex ones?

MB: It's true that there are many things HTTP is not suitable for, but in general the SOAP/WS-* use cases are not amongst them.

What HTTP is *not* good for is fine-grained control-message style interactions, because it has a high per-message overhead, so I wouldn't try to implement the equivalent of TELNET with HTTP for example. It's also not good for - or at least optimal for - high throughput stateful messaging, again because of the overhead. There are other cases too, but I've yet to see a Web services scenario that didn't have a straightforward solution within the constraints of REST.

InfoQ: To take one specific example: How would you handle reliability? Isn't this something that every enterprise application needs, and WS- ReliableMessaging offers? What would be the REST way to do this?

MB: There's likely more than one way, depending upon your definition of "reliability". But REST does change the problem substantially because reliability is defined in terms of state transfer rather than message exchange; i.e. what's important is not "was the message sent?", but "was the state transferred?". An example implication of this difference is that we don't need an HTTP GET message to be reliable; because GET is safe, we only care that we retrieved a representation the state of a resource, and that may take any number of GET messages. Similarly PUT requests, though not safe, are still idempotent, so we know we can keep blasting them until we get an authoritative response. POST is not idempotent, so a reliable solution for it would look somewhat similar to a message-level reliability mechanism.

InfoQ: Do you think there's no value at all in the SOAP and WS-* stuff?

MB: Well, let's just say that I think there's significantly less value there than there is in standards like HTTP and URIs. And it depends on the spec too.

For what it's worth, I believed until quite recently that SOAP was salvageable as a generic XML envelope. But I think Atom has quietly usurped that role while SOAP was distracted being used for RPC.

InfoQ: Can you elaborate a little? How is a format for syndicating weblog contents comparable to a messaging envelope?

MB: Well, if you look at the SOAP 1.1 and 1.2 specifications, and ignore the specifications that have been developed on top (e.g. WS-Addressing), they really are simply an envelope for XML content. Sure, SOAP has features like a rich processing model with intermediary targetting, while Atom has a rather simple processing model. And Atom handles "packaging" natively while SOAP has solutions developed on top. But both are, at their core, generic containers for XML content.

InfoQ: One thing that seems to missing in the REST picture is a description language. What do you think about WSDL?

MB: I'm not a fan of WSDL, nor of a service description language for REST. The fundamental advance of REST over the styles of CORBA and DCOM is that all service interfaces are the same, so there are no differences that need describing.

InfoQ: This seems to be evading the question. While the operations might be the same, the parameters - or the data exchanged, if you prefer - is different. So there still seems to be a need for a description, at least for the data part.

MB: Sure, the data needs describing, but because REST uses self-descriptive messages it requires that the message itself describe the data. On the Web, that's done with a media type (using the Content-Type) header. REST also requires the use of standardized media types, so all the information that a Web services developer would place in a schema document (format extensibility, primarily), would be standardized in REST.

InfoQ: So to be clear about this - does this mean that if I want to develop a RESTful application, I can only use standardized message formats? Some custom XML format of my own would be ruled out?

MB: Strictly, yes, but remember that REST is an architectural style for very large scale systems. If you've got a proprietary data format that you need to use, say, behind your firewall, or even between two companies that long ago agreed to use this format, then you can still gain a lot by moving *towards* REST (by adopting the uniform interface, URIs for identification, etc..), but you won't quite reach it in this case because of the data. And there's nothing wrong with that per se. All it means, and all that REST is telling you, is that the data in these messages won't mean much to the outside world.

InfoQ: What about UDDI?

MB: I'm also not a fan of UDDI. A Web server serving up XML documents which contained links to other XML documents, perhaps on other Web servers, would be a far superior solution.

MB: They're fine, I suppose. "Web Style", at least, is well-defined because it was defined in terms of REST's own constraints. But Hi-REST and Lo-REST use some rather odd criteria that isn't strictly architectural. But I think it's good that people are talking about the architectural styles that they're using, even if it's a bit ad-hoc; it's a first step towards the industry as a whole moving towards having technical discussions in the language of software architecture.

InfoQ: What would be your advice for someone developing a "service" today? Is it possible to combine a REST approach with a more "conventional" Web services approach?

MB: It's technically possible, but from what I've seen of Web services toolkits (even those that claim to "support REST"), there's a very good chance that it'll be more trouble than it's worth. My advice would be to start with mature client and server Web frameworks in your language of choice, and to read up on what others are doing with REST/POX/XML-HTTP style services as replacements for traditional "Web services". Then apply that to the simplest possible problem you have, and once you've conquered that, move on to more complex problems.

InfoQ: Can you point to some good resources to find out more about REST?

And of course I'd be remiss not to mention Roy's dissertation itself. It's a little bit dry I suppose (though far less so than many other academic papers I've read), but very information-rich. It's one of those papers that you can read again and again, and get something new out of it each time.

Community comments

REST and SOA

Your message is awaiting moderation. Thank you for participating in the discussion.

Very interesting Mark, but aside from the detail REST which is excellent and informative, you seem to be perpetuating the standard confusion in here about what SOA is about.

RESTful services are still services. I agree in the main with your comments about Web-Services, and have never been a fan of SOAP, but SOA is not about Web-Services, it is about realising a documented set of re-useable services on an ESB.

Probably the best way of achieving this is to think about what your ESB needs to look like. - It will need fine grained secure, reliable messaging, which neither WS-Anything nor REST is good for, but JMS or other MOM products are,- It will need REST for the reasons you describe above,- It will need Web-Services for interoperability- It will need a control bus.- It will need a mechanism for identifying and choreographing these services.

So please, when you use the term SOA can you articulate what you mean by that term. If you mean SOA IS Web-Services then you have to find a different term to describe what (certain proprietary vendors aside) the rest of us mean by SOA.

SOA could just as easily be SA, only software naming history is on the side of SOA, nonetheless we are talking about Services Architecture, not Web-Services Architecture.

This is about architecture and not products, but if you need tools in this space, we at Base2Services are finding JBoss JEMS pretty compelling.

I use the term "SOA" as I believe most people use it - as a name for an architectural style, an abstract description of an architecture. In this case, a name for a style which espouses exposing data and functionality over a network via a variety of interfaces.

I try to avoid the term "ESB" because I still don't know what one is; I've seen wildly different products described as one, so from my POV is meaningless.

I don't believe that SOA = Web services, though I'm lead to believe that Web services follow SOA principles (at least I haven't heard anybody large outcry that they're not). I agree with this part of one of your statements; "SOA is not about Web-Services, it is about realising a documented set of re-useable services". I left off the ESB bit for the reasons mentioned above.

As for your five points about what's needed, I'm unclear about;- why JMS/MOM provides the kind of messaging you want, but REST or WS-* do not (or can't)- why "it will need Web-Services for interoperability"; on the Web, HTTP does a good job at providing interoperability- I don't know what a control bus means in this context- and for choreography, REST has hypermedia (or maybe that's orchestration - I can't get those straight).

Re: REST and SOA

Your message is awaiting moderation. Thank you for participating in the discussion.

I wrote a post on this topic. Here's an excerpt...

REST and WS can both be used to expose and consume services. The REST argument often implies that you have a choice, i.e. that you can dictate to your consumers that they will use REST. Or, that a service you need to consume will exist in a REST form. This usually is not the case...

SOA aims to become REST but will usually fall short

Your message is awaiting moderation. Thank you for participating in the discussion.

I'll put my naive hat on:

1. REST is unpopular among many architects not because people don't agree with the the principles, but because they don't like HTTP. They love their MOM. They love the flexibility of many message exchange patterns available. They want to standardize reliability & security across classic distributed systems AND services-oriented systems. They tend to look at problems "application first", "database first", or "objects first", not "network first". HTTP's statelessness is something to be worked around, with cookies & sessions, because they haven't been exposed to the principles behind its design in a way they can understand.

2. SOA doesn't preclude REST, in my opinion. I would liken SOA as a style that has a weaker set of uniform interface constraints. It still requires self-descriptive messages and also requires maniupulation of resources through messages. It doesn't emphasize universal indentifiers or hypermedia as the engine of state. Most advanced discussions of SOA out there emphasize the need for governed, if not uniform, interfaces and media types. This is unacceptable to many because uniformity requires a political investment that many can not or will not make. SOA is a style for people that will try to become more governed, aiming for uniformity, even if they fall short. REST is the ideal.

3. I think most architects will eventually recognize the biggest architectural challenge to SOA coming soon is the lack of uniform identifiers. If I have two services with their own sets of primary keys, how do they match up? How do I dereference them? Oops. URI's are a solid way forwards here but people seem to undervalue them and underuse them. SOAP/WS-* do not help matters. WS-Transfer kinda does.

4. The biggest challenge and most controversial constraint that precludes many SOA's from becoming RESTified is the role of hypermedia as the engine application state. Classic distributed systems, the application, or orchestration service, etc. is in control of all shared state. It's fully encapsulated. REST and the web says: "View Source". It keeps encapsulation for "resources" (which can be anything) but asks for openly viewable, standard representations, metadata, and methods for manipulating them.

Many in positions of power really do not like the web, for a variety of social, technical, and economic reasons. They may use web technologies used for cost avoidance & productivity, but want their systems to be as fat-client-like as possible. Pushing web-style inside a corporation can at times be a career limiting move. I do not know if this is temporary or permanent; I will observe that power tends to endure until a major crisis.