This is the blog site for the JBossESB team. Although we will predominately concentrate on the JBossESB project, expect entries on ESB and SOA concepts in general.

Thursday, August 2, 2007

How to design new services? Best practices.

If you have existing applications out there that need integration, then the problem space is fairly well understood. But what if you you are building a NEW application? Here is a question I received:

We have a big project here. It doesn't involve integrating existing software, but does involve a lot of parts that I think could best be written as web services so that they can be reused by other parts of the business, etc. Is it good to break up a project into services or is SOA better when you are trying to integrate things that aren't meant to go together or stuff that has remote components, etc. Basically, what I'm saying, is that I could put all this stuff into nicely organized java code, or I could break it out into services, and I don't know which to do.

I think the answer is that you should keep on writing your application like you always did. Write a tiered application, and make sure the business logic ends up in the (stateless) middle tier. Now this middle tier will exist of EJBs or POJOs. Next you will have to do two things:

1. Turn them into a service.2. Register the service with the ESB.

There are multiple ways to go about achieving the above:

EJB

Use EJB annotations to turn your POJO into an EJB. You can now register the EJB with the ESB by writing a custom action which calls the EJB. This is demonstrated in the bussines_service quickstart. The action receives a message, some objects can be pulled out and used to send to the EJB. The return value of the EJB can be used to update the message.

WebService

Add JSR-181 annotations to bring up a WebService. Now if you do this using JBossWS something cool happens. If the ESB runs on the same instance it auto-discovers these services and registers them with the ESB. Now can can send a soap message to a gateway and the ESB will forward this message to your WebService. So, for example, you can now send SOAP messages in a file to an ftp server. See the webservice_producer quickstart.

Seam

If you have used seam to build your app, you can take take a look at Michael Yuan's write up called "Seam-and-SOA" when he attended a JBESB training earlier this week. Also we have an integration example with the Seam DVD store that Burr created. Be on the lookout for this to become a quickstart too.

Spring POJO

Use Spring annotations to turn your POJO into a Spring enabled POJO. Now write a custom action to call the bean. See the Spring_OAP quickstart.

I hope this demonstrates the pattern. To be clear on why you should take the extra step to register your service to an ESB, this is to obtain capabilities like routing to multiple destinations, content-based-routing, message transformation, service orchestration (jBPM or BPEL), service-human workflow integration (jBPM), bringing up services over different protocols (ftp, file, etc). I guess that list is pretty well documented on the project page.

48 comments:

I think it's a bad idea for people to consider an ESB as an "application/service development/deployment platform". That problem is being solved all over the place already via JBoss App Server etc and I'd certainly recommend people to continue to use these tools to build and deploy their business logic, as they always have.

For me the big win with ESB is being able to "easily" expose/publish these services over multiple endpoints, making them more available. Then on the other side, making it "easy" for clients to make invocations (async/sync) on these service through the bus using the standards and tools most natural to them (SOAP, RMI etc over HTTP, FTP, JMS etc). Basically, the ESB is an integration platform and IMO, deploying business logic there is not a good idea.

I disagree :-) Check out my earlier entry to see what I mean. Sure, the ESB (or SOA infrastructure in our case) exposes existing applications, but *how* it exposes them and *how* you go about developing SOA applications, is critically important. Just because I expose an EJB as a Web Service using something like Java2WSDL doesn't mean I'm "doing" SOA. We have a real problem as an industry with the current tools, which are all object based. That has to change and encouraging people to think in terms of loose coupling and message oriented patterns is the way things have been shifting over the past few years. That will continue.

So although I agree with Kurt in that you should continue developing your higher level applications in the same way, how you expose them on to the SOA platform is very important. Ideally the platform will provide you with hooks, tools and advice on how to do that to get the most benefit from SOA. But even if it doesn't, there are enough patterns for developers to do it themselves.

And when developing your services: do not expose back-end implementation decisions on to the "bus". Work on the contract, SLA and MEPs. Time and effort spend here will pay dividends when/if you come to change the implementations.

Kurt, none of what you mention encourages or is related to service oriented architecture. Was that the aim? From an integration perspective you're right with what you say and we know we can do it with the ESB. But JBossESB is also supposed to be a SOA infrastructure: that's why we're building the SOA platform around it ;-)

An ESB is part of an SOA infrastructure. However, SOA is not simply a technology or a product: it's a style of design, with many aspects (such as architectural, methodological and organisational) unrelated to the actual technology. But obviously at some point it becomes necessary to map the abstract SOA to a concrete implementation and that's where JBossESB comes in to play!

At the end of the day though, IMO you need to give people (like the guy that asked Kurt the original question) a clear and unambiguous answer to those sorts of questions. Waxing lyrical about theories on SOA doesn't really help him at the end of the day ;-) Also, I see nothing in anything anyone said there that contradicts SOA or suggest tight coupling, RPC etc

Some guy over on WebServices.org said it best ;-) If all you want is EAI then we'll do that. If you want a narrow definition of ESB (say 3 years or so ago), then we'll do that too. But if you want an infrastructure that will scale with your requirements and support loosely coupled, long duration interactions using SOA principles, then we'll do that, because everything else is just a narrower definition.

Waxing lyrical about theories on SOA doesn't really help him at the end of the day ;-)

I'm not "waxing lyrically" and this has nothing to do with PhD's before you even go there ;-)

This project started as a SOA infrastructure and that's where it remains. There are patterns for SOA. There are books for SOA (that include those patterns if you can't find them via google). We should be giving encouragement to developers to write SOA applications from scratch if that's what they want (and that individual's question mentioned SOA). If they're not interested in loose coupling etc, then that's a different matter.

Also, I see nothing in anything anyone said there that contradicts SOA or suggest tight coupling, RPC etc

Have you looked at Java2WSDL and compared it with SOA principles? Just because you're developing using Web Services doesn't mean you're developing according to a SOA pattern.

The guy had a problem and he was looking for an answer!

Yes, and I disagree with the answer. For existing infrastructural investments it's correct. For SOA it's not the full picture. It says nothing about how to develop services and how to make sure they are loosely coupled, stateless versus stateful etc. It's very much just the traditional EAI approach, which is where ESB was 4 years ago. The world has moved on since then.

If we differentiate the ESB from the SOA [latform, then the ESB does very little: it delivers messages from endpoints to endpoints. But the platform adds capabilities, such as CBR, rules, jBPM, BPEL etc. Taken together, the platform should be encouraging a SOA approach to developing new applications. This does not equate to developing distributed objects, or distributed procedure invocations etc. It's service based: coarse grained.

Actually there are really a number of different levels at which you have to develop: the higher level business logic, which can be EJBs etc. But then you need to expose those backend implementation decisions as services. How you do that is critically important. If you do it in a fine grained manner (e.g., one-to-one mapping from service to object) then you're not building for loose coupled interactions, you're basically back to developing with a CORBA-like approach (and CORBA's pretty good for that). If you take a coarse grained approach and don't expose individual objects, then there are obvious advantages (and that's part of what SOA is about).

The service-to-object/backend implementation work is part of the governance and service lifecycle that you have to do when developing your applications. We don't have many tools for this yet, but that'll change with donations like that from Thomas Erl. At the moment all we can offer are patterns, which we should document.

Re the rest of it... I can't see anything useful there for Kurt's friend. SOA principals apply with or without an ESB or a SOA platform and have been around for donkeys ages, so I don't think Kurt's friend was asking a general question about SOA and SOA principals.

So what I saw there were questiona re "should I use webservices", "Is it good to break up a project into services or is SOA better when you are trying to integrate things that aren't meant to go together or stuff that has remote components", "could put all this stuff into nicely organized java code, or I could break it out into services".

So leaving the SOA principals to one side (coz they're wise principals that apply everywhere - cource grained, loosely coupled, message oriented etc), how should this guy develop he's application using the tools at his disposal, adhering to sound SOA principals etc? Should he implement the basic Services using Webservice, EJB, blah blah blah

Re the rest of it... I can't see anything useful there for Kurt's friend.

What, not even then "don't expose individual objects on to the bus" (I paraphrase) ;-)?

SOA principals apply with or without an ESB or a SOA platform and have been around for donkeys ages, so I don't think Kurt's friend was asking a general question about SOA and SOA principals.

I read it differently.

So what I saw there were questiona re "should I use webservices",

Yes, if you want interoperability across heterogeneous implementations.

"Is it good to break up a project into services or is SOA better when you are trying to integrate things that aren't meant to go together or stuff that has remote components", "could put all this stuff into nicely organized java code, or I could break it out into services".

So leaving the SOA principals to one side (coz they're wise principals that apply everywhere - cource grained, loosely coupled, message oriented etc), how should this guy develop he's application using the tools at his disposal, adhering to sound SOA principals etc? Should he implement the basic Services using Webservice, EJB, blah blah blah

Start by deciding whether or not SOA is the way you want to go. If not, then exposing individual objects and taking a more traditional closely coupled approach may make sense. If SOA is the way you want to go then start by reading a good book on SOA (there are a few).

From our perspective, we need to provide patterns that match how to do this from a conceptual level into the implementation, Actions etc. We're missing that. The current quickstarts aren't necessarily the best way of doing this education: we need more docs.

none of what you mention encourages or is related to service oriented architecture. Was that the aim

Yes this was my AIM. The question is what the impact of SOA is to the application developer. My friend understands the benefits of SOA, and he wants to move there. His long term planning horizon is 2 weeks out and he needs to figure out how to move away from doing JSPs! I think there are a lot of people in this situation! This post was on how to produce services, and how to hook them into JBossESB.

This is the first step. He already asked "so how do I talk to other services?". I guess I can work on another post entry there. Then once he is comfortable with getting on and off the bus we can start delving into what cool stuff JBossESB can do.

We need to make it EASY for poeple. Give straightforward guidelines so they can start to work towards true SOA. He's already looking into using Seam. You need to realize that he needs to deliver his new app in 2 weeks. Plenty pressure to learn new technologies that will pay off in the medium to long run. Already he is stressed about doing Seam just by itself. I'm pretty sure he won't read the article. But hey I'll ask him :).

Excuse the meta discussion here, but part of the issue you all had in understanding my question or the context of my question was that you didn't have all of the context around the question.

Kurt is right in that I am in the same position as many developers. I (and my quite small staff) have a good amount of experience developing sound, scalable Java webapps using the "traditional" frameworks like Struts, Spring, Hibernate, and standard J2EE patterns (DAO, etc.). The nature of our company (and so many others) is that there is never enough time to do stuff the right way. :P We are often given 2 weeks max to implement something that works and is scalable for over 1000 customers on the admin side and over 1MM visitors per day on the public-facing side. So, weighing being able to start or refactor a project into SOA vs. doing stuff the way we already know how is, as Kurt points out, primarily a matter of how *easy* it is.

I've been reading "Enterprise Integration Patters" [Hohpe] as well as the somewhat lighter "SOA for Dummies". I've read as much as I can online, including on the JBESB site, and have installed the ESB and gone through some of the sample apps and trailblazers. Most of the examples, particularly your "flagship" Loan Broker example, don't really talk about green field projects -- they talk about integrating a bunch of disparate stuff. My question to Kurt (and I'm hoping my words won't be parsed again) was basically, if you are starting a new project, that you are completely in control of, and have an interest in doing stuff the "Right Way" (tm), including SOA/ESB, where do you begin, what do you look out for, etc.

I read Kurt's response to my question this morning. Some of you argue that he didn't address the exact words in my post. But, as I mentioned earlier, he knew what I meant. And the value that more experienced developers and product developers and evangelizers can offer is exactly that -- reading into the question, understanding what real-life situations many developers are in, and offering solutions that are in line with your thinking/product offering, even if they are not 100% in line with the way that you hope everyone might use your product. From Kurt's response, I have a place to start, some things to explore, and probably many more questions to ask. But that's a hell of a lot further along than I was before I asked the question.

I've been reading "Enterprise Integration Patters" [Hohpe] as well as the somewhat lighter "SOA for Dummies".

Enterprise Service Oriented Architecture is another good one ;-)

My question to Kurt (and I'm hoping my words won't be parsed again) was basically, if you are starting a new project, that you are completely in control of, and have an interest in doing stuff the "Right Way" (tm), including SOA/ESB, where do you begin, what do you look out for, etc.

Understood and JBossESB material is lacking that regard at the moment. But doing it the right way (tm) really means starting off on the right footing and not trying to retro-fit SOA principles and practices later. It's a top-down and bottom-up approach that needs to run concurrently, with a lot of tool support and/or patterns to follow.

From Kurt's response, I have a place to start, some things to explore, and probably many more questions to ask. But that's a hell of a lot further along than I was before I asked the question.

Again, understood, but I want to make sure you start off right. If you're not interested in SOA and only in the EAI aspect of our ESB, then what Kurt suggested is fine. If you are interested in SOA, then it isn't. I'd be more than happy to take this up "off line" (via email) if you want - it'll be easier than blog comments ;-)

Naf, I'm going to try to find time to write something more about this, but for now my tip for getting on the right path would be to take on board what Kurt has said (that's essentially the top-down bit I mentioned earlier) but the bit about exposing this as a service (the bottom-up bit) is more complicated.

Try to separate your service design from the higher level implementation (EJBs, POJOs etc.) and decide what messages you need the service to accept to do it's job. The further from the internal implementation details you can abstract the service, the better. Plus, try to get away from synchronous request/response interactions if the time between sending the request and getting the response is longer than a "few" seconds. Correlated one-way messages, where you tie up the response to the request at the application level, give you better scope for tolerating failures and restructuring your interaction patterns later (e.g., deciding to have responses sent to a different recipient won't mean that you have to reimplement the service or clients).

As we've said in several places within this blog, none of this is really new. People have been practicing this for years in different areas. But where it works well is when you start out thinking about the entire architecture (top and bottom of the stack). Of course you can retro-fit this (that's where the EAI aspect comes in), but once you publish a service in to your environment it may be difficult/impossible to change it. So if you develop a service that isn't quite right (tm) with an aim to improve it later, it may well be that you can't change it.

There's a lot of pragmatism involved in all of this. As they say, your mileage may vary.