This article shows the latest performance results of Apache Axis2 vs. Codehaus XFire, both Java implementations. The results demonstrate that modern Web Services engines can perform at very high transaction rates.

The blog entries have stressed the claimed improvement over XFire more clearly — for example, Sanjiva wrote,

Axis2 is consistently faster than XFire and is 50-70% faster as the data sizes get larger. Move over XFire, Axis2 is now the new hotrod.

XFire committer Dan Diephouse, when asked for a statement, pointed out that it’s more a benchmark of data bindings:

First, congratulations to the Axis team and their recent performance improvements in their last release. And I commend them for open sourcing their test suite so we can all run it ourselves. Second, this is really a benchmark of two different databindings - ADB vs JAXB, not XFire vs. Axis. If you look at the benchmarks they do seem to imply that XFire has less overhead when using the same databinding. But, the question many are probably wondering is - does XFire provide a higher performance databinding than JAXB? The answer is yes, JiBX. JiBX can provide significant performance improvements over JAXB as it does not use reflection, but instead does byte code generation to optimize databinding. I would encourage performance conscious users to try JiBX and to also run benchmarks for themselves.

The bottom line? Axis2 with JIBX is 2.5% faster than XFire with JIBX, and Axis2/ADB is 5% faster. These are averages - on some tests XFire was a little faster. I don’t want to make out that these are huge performance leads. They aren’t. Most user’s wouldn’t see any real difference between the stacks in terms of performance. In the article we were trying to point out that both stacks are fast and that SOAP is not slow. However, these numbers clearly show that there is no “sharp pointy plunger” in the vicinity of Axis2.

When you build a service, what is your goal? Is it maximum throughput for a single node, minimum latency per call, or ability to scale well across multiple machines or multicpu systems? Similarly, when you choose a data representation for the incoming XML content, is it based purely on how well the SOAP stack parses it, or how easy is it to work with?

I would also like to add that after looking at their benchmarks I think the latest claim about Axis2 being 2.5% faster than XFire with JiBX is not really true. I noticed that they added in a <soap:Header/> to every message. XFire is optimized for the non header case as that is what most clients will send. Sending a Header will trigger the creation of a JDOM Document to hold the headers in. This is why XFire looks slower in their benchmarks for small messages. For at least 90% of the people out there, XFire will in fact be faster.

If they want to look at scenarios that use headers, like WS-Addressing or user headers, thats great, and they might win out there, but this benchmark seems to be about the performance of moving data around without the bells and whistles.

Knowing that Axis2 had a bug for a long time where it wouldn't actually work without a soap:Header, I can only assume this was added so it Axis2 would actually work. (I don't know how you can even release a SOAP framework that doesn't work without a SOAP Header, but quality issues are something for another post...) Otherwise I would be quite suspicious that it was purposefully put there so XFire would look worse.

Who said benchmarks are relevant? :-) Personally I think Steve had some good comments in that area. It really depends on your application and what you're trying to achieve.

For StAX based web services frameworks (like XFire or Axis2), without lots of WS-* stuff enabled, performance isn't likely to matter in the context of most applications. And if it does matter, make sure you chose the proper databinding toolkit as that will have the biggest affect on your performance.

The people who should really care about web service performance though are the ones who are still using a DOM based web service framework (like Axis 1). By switching to a StAX based framework you're likely to see something like a 3-5x speedup (or something like 30x if you switch from rpc/encoded to doc/literal as well). This can make a significant difference in your application's responsiveness and load handling.

As pointed out in the original benchmark, both frameworks can handle a decent load. Its still not as fast as binary data over JMS, but for most applications it will work just fine.

The point I tried to make in the original article, which maybe got lost in the blogging, is that both XFire and Axis2 show that for most users SOAP is an acceptably fast solution. With older toolkits, that simply wasn't true. The other reason for benchmarking against XFire was that some users had somehow got the false impression that XFire was much faster than Axis2, which as you can see is not the case.

I still think there is lots more work to do here. Dims and I will be happy to improve the benchmarks anyway that Dan or any others like. In fact we are posting them on an SVN forge so that anyone can join in.

But the bottom line is that SOAP is no longer the major overhead it used to be.

The point I tried to make in the original article, which maybe got lost in the blogging, is that both XFire and Axis2 show that for most users SOAP is an acceptably fast solution. With older toolkits, that simply wasn't true. The other reason for benchmarking against XFire was that some users had somehow got the false impression that XFire was much faster than Axis2, which as you can see is not the case.

You also managed to accuse Hani of being racist. So, is it possible nowadays to criticize any software made by Sri Lankans without being accused of racism?

Regarding benchmarks: I do not care much about performance benchmarks, but I do care about software quality, development productivity and my own mental health. About half a year ago I had to choose one of the OSS web services frameworks and after evaluation of Axis/Axis2/XFire I was very happy to choose the latest as both Axis 'frameworks' were imho quite bizarre.

Have you participated in the Axis community? If so you'd know that the project is not a bunch of Sri Lankans. Even if it were, if you want to critize the project go ahead but why do you need to refer to developers' national origin for it?

As far as Axis/Axis2 being "quite bizarre"- obviously you are welcome to own opinion but I'm sure you know there are hundreds of thousands of people who don't find it bizarre and use it quite happily for lots of different things.

Have you participated in the Axis community? If so you'd know that the project is not a bunch of Sri Lankans. Even if it were, if you want to critize the project go ahead but why do you need to refer to developers' national origin for it?

Why are you asking me this question? I do not care about such thing as nationality. But if Hani wants to mention "Sri Lanka" in a negative context he should be free to do that. BTW the statement "all Sri Lanka developers are bad" is racist, while the statement that "particular Sri Lanka developer is bad" is not. Do you agree?

Sorry, I used "you" in the "one" sense. Didn't mean to imply anything about you in particular!

Axis2 is an open source project. IMO its not the right thing to pick on an individual developer of a project, whether the person is Sri Lankan, Japanese or American. In Hani's case he was picking on someone's ability to spell- while its great if everyone spelt the same but even Americans don't spell English correctly ;-) (or is it the English who haven't kept up with the times?!).

The world has declared that American English and the real English are the right spellings but if some other spelling is given its wrong. I'm very ok with that (its better to do things the "right" way) but at the same time you don't thump a young developer with all kinds of abuse because he spelt badly.

Maybe Hani was born into an English speaking environment. Many of the Sri Lankan developers who've participated in Axis2 came from very traditional homes with little English background. In fact, many spoke *ZERO* English until they entered university (where they are required to study in English). The right thing to do is to encourage people to improve their English, not ridicule them in public. If you knew the story of that particular developer you'd know he has already achieved far more in his life than most people.

That's what annoys me about Hani's racist nonsense. If not for those I'd never bother responding to his crap .. its just someone writing obviously intentionally inflamatory material for more PR. There's nothing novel in that strategy.

You know - there is a very large difference between being prejudiced and racist. Everyone has built in prejudices of some sort whether they choose to acknowledge them or not.

Cutting out even more b.s. - let's be real here. We've all been witness to the overwhelming amount of mailing list posts of "I have problem X - help me solve it now please thank you!" type items that have unfortunately built up a certain prejudiced way of thinking of certain groups of developers....

If you took that and said "all Sri Lankan people are stupid with software" it would be racist...If you took it and said "the majority of Sri Lankan people I've encountered appear to be struggling with some basic development concpets" you'd be prejudiced...But only in the practical / pragmatic / obvious sense. ...(and please don't start calling me racist as I have a few "Sri Lanken" friends and know how they feel about this already...)

Is your profile up-to-date? Please take a moment to review and update.

Email Address

Note: If updating/changing your email, a validation request will be sent

Company name:

Keep current company name

Update Company name to:

Company role:

Keep current company role

Update company role to:

Company size:

Keep current company Size

Update company size to:

Country/Zone:

Keep current country/zone

Update country/zone to:

State/Province/Region:

Keep current state/province/region

Update state/province/region to:

Subscribe to our newsletter?

Subscribe to our industry email notices?

You will be sent an email to validate the new email address. This pop-up will close itself in a few moments.

We notice you're using an ad blocker

We understand why you use ad blockers. However to keep InfoQ free we need your support. InfoQ will not provide your data to third parties without individual opt-in consent. We only work with advertisers relevant to our readers. Please consider whitelisting us.