Performance is important in a web
framework, even though it is certainly not the only consideration.
But anyway, it is a topic that deserves an in-deep analysis. This is
not a topic that can be reduced to just some few numbers, instead,
web frameworks try to make a balance performance and other aspects
like usability, extensibility, and much more.

In the last year (2011-2012), a
lot of performance improvements has been done inside Apache MyFaces
JSF Implementation (aka MyFaces Core). To see how good are the
improvements, a comparison was done against a similar Java Web
Framework like Wicket. In order to save some work, the reference
application provided in this link was used:

Fortunately, the
wicket guys have been so kind to updated the original application
running with 1.4.x to version 1.5.x, so we'll compare JSF against
two Wicket versions (1.4.20 and 1.5.5). A comparison against Tapestry
might happen later. In the end, the intention is to look at how JSF
works from a performance perspective, so Wicket will be taken only as
a reference point.

Checking the code
provided in that blog and the related data, some rather serious
problems were discovered. Anyway, the code example is good enough to
reuse it, while obviously fixing the detected problems. The updated
code, configuration used, detailed documentation and experimental
data can be found in the internet (github).

Keep in mind that
this blog entry is just a resume of the detailed document.

A
good benchmark should consider different aspects. So, for this
comparison, the following aspects has been checked:

Speed
benchmark: Check the speed of the code under different situations.

Memory
Consumption : Involves checking how much memory does the application
need to allocate for a specific task, and how the application
behaves under low memory conditions.

Session
Size : Check how much memory is used to store the session.

It
is quite obvious why these three aspects should be considered:

If
the framework is fast it will process more requests with less CPU
consumption.

If
the framework requires less memory, less time will be used in
garbage collection and the framework will perform better under
stress conditions.

If it uses less
memory as session storage, it will be able to serve more users with
the same memory size, and if the session is stored somewhere
(database, file, memory cache... ), less time will be used in
processing that information (usually serialization/deserialization
and dealing with I/O operations).

For the guys who are
impatient these are the results: NOTE: This
information is just related to the test application used and its
results cannot be generalized. If you want to see the underlying
discussion please read the detailed document.

Comparing MyFaces
and Mojarra in the graph, you can see MyFaces 2.1.7 has become the
fastest by a wide margin (more than 40%!). Comparing Wicket 1.4.20
and 1.5.5 you can see 1.4.20 is faster. In this case, a better
programming model in 1.5.5 has cost some speed. In Wicket, disk
storage (default) and http session storage has different trade-offs.

In
this case, MyFaces uses a lot less memory to do the same task and the
memory used between Mojarra and Wicket 1.4.20 is more or less the
same. Wicket disk storage requires more memory than session storage,
because the pages need to be saved and restored from disk. Most
memory is allocated by Wicket 1.5.5, but it is interesting that there
is no significant difference between disk storage and session
storage.

From a memory
perspective, MyFaces is the winner but note that allocating more or
less memory doesn't say much because part of this allocated memory
could have a short life, but it is relevant if the web server has not
a lot of memory. In a few words, the less bytes and objects allocated
the better.

JSF average view
size is very small compared to the user session size, imposing a
reasonable overhead. In this case, MyFaces do the best job, keeping
the state size very small, and the important part here is note that
the state overhead for JSF grows very slowly. In Wicket case, since
it stores the whole page into the state, it easily grow more quickly.

CONCLUSION

Definitively JSF 2.0 was a big
move in the right direction. JSF session size overhead is
significantly lower than Wicket. MyFaces Core 2.1.7 looks very well
from a performance perspective against a web framework like Wicket.
In the end, MyFaces provides the better trade-off between CPU usage
and session size. The bet of Wicket on disk storage is good, but the
JSF bet on partial state is even better. MyFaces also provides the
lowest memory usage, which allows it to perform better in
environments with limited CPU and memory.

Comparing Wicket 1.4 vs Wicket
1.5, it seems that the improvement in its programming model is taking
a toll. In my opinion it is something completely reasonable. A good
web framework needs to balance different aspects in order to keep
evolving over the time. Including more assisting code usually means
lower response times, but it could improve developer productivity.

This is not the end
of the story. Web frameworks will keep improving and the hope is this
information can be useful to find new ways to enhance them (community
over code is the Apache way). Remember: performance is just one
aspect that you have to consider when choosing a web framework;
usually it is necessary to strike a balance between this and several
other aspects.

11 comentarios:

I'm Martin Grigorov from the Wicket team.I was the one who helped you with the migration of the app to 1.5.5. I was also the one who asked you to do this perf benchmarking together, to make it in the Apache way - open. By 'open' I meant by creating an public repository and communicating via open channels - mailing list, forum, etc what and how should be done to make this perf benchmark useful for both of our communities. But you just ignored me once after receiving the upgraded app...

Well, all I can say is: Congratulations for the good product you have!

Well, that's the idea to upload all doc in github, which is a public repo where anyone can create branches, make updates and do it in a more colaborative way. I did not ignore you, just I had too much work, and stuff like that needs to be done right from the start. Thanks for the app.

I'm very surprised with the difference between Mojarra and Myfaces, it's very interesting. I always thought that Mojarra impl. was far better than Myfaces one. Well, Myfaces team deserves congratulations.

I don't know if you've already read this blog post about stateless jsf, but probably it's possible to improve jsf performance when using this stateless approach. http://industrieit.com/blog/2011/11/stateless-jsf-high-performance-zero-per-request-memory-overhead/

In fact, Mojarra was faster than MyFaces in versions before 2.1.5, because the priority in MyFaces side has been always get the code into an stable state, which means resolve "the most difficult and craziest problems" found by JSF users when they start to mix code.

I have read the blog related to stateless jsf carefully and this one too:

because I think both are very useful feedback from the community. It is too hard to explain the details behind the improvements, but the interesting part is it is possible to have a good performance in JSF without get into "stateless" mode.

I think it is possible to get a better performance with a "stateless JSF", but with the changes done in MyFaces it will be a minimal gain. Note it is necessary to investigate it in deep, I still have some ideas left in my hand, but maybe it will be done for JSF 2.2. Help and feedback is always welcome in MyFaces Community.

If you compare with previous versions of MyFaces, the improvement was huge (very, very huge) in all aspects (speed, memory, session size, bug fixing, ...). MyFaces code has been always high quality stuff, it just needed some polishing. Is like a good wine, you need to taste it to enjoy it.

"... the memory usage of our application dropped enormously. From a staggering 50MB+ per session to almost none. We used to consume at least 1GB for every 20 users and that dropped to less then 200MB for any amount of users (<50 tested). ..."

I think the improvements in Partial State Saving are the best ones, more than the improvements in speed/memory. In few words, MyFaces puts a thin layer to get a minimal state and reuse all "stateless" part of the component tree, even when using dynamic facelets tags that changes the component tree structure (c:if, ui:include src="#{}", ...).

It could be, a hard coded servlet app could be faster, but note the tricky part in that case could be the "balance" between what the framework do for you and what you have to do when you build your custom web application. It is easy to fall into wrong conclusions looking only one aspect, but at the end it is a matter of "taste". I think in MyFaces there is a good balance and the code performs very well, compared with the functionality provided.

If you are using @ViewScope, it means you store the bean in the view, and if you are using server side state saving it means the bean is stored into session, so basically your session will be bigger or smaller according to what you put there. But the interesting part is note how tiny is the overhead of MyFaces into the state, and in that way, into session. In this case you should take a look at your code, and keep into view scope only what you really need, and instead use request scope to fetch in each request what can be retrieved someway.