Is Rubinius 2.2.3 production ready ?

4th January 2014

Ruby 2.1 was released this Christmas, great news everyone! It sports a better GC (RGenGC - gerational GC), hierarchical method caching, some small syntax changes and non-experimental rafinements. All in all one can expect 5% to 15% performance increase which is quite awesome.

As I was reading the comments in the hacker news thread related to the release, one of them caught my eye - we need JIT in the MRI Ruby VM. OK but looks like everyone forgets about Rubinius that sports for some time a LLVM-based JIT, native threads, low-pause generational garbage collector and almost perfect support of C extensions.

Yes we also have JRuby - it might just be the fastest implementation and when coupled with a server like TorqueBox is even faster, the main issue: some C libraries need to be swapped but as one can see later in the article this was just old thinking at play, clearly things are much better now as most gems do support JRuby without any issue.

So we have this awesome middle-ground so to say between Ruby VMs: it supports both C extensions without a problem (yet, empirically at least, some more exotic gems might fail to install) but somehow it gets ignored ?

The plan is simple, take one production Rails 4.0.2 app with all its dependences and convert it to Rubinius, install Puma and do some benchmarking then if all good deploy to staging.

The setup

Rubinius extracted most of the standard library into gems so in order to properly boot any Ruby script that uses them one needs this into the Gemfile:

gem 'racc'
gem 'rubysl'
gem 'puma'

Notes:

rubysl - is a rather cryptic name for ruby standard library gem

racc is a LALR(1) parser generator written by tenderlove - is also a
hard requirement for Rubinius or else no booting up Rails.

puma - well this is the best server choice for Rubinius as it supports
native threads

Gems that won't work with Rubinius (will update if I'll find more):

# gem 'oj'

Remember to comment them out or again Rails won't boot up.

For VM install and switching I'm using the good old RVM with the latest version of Rubinius 2.2.3 and Ruby 2.1.0.

On to the benchmarks!

OK for obvious reasons I can't share the source code of the app, at some point I might create a public repository with something meaty for
testing. These benchmarks are for a special case only and for some good fun also so before making any decisions based on them DO TEST first on your own.

The Rails app is actually an API so most of the important parts are disabled (i.e. Streaming DataStreaming Rendering RequestForgeryProtection) also the sprockets railtie.

ApacheBench config

I used the simple ApacheBench, Version 2.3 - clearly not the ideal tool for benchmarking (one should use siege or something similar) but
for a quick glance like this test it fits the job nicely.

The command to start it up:ab -n400 -c16 -T'application/json' http://localhost:3000/entries

Conclusion

The poor Rubinius performance might be related to the racc gem as it might be really slow as detailed in this Github thread

14 req/s vs 60 req/s (I disabled cache and the app produces lots of ActiveRecord objects that's why the numbers are rather low) makes Rubinius, for now, not a good choice for this particular Rails app.

Update

Thanks to headius I've revised the benchmarks:
- apparently my VM was accessing only one core (thus the initial abysmal performance of Rubinius and JRuby) - bumped to four
- updated all benchmarks and also added JRuby 1.7.9