[JRuby] JRuby perf questions answered

(Sorry if this is a duplicate post...the original seems to have disappeared)

Many folks still seem to be in the dark about JRuby performance relative
to Ruby 1.8.x. I figured a short post with some basic info could start a
discussion for anyone wanting to know more.

Long story short, JRuby 1.1 is now generally faster than Ruby 1.8.6, and
for most execution benchmarks it is the fastest 1.8-compatible Ruby
implementation available. This is based on the YARV benchmarks, the
Alioth benchmarks, a number of benchmarks we created for JRuby, and a
few other external benchmarks we run from time to time. Many (most?) of
these are microbenchmarks, but a number are nontrivial code.

I expect we'll see Alioth updated soon after the 1.1 or 1.1 beta 1 releases.

What about Rails?

One of the biggest uses for JRuby recently has been Rails. Rails
performance is a more complicated thing to measure; there's a lot of
non-execution bottlenecks that get in the way. But both Ola and Nick
Sieger (also of JRuby team) have found that JRuby on Rails performance
is either very near or exceeding Ruby 1.8.x on Rails:

Rails is probably the ultimate real-world benchmark, and we're excited
to see that JRuby's starting to pass Ruby 1.8.x performance here as well.

I won't drag this on any longer, but I'd encourage you to try out your
own Ruby code and let us know how it performs. We've spent a lot of time
on performance issues for the 1.1 release, and have many future plans to
continue increasing performance. We want your input.

Many people believed we'd never be faster than the C implementation, and
many still think we're slower. Now that I've set that record straight,
any questions?

Now, I am thinking that jruby might work well if we know how to find the
sweet spot. Why the pre compiled version is so very much slower than a
home compiled version makes me wonder why they do not compile it better.
As things can be so radically different in jruby if only you know how to
tweak it perfectly is another thing that makes me think.

So, do you find your own benchmarks and work your compiler to make it
run at its fastest or is performance going to be "roll your own" for
those with insider knowledge?
--
Posted via http://www.ruby-forum.com/.

There's been reports that on server-class machines the startup time can
be as high as 5-10 seconds. The slow startup time is partially due to
the JVM and partially our fault (JRuby's complicated, yo).

As for Ruby 1.9...it's hard to make a comparison right now. Most of the
benchmarks we use to measure Ruby 1.9 (like those I described
previously) are usually a bit faster in Ruby 1.9. And of course almost
all of Ruby 1.9's benchmarks are fastest in Ruby 1.9.

Part of the difficulty comparing is that Ruby 1.9 contains a bunch of
changes to Ruby language semantics that we have not yet put in JRuby,
since we've focused on 1.8 compatibility. Things like:

- fast math operations, that speed up +, -, etc, but you can't override them
- constant-time case/when when literal values are used, but you can't
override === for those literal types

Here's A few numbers, for your edification. Some things are faster in
Ruby 1.9:

Lloyd Linklater wrote:
> There was a recent thread which asked how fast ruby ran for each user.
>
> http://www.ruby-forum.com/topic/125344#new
>
> Josip Gracin wrote:
>> Here's something interesting. The processor is Intel T7200 @ 2.00GHz,
>> Linux Ubuntu.
>>
>> 1. With ruby 1.8.5. which comes packaged with Ubuntu:
>>
>> Ruby 1.8.5 on i486-linux
>> It took 20.346013 seconds to run. 49149 iterations per
>> second.
>>
>> 2. With ruby 1.9.0 (2006-06-08) which comes packaged in Ubuntu:
>>
>> Ruby 1.9.0 on i486-linux
>> It took 23.875774 seconds to run. 41883 iterations per
>> second.
>>
>>
>> 3. With ruby 1.8.6 compiled from sources on the machine:
>>
>> Ruby 1.8.6 on i686-linux
>> It took 7.107875 seconds to run. 140689 iterations per
>> second.
>>
>>
>> Obviously, the target instruction set matters quite a lot. I think I'll
>> be compiling my own ruby from now on.
>
> Greg Halsey wrote:
>> windows xp. intel core2 1.86GHz
>>
>> JRUBY:
>>
>> C:\jruby\jruby-trunk\bin>jruby calculate.rb
>> 55
>>
>> It took 41.125 seconds to run. 24316 iterations per
>> second.
>>
>> Not so good. Let's disable ObjectSpace:
>> C:\jruby\jruby-trunk\bin>jruby -O calculate.rb
>> 55
>>
>> It took 11.109 seconds to run. 90017 iterations per
>> second.
>>
>> That was good. Now let's give it some more juice:
>> C:\jruby\jruby-trunk\bin>jruby -J-server -O calculate.rb
>> 55
>>
>> It took 5.75 seconds to run. 173913 iterations per
>> second.
>> --------------------------------------------
>> MRI:One-Click
>> ruby calculate.rb
>> 55
>>
>> Ruby 1.8.6 patch 0 on i386-mswin32
>> It took 15.828 seconds to run. 63179 iterations per
>> second.
>
> Now, I am thinking that jruby might work well if we know how to find the
> sweet spot. Why the pre compiled version is so very much slower than a
> home compiled version makes me wonder why they do not compile it better.
> As things can be so radically different in jruby if only you know how to
> tweak it perfectly is another thing that makes me think.
>
> So, do you find your own benchmarks and work your compiler to make it
> run at its fastest or is performance going to be "roll your own" for
> those with insider knowledge?

We have been running whatever benchmarks we can, and they're all pretty
generic. For all the numbers I've reported (and I believe most of the
numbers others have reported for JRuby), it's been JRuby trunk (soon to
be 1.1) with the Java 6 "server" VM. There's really no more tweaking
than that required to get the same performance.

As for the compiler...there's no additional user tweaking necessary to
make it run best; it should run its best all the time. The additional
tweaks you may want to apply would be using a different Java version,
tweaking the JVM or GC settings, and so on. Normal server tuning most
people won't need.

The reality with the JVM, and especially with JRuby, is that short tests
aren't going to show the real long-term performance. For most execution,
JRuby starts out interpreted, and the interpreter is slower than Ruby
1.8.x. But after running for a while, code will eventually compile to
Java bytecode (or you can precompile to skip that time). Then there's an
additional delay while the JVM itself eventually compiles that bytecode
down to platform-native code.

We expect that for JRuby 1.1, the main sweet spot will be on
longer-running apps e.g. for server-side apps or for heavy data processing.

Share This Page

Welcome to The Coding Forums!

Welcome to the Coding Forums, the place to chat about anything related to programming and coding languages.

Please join our friendly community by clicking the button below - it only takes a few seconds and is totally free. You'll be able to ask questions about coding or chat with the community and help others.
Sign up now!