Topics

Featured in Development

Peter Alvaro talks about the reasons one should engage in language design and why many of us would (or should) do something so perverse as to design a language that no one will ever use. He shares some of the extreme and sometimes obnoxious opinions that guided his design process.

Featured in AI, ML & Data Engineering

Today on The InfoQ Podcast, Wes talks with Katharine Jarmul about privacy and fairness in machine learning algorithms. Jarul discusses what’s meant by Ethical Machine Learning and some things to consider when working towards achieving fairness. Jarmul is the co-founder at KIProtect a machine learning security and privacy firm based in Germany and is one of the three keynote speakers at QCon.ai.

Featured in Culture & Methods

Organizations struggle to scale their agility. While every organization is different, common patterns explain the major challenges that most organizations face: organizational design, trying to copy others, “one-size-fits-all” scaling, scaling in siloes, and neglecting engineering practices. This article explains why, what to do about it, and how the three leading scaling frameworks compare.

JRuby 1.6 Released: Ruby 1.9.2 Compatible and C Extensions

The JRuby team has just released the final version of JRuby 1.6.0. One of the major new feature is a very high Ruby 1.9.2 compatibility (Encoding::Converter and ripper are not implemented). A lot of effort has also been invested into JRuby's Windows support, which is now a primary platform for JRuby and part of their continuous integration setup.

A new experimental feature in JRuby is the support for extensions written in C, which removes one last big obstacle in JRuby's adoption.

InfoQ had the chance to talk to JRuby's Thomas Enebo to get more insight: JRuby is now 1.9.2 compatible. Could you elaborate on how you manage to support different Ruby versions with the same codebase?

Currently, we share as much common code as we can between Ruby 1.8 and Ruby 1.9 source. By and large, the source is the same between the two versions.

In the cases where behavior is different, we use @JRubyMethod annotations to mark the different versions of the same method:

@JRubyMethod(name = "<=>", compat = RUBY1_9)

This spaceship operator is marked as being for 1.9 support. We have a second one marked for 1.8. For most methods which differ, the semantics are largely the same except for 1.9 having more options. We process those options and then call a common method which can handle the semantics properly for both versions of the method. We don't universally use this pattern, but it is a very common technique we employ.

We also have lib/ruby/1.8 and lib/ruby/1.9 directories which contain the .rb sources which correspond to both Rubys standard libraries. Having two copies of most standard libraries is a little inconvenient, but these libraries are all subtly different. We do what we have to, to get good compatibility.

InfoQ: A surprise was that you now support C extensions. How does this work? And do you support them on all platforms?

It is fantastic that Ruby Summer of Code happened last year. Not only did we get a bunch of Ruby programmers working on Open Source, but we got useful software to boot. One great example of this was JRuby's C extension support. Tim Felgentreff made an amazing amount of progress on this feature. Basically, all the common C extension API entry points are translated and compiled into something which Java likes: Java Native Interface (JNI). Each of the C extension methods have an implementation in JNI which knows how to call into our JRuby runtime safely.

The amazing thing is this seems to be working rather well. Much better than any of us thought was possible. Many of the common C extensions work without needing any source modification. We don't support 100% of the C extension API because the C extension API seems to be the entire MRI codebase. However, the strategic methods selected seem to work for a vast majority of C extensions out there.

At this point we support MacOS, Linux and Windows(32). During our follow up point releases, we plan on releasing all the other platforms we support. By downloading our source distribution, you can build cext support for your platform (if we don't ship the bits for your platform).

The two caveats worth mentioning: Thread-safety and Performance. Hitting a C library which is not thread-safe will obviously not work. MRI not having real native threads (or native threads+GIL) will not blow up. We will. Make sure you understand how you will be interacting with any C extensions you work with. I find this to be a major difficiency in MRI's C extension API.

Performance is not as nice as we would like and may not ever be stellar. The crosstalk between Java and a C library has some amount of overhead which might be really visible. Consider a C library which works with UTF-8 strings; then consider all Java Strings are internally represented as UTF-16. This means interacting with that C library will have lots of encoding/decoding between UTF-8 and UTF-16 characters.

InfoQ: The JDK 7 Developer Preview has recently been released. Does JRuby 1.6 already take advantage of any of the new features in JDK 7?

We do have support for invokedynamic in our codebase already for method dispatch. If you run JRuby using Java 1.7, you can run JRuby via invokedynamic. We also see huge future potential in invokedynamic once it starts getting performance-tuned. Currently, it is close to having parity in terms of performance compared to what we do with bytecode generation on Java 6. Over time, expect to see the speed increase and the amount of classloading we do go way down.

We also plan on using NIO.2 to get access to all the filesystem features which should have been in Java to begin with. This will push our boundaries of what we can do in JRuby without having to use JNI to get access to the OS's underlying filesystem APIs.

InfoQ: What do you have planned for JRuby 1.7?

We are still in the planning stages of 1.7 since we expect to have several 1.6.x point releases before then:

Java 7 APIs:
We are looking forward to Java 7 coming out this fall and we want to take advantage of all it has to offer. NIO.2, as mentioned above will be great. There is also some new concurrency (lightweight fork/join) and collections APIs we can take advantage of.

Our next-generation Internal Representation (IR):
We have been working on a new IR and a new backend runtime. It may not be done for 1.7, but we are continuing to make progress on it. This new system will be exciting because it will allow us to do more optimizations at a Ruby-level and then defer what is left after that to the JVM to do its magic.

Modularization:
Right now we ship jruby.jar and jruby-complete.jar. These jars contain everything you could want with JRuby. Unfortunately, this is sometimes more than you want. If you are on Android you do not neccesarily want to bootstrap the entire Ruby core runtime. If you are on GAE you don't really want to load anything which AppEngine cannot support (like File APIs). Modularization will have two aspects:

Source decoupling:
We want to make all core types and native extensions to be included/excluded based on a profile. This will allow people to build a subset of JRubys source.

Packaging decoupling:
We want people to specify which bits they want in their resulting .jar file. If you don't need any it lib/ruby/1.8/net/** then it will be easy to specify which parts to include and exclude.