Featured in
Architecture & Design

Mini-talks: The Machine Intelligence Landscape: A Venture Capital Perspective by David Beyer. The future of global, trustless transactions on the largest graph: blockchain by Olaf Carlson-Wee. Algorithms for Anti-Money Laundering by Richard Minerich.

Featured in
Process & Practices

In-App Subscriptions Made Easy

There are various types of subscriptions: recurring, non-recurring, free-trial periods, various billing cycles and any possible billing variation one can imagine. But with lack of information online, you might discover that mobile subscriptions behave differently from what you expected. This article will make your life somewhat easier when addressing an in-app subscriptions implementation.

Featured in
Operations & Infrastructure

Mini-talks: The Machine Intelligence Landscape: A Venture Capital Perspective by David Beyer. The future of global, trustless transactions on the largest graph: blockchain by Olaf Carlson-Wee. Algorithms for Anti-Money Laundering by Richard Minerich.

Featured in
Enterprise Architecture

Mini-talks: The Machine Intelligence Landscape: A Venture Capital Perspective by David Beyer. The future of global, trustless transactions on the largest graph: blockchain by Olaf Carlson-Wee. Algorithms for Anti-Money Laundering by Richard Minerich.

Microsoft Surpasses Java's Dynamic Language Support?

When .NET was first released in 2000/2001, the Java community considered it a "clone" of Java, both language and standard library. Comparing simple code samples surely support this impression. However, MS profited from many years of experience with Java, and managed to solve some issues that Sun only now realizes as problems. The impression that the .NET and the CLR are evolving faster than Java is not lost on the Java community. Neil Bartlett:

I think it’s clear that Microsoft are innovating faster on the CLR. For example: LINQ is an extremely powerful new feature (based on Haskell monads, I might add); Generics were supported earlier and better in C# than in Java (both flavours were inspired by Haskell’s polymorphic type classes… hmm!); the CLR has better support for multiple languages than the JVM; and now it has the DLR, which is probably two years ahead of the JVM being able to offer anything comparable.

Other examples of this are modularization and versioning, which.NET solved by choosing the assembly, a collection of classes, as the basic deployment unit. Assemblies are equipped with metadata such as version information, unlike Java's Jar file which lack versioning metadata. This is troublesome for increasingly large applications, which load many libraries. OSGi now provides a solution for this, Sun is busy adding something similar to Java 7.

The Java language keeps on catching up with C#, adding features such as Generics and features such as AutoBoxing, Enumerated types or Annotations. C# now has anonymous expression support, which forms the underpinning of the LINQ technology. LINQ can be thought of a statically typed query language for many different types of data sources, such as XML, relational databases, but also arbitrary object graphs. The Java space, meanwhile, debates language minutiae such as language support for properties and which of four types of anonymous function to include in the language.

With the release of the DLR, Microsoft has moved ahead again, this time in the area of support for dynamic or scripting languages on the CLR. The Java space does not have a comparable effort going on right now. Miguel de Icaza, who started the Mono project, a clean room implementation of .NET, summarizes the features of the DLR:

A shared type system for dynamic languages.

A shared AST that can be used by language developers to create new dynamic languages.

Helper/utility routines for compiler developers.

A common hosting interface, to embed the general purpose scripting language interface into your program, and allowing developers to extend applications with one or more dynamic languages.

Console support, they even have a simple console interface for doing interactive programming.

The wrapper approach can also have deeper problems. One challenge is just to figure out what object to pass.For example, if Python has a PyString and it calls a C# function that expects an Object, should it pass thePyString or should it unwrap it into a String? These kinds of subtle type issues never have a good answer.Even worse are the nasty problems that can be caused by loss of object identity when objects are silently wrapped and unwrapped behind the programmers back.

Java strings passing into Ruby code will be encoded as UTF-8, with the implication that you should expect to be working with UTF-8 byte[] in the receiving code

Ruby strings passing out of Ruby into Java libraries will be assumed to be UTF-8, and the resulting string on the Java side of the call will reflect that assumption

The Java space doesn't really any of the mentioned items, except for the hosting interface, which was added in Java 6, under the name of JSR 223. This is basically just framework to add new language runtimes and initialize and access them in a standardized way.

JSR 292 is an initiative I started to address these problems. I hope it will continue in my absence, but it will be some years until it comes to fruition (if ever). It is frankly much harder to add support for these features to a JVM then to stabilize Strongtalk.

Note: Strongtalk is a Smalltalk implementation, whose VM is the base for the Hotspot technology, that has been shipped with Sun's JVM for a long time now.

Creators of dynamic language engines are in the business of taking, say, Ruby code and turning that into Java bytecode. When today's JRuby engine tries to convert that method call into bytecode, it has to create a synthetic interface to represent the return type. That isn't an interface a developer creates, but is one purely created by the JRuby engine [so] that it can take that method invocation and [turn that into] the bytecode. And that was just the return type—the same holds true for method parameters and exceptions.

JSR 292 removes the need for that synthetic interface. Today, dynamic language interpreters must output the methodinvoke bytecode, even when interpreting, say, a piece of Ruby code. Tomorrow, with JSR 292, they will be using the invokedynamic version. It will streamline the engine implementations because a lot of today's engines are worrying about creating new synthetic types and doing lots of book-keeping: when a method is called in seven or eight different places, they have to re-use that synthetic type all over the place.

It is important to note, that these changes will go into the JVM Specification, which means that they'll be hard coded and will not be easy to update in the future. A library based approach has the advantage that if a better way of handling these systems is found, it can be used immediately. The JVM based approach will remain the same for a long time, because JVMs tend to have a long shelf time (as a reference: Java 1.3 is still in use in companies). It also remains to be seen whether the JVM can actually make use of this bytecode and really improve the speed of dynamic method invocations.

Another issue is the official support and backing of JVM based languages. Currently, JRuby has two of its developers on the Sun payroll. One of them, Charles O. Nutter, has started to reach out to communities such as Jython and Groovy, and it remains to be seen if these efforts will start. Considering that MS has teams closely cooperating on IronPython, IronRuby, and the JavaScript and dynamic VB support, it has a certain edge here.After all, the DLR is a product of different teams sharing their experiences and factoring them out into a common library and knowledge base, whereas the JVM based language teams often have to relearn important lessons.

To add some perspective: Microsoft's DLR must still prove itself, at the moment it is available online together with IronPython. IronRuby hasn't been released yet, and it's real world speed and general interoperability remain to be seen. Java still has the benefit of being viewed as more open and is available on more platforms than .NET, although Miguel de Icaza seems convinced that Mono can ship support for Silverlight by the end of the year.

May 10th update: Added a "?" to the end of the title to reflect the lack of consensus on this issue that came out in the discussion thread - Floyd.

Wow Werner, I kinda expected better from you, unless of course your goal was to invite flames from many communities. To say that the CLR is ahead of the JVM for dynamic language support ignores so many details...

- How many languages are implement for the JVM and for the CLR? How about multiple implementations of those languages? JVM wins hands down here.- How much of DLR is finished and being used by many languages? Only a handful of languages are actually based on the DLR...fewer even than Microsoft's previous attempts to provide language-agnostic interop layers.- When I told you the JRuby JIT didn't handle set_trace_func, I also told you it would be easy to add. I chose not to do it since most people weren't running with trace functions and JIT at the same time. It's like one line of code, man...you can't characterize the JVM or its bytecode as being some critical blocker here.- IronRuby is probably one notch above vaporware here...and already it's introducing gross incompatibilities like require returning component objects and initialize being a callable method to set those components up. We're a few weeks into IronRuby and already they're doing things that will damage support for popular Ruby apps and libraries. That doesn't bode well.

DLR seems to be little more than IronPython's underlying libraries pulled out into a reusable form, which although admirable is certainly not the way I'd approach implementing a common runtime for dynamic languages. To begin with, it takes only one (or a few) language's perspective into consideration...and by most accounts, Ruby is not Python. Second, signs point toward it forcing languages into incompatible boxes from which they'll never escape. DLR is early research and headline-fodder at best right now. All due respect to the guys working on it...it's a damn hard problem to solve...but articles and headlines claiming CLR has suddenly surpassed JVM for language support are absurd.

And then there's the type interop of which you speak, and try to make JRuby an example again. JRuby supports a different string representation for a key reason: we feel that compatibility with the language and its libraries is more important than breaking the language to accommodate the limtations of the platform. Our approach may be wrong, but it's certainly captured the hearts and minds of many folks in the Ruby community. I'd wager that the largest reason why projects such as Jython haven't gained as much mindshare is because of continuing incompatibilities with the core implementation...incompatibilities that prevented key apps and libraries from working...incompatibilities which Microsoft seems to happily swallow and propagate to other languages through the DLR.

So unless your post was intended as a gross misrepresentation of the facts, I'd say you have some research to do.

Wow Werner, I kinda expected better from you, unless of course your goal was to invite flames from many communities. To say that the CLR is ahead of the JVM for dynamic language support ignores so many details...

What communities? Sun developers like you?

- How many languages are implement for the JVM and for the CLR? How about multiple implementations of those languages? JVM wins hands down here.

That's just plain wrong. There is a website that lists lots of so-called "languages", but in reality that list doesn't amount to much. What JVM languages are truly usable as of right now? Groovy, Scala, Nice (if you count a defunct project), JRuby, CAL (a Haskell like language). What else? It's pretty embarrassing, but not surprising, considering that Sun has always had the attitude that Java is the only language you need....well, until recently.

There's no comparison. .NET wins hands down. Sun never understood enough to to be able to mentally divorce the Java language from the rest of the Java stack.

- How much of DLR is finished and being used by many languages? Only a handful of languages are actually based on the DLR...fewer even than Microsoft's previous attempts to provide language-agnostic interop layers.

As opposed to absolutely no equivalent in Java land?

- IronRuby is probably one notch above vaporware here...and already it's introducing gross incompatibilities like require returning component objects and initialize being a callable method to set those components up. We're a few weeks into IronRuby and already they're doing things that will damage support for popular Ruby apps and libraries. That doesn't bode well.

One step above vaporware is a wrong statement to make.

DLR seems to be little more than IronPython's underlying libraries pulled out into a reusable form, which although admirable is certainly not the way I'd approach implementing a common runtime for dynamic languages. To begin with, it takes only one (or a few) language's perspective into consideration...and by most accounts, Ruby is not Python. Second, signs point toward it forcing languages into incompatible boxes from which they'll never escape. DLR is early research and headline-fodder at best right now. All due respect to the guys working on it...it's a damn hard problem to solve...but articles and headlines claiming CLR has suddenly surpassed JVM for language support are absurd.

Why, because you're a Sun employee and can't handle the truth. I'll leave out the dynamic language support comparison for the moment, but the CLR is way ahead of Java on the number of usable languages that is has.

So unless your post was intended as a gross misrepresentation of the facts, I'd say you have some research to do.

Charles, it's you the one that needs to do some research here. I guess nobody is surprised that a Sun employee is going to be throwing out lies, but it's pretty embarrassing that Java has fallen so far behind than the CLR in so many areas.

My only grip with this "article" is the title, which is clearly done in that way to attract maximum attention for InfoQ, which I find disappointing and slightly cheap. Otherwise its an interesting read (Charles also has some gripes with the content I guess).

>What communities? Sun developers like you?2 minutes of research will show Charles was working on JRuby (with many others) long before Sun took it on (now he has spare time ! yay !). Now JRuby is used fairly wideley, there are even commercial products built on it. I guess I share his gripe that a headline like this is really touting research ware (or vapourware if you want to by cynical about it) - which dilutes the value of the research, and also the work and expertise of people like Charles.

CLR is great, the DLR looks fascinating for sure, as an R&D effort. Its all good, and competition is good.

>What communities? Sun developers like you?2 minutes of research will show Charles was working on JRuby (with many others) long before Sun took it on (now he has spare time ! yay !). Now JRuby is used fairly wideley, there are even commercial products built on it. I guess I share his gripe that a headline like this is really touting research ware (or vapourware if you want to by cynical about it) - which dilutes the value of the research, and also the work and expertise of people like Charles.

And what does that have to do with the original question of "What communities?". You mean InfoQ should watch out not to offend the JRuby "community" because the article says that Microsoft is ahead of "Java" on the dynamic language front?

With the release of the DLR, Microsoft has moved ahead again, this time in the area of support for dynamic or scripting languages on the CLR. The Java space does not have a comparable effort going on right now.

Yes, they are in the process of adding support for them in Java 7.

JVMs tend to have a long shelf time (as a reference: Java 1.3 is still in use in companies)

The same can be said for .NET (.NET 1.1 is still in use in many companies).

A library based approach has the advantage that if a better way of handling these systems is found, it can be used immediately.<quote></quote>Not immediately, you have to distribute the library and do the applicable code refactorings.

The same can be said for .NET (.NET 1.1 is still in use in many companies).

I have no statistics about 1.1, but if I had to guess which version is the dominant .NET version in live deployments by far - I'd say 1.1. The same thing will happen with other .NET versions - very slow transition.

Transitioning from 1.1 is not actually plug and play. I don't know how bad it is from 2.0 to 3.0, but if it is anything like 1.1->2.0, we won't be doing it in the next 5 years, and who cares about the versions after that, 10 years maybe.

This is a great article, but why post with a title that deliberately presents an obviously subjective opinion as fact? Floyd, where are you man? I expect better from you guys. Don't let InfoQ become another TSS, please!

Flame me all you want, BTW, but I won't be back to read further comments after posting.

I don't think this article is a dig against, JRuby - after all, I use JRuby and do throw bug reports and - possible - features your way, so I'm certainly not biased against JRuby.

With that out of the way, your questions:Number of languages on the JVM:Yes, I know about that old list with 100+ languages,all of which have something to do with the JVM:www.robert-tolksdorf.de/vmlanguages.htmlMost of them are research toys and pretty much dead and not maintained anymore. The ones that are alive, can be grouped into Java preprocessors, or other Java derivates. There are a few dynamic languages, like Jython, in there. But: you also know how Jython has fallen back in it's support of the official Python - I think we can agree on that. There are folks working on it - as far as I can tell, but let's see what comes of that. And after all: go back two years and look at that list, find JRuby: think of JRuby 2 years back. Would you call that a useable tool? You obviously know how _much_ work went into making it into something that actually works like Matz' Ruby. So... with that eye, look at all the other languages in that list, who have much fewer developers, much less public interest, much worse PR and hence smaller communities. JRuby has now 2 (or three, I guess, as Ola works on JRuby at TW too) paid developers, and there's still much work to do.What I'm - rambingly - saying: the list of JVM languages is long, the actually usable language list is shorter.

How many languages on the DLR:4 languages (IronPython, IronRuby, VBx, JavaScript), at various levels of completion. Counter question: how many languages use invokedynamic to improve their speed? - Nonsense question of course, as no one can even explain what the thing is supposed to do ... sure I know Gilad's slides and other hand waving... but no one has a concrete, detailed explanation how exactly it should solve all problems. DLR is here now, it has momentum (paid developers), will be developed openly (as it seems) and will gather experience with speeding up the implementations. invokedynamic is a mystery... and you say so yourself:headius.blogspot.com/2007/01/invokedynamic-actu...This is what I was trying to say here... Sun's a lot of talk, but MS actually _shows_ the code.

set_trace_func:The fact that 1.0 is near, and the JIT generated code doesn't call it is a bug - I probably should have filed it. If I use a debugger, and happen to have the JIT turned on, I still want the debugger to work correctly. The fact that you can add it easily is great, and if it happens before 1.0 the issue is settled. BUT: IronRuby already compiles down to CLR IL, and already has debugging information in it - despite the whole thing being only a few months old. This is another thing I was trying to say: JRuby, XRuby, Jython, Rhino, etc etc etc are different communities and they all have to do these things over and over. Remember the recent discussions about getting better debugging support in JRuby (jruby-debug)? IronRuby, as it does the same as all the DLR langs, already has this, mostly because it can use the libs from IronPython and company.

IronRuby:I don't know how IronRuby is today, but John Lam keeps on stating that he intends to be compatible. Let the .NET Ruby community keep him honest... they won't like incompatibilities. The guys who work on the Google SoC RSpec projects (I have an upcoming interview with them here on InfoQ) will surely provide some helpful tools for shaming him into making IronRuby compatible - in case he or MS doesn't.And please also remember: JRuby also has it's incompatibilities, like no Continuation support, or things like subtle differences in Regexes (which, I believe are solved for 1.0). I _know_ the reasons for them, and they're good reasons, but still: it behaves differently from Matz' Ruby. Same standards for everyone.

As for languages: they have different languages, even VB which, for instance, needs method dispatch to be case insensitive, etc. And all this is available in a library that's available and developed online. invokedynamic's behavior is not known yet, will be cast into the JVM/JVM Spec, etc, so Sun has exactly one shot at getting this right. If you have confidence in that they get that right: fine, we'll see about that in a year or two. For now: invokedynamic is vaporware... actually it's even a vapor idea, so MS has an advantage as it has already shipped some code, as alpha or beta as it might be.

I give you the string argument; your (or the JRuby's team) decision was certainly the right way to go.

As for flames: this was not intended to ignite flaming, I actually spent a lot of time making it as neutral as I could.If the Java community feels upset... well, I've been in the Java community for 10 years, and being upset is what Java community members do best. Whether it's a C programmer claiming that Java is slow, or MS releasing .NET, or DHH saying nasty things about Java web apps... there will always be people taking out the pitch forks and torches to riot.

@Crazy Photon: About metadata in JAR files:You're right, about the manifest file. But this sentence was trimmed to much and lost some of my argument. The version number in the JAR is absolutely worthless. If you have code that wants to use, say, JDOM 0.9, then it's nice if the JAR manifest contains the version number... it won't help you if JDOM 1.0 is also on the classpath and happens to be loaded earlier.With .NET, you can specify which versions you want to use. In the Java space, one solution right now is OSGi, which allows you to have different versions of the same library in the same address space, and your bundle can specify the exact version it wants to use.

If you've ever worked on a large, extensible application (non-JEE), like Eclipse, you'll know that versioning like this is crucial.

So, I'm not going to go through your answer, just provide a few small comments. First, I agree with other commenters that the article, and especially the title felt very subjective and flamebaity. This has nothing to do with the Java community, it's about the style of your writing in this article.

Secondly, regarding incompatibilites: there is a huge difference between incompatibilities that are caused by the implementation difficulties (we _could_ provide Continuation support, but that would make JRuby worse in many ways, since we would need to get away from native threads. .NET can't possibly support this either). The compatibility issues we are fearing from IronRuby looks like they are on a totally different level, changes which have been introduced with no justifiable implementation problem being the cause. And, it's also a difference between features people use all the time (like require and the top level scope), and things which isn't widely used (like continuations, and very obscure parts of regular expressions).

Microsoft's DLR is a step forward. How big is still unknown and no one will know for a while. There are several other languages i can think of that run on JVM.

LISPBeanshelljelly

Since I've written my own "LISP-like" interpreter, from my experience one need not add features to Java language or the JVM to support dynamic languages. Does it help? Yes, it helps, but it's an exageration to say adding things like invokeDynamic make a "huge" difference. It does make a difference if you want to generate bytecode, but I would argue why? If you can run interpreted and be just as fast or faster, the difference is purely a preference.

take JESS rule engine as an example. It supports CLIPS language, which is derived from LISP. JESS run wicked fast in interpreted mode without the support of invokeDynamic. my bias 2 bits

I take full responsibility for the title. We normally never state a subjective opinion as a fact, and its unfortunate that we've descended into a thread somewhat similar to the other mentioned sites. ;)

The title was my idea because I understood that the DLR did represent unquestionably better support for dynamic languages than what's in the Java platform (that's Java, not JRuby itself). Perhaps from this thread that was not a correct conclusion, so I've added a "?" at the end.