First of all, I should say that, although I work on OpenJDK/IcedTea for Red Hat, this blog represents my personal views and not necessarily those of Red Hat itself.

I originally thought of writing this after reading Joe Darcy’s response to a ‘Java Posse’ podcast (whatever that is). In the end, I settled for just adding comments to the discussion on his blog, and I still stand by the overall message I gave there; the biggest failure with OpenJDK at present is the size of community and more people from outside Sun would surely help. There are also major issues with the proprietary nature of JCP and TCK which I won’t go into here.

At the time, I was in a fairly positive mood about the project. It’s now nearly three years since Sun’s initial announcement of their plan to release their JDK as Free Software (25th of October, 2006) and the release of the HotSpot and javac code (13th of November, 2006). We’ve come a long way in that time; there are now a mass of Mercurial repositories for OpenJDK, and a number of external contributors, including myself, have push access to these. IcedTea has also grown and blossomed from its initial development merely as a means to build the JDK with a Free toolchain. It now includes the only Free Java browser plugin and Web Start implementation, and extended platform support via Gary Benson‘s work on Zero and Shark. A number of Sun employees have notably left the dark dank dungeons of proprietary JDK development and joined us in breathing the fresh clean air of Freedom; Kelly O’Hair, Joe Darcy, Mark Reinhold, Tim Bell and Alan Bateman have all being helpful on the various mailing lists, reviewing patches and fixing problems, and even joining us on IRC in some cases. We also have a little trio of former GNU Classpath developers now at Sun: Dalibor Topic (robilad), Christian Thalinger (twisti) and Roman Kennke (rkennke).

Unfortunately my reason for actually writing this blog now is rather sad. There are still a number of issues with OpenJDK that are still unresolved. Patches still take a very long time to be approved and committed; how long seems to be a mix of which area is being patched (and thus which project developers need to look it over) and just plain luck in some cases. At a guess, I probably have at least four or five patches still waiting on replies now. This has stopped me from creating any more because it just becomes too cumbersome to track them all. There is still a huge pile of patches in IcedTea, many of which need to go upstream; there’s not been that much activity on IcedTea over the summer, but it’s still probably enough for the number of new patches to outpace the rate at which older ones are going upstream. HotSpot patches for OpenJDK6 are completely stalled until the HotSpot 14 merge is completed. I don’t think most of this is the fault of the Sun engineers, but a problem higher up; they just aren’t enough people and they aren’t being allowed to commit enough time to working with the OpenJDK community. Another problem which reeks of higher management issues is the plugin debacle; long story short, don’t expect Sun’s plugin to be released any time soon and please continue to contribute your bug reports to the IcedTea alternative.

But a much worse problem, and the reason for my blog, is the continued lack of openness and the creation of an unfair playing field. A couple of weeks ago, I started looking at an issue with elliptic curve cryptography. It appears there is support for this in OpenJDK which is enabled when an optional provider is enabled and pointed at your system NSS library. This extra bit of configuration is pretty simple and we can automate it in IcedTea; I’ve done exactly this with a patch I submitted for review (yeah, seems we aren’t too fast either, but it was a long weekend in Canada…). This reminded me of some discussion of elliptic curve support I’d heard about with respect to OpenJDK, and, of all things, the changeset for this turned up around the same time.

Unfortunately, this wasn’t exactly what I expected. The original plan was for this to be a Java implementation of EC; not a good idea, in my opinion, as implementations already exist and the only problem with using NSS is that Sun can’t rely on it as an external library for their proprietary builds. In contrast, the current solution works just fine for distributions. But this isn’t what we got in the end. No, this changeset dumps a copy of a number of files from an unknown version of NSS into the OpenJDK source tree. They’ve all been altered with additional copyright notices and whitespace changes, and appear older than the system copy I have (we found a bug when that version is used, a fix for which is one of my pending patches). Not only does this mean that we now end up with a duplicate copy of NSS lurking inside OpenJDK, which is already outdated, but it seems there are also legal issues with EC support which means Fedora doesn’t ship it. So, one way or another, this is going to have to be ripped out.

The bigger problem here is that there was no discussion or public review of this fix. The changeset just appeared. Unfortunately, this isn’t a sole occurrence. It’s the case for most OpenJDK changesets. Reviews for patches from Sun developers happen quickly and internally, and we see nothing but a changeset. Had this patch been up for public review, this issues would have been discussed. Instead, the change is being forced on us without discussion and we have to work around it. I would suggest that this is a sign of a lack of respect for the OpenJDK community, but there is neither enough of a community nor enough awareness within Sun that these practices are wrong for this assumption to be correct.

This culture needs to change. We can never have a true open community around OpenJDK until everyone is playing by the same rules. This means public discussion on the mailing lists and equal say for all. Only then will this actually be a collaborative project and not just Sun engineers hacking in the open.

I think the issue is that the Sun hackers really don’t see that they are very much inside an ivory tower. And that perception is everything.

You can rationalize away every side-stepping of the community, not having transparency, failing to do things openly. And often every occurrence on itself isn’t the problem. It is the consistent pattern of having to track down after the fact why there was no feedback at all on your patch (or only once every other month), or no transparent, open, community involvement in every specific case that is creating this tired feeling.

The sad thing is that from a Free Software perspective Sun did everything right (for those parts of the code that they release under a free license at least). And I am extremely happy and grateful they did the right thing. But because ongoing development isn’t really an open, transparent, community thing, it is hard for even the biggest fan to be truly excited about collaborating on the future of the free java platform.

There is a kind of chicken and egg problem. To break this pattern you need more outside Sun hackers involved, but to get more of such hackers involved (and to try to keep those that already desperately try to contribute) you need a more open and transparent community in the first place, where even Sun hackers must obey the contributor rules and have all participants being treated as equals (if only because the non-Sun contributors don’t take it any longer…)

http://kennke.org/blog/ Roman Kennke

I totally agree with you Andrew. Problem is, many Sun developers are still in the closed mindset and do not fully realize that when they work on (Open)JDK7 (parenthesis because here lies the problem: Sun engineers work on JDK7…) that they work on an Open Source project, nor do they actually _know_ how to deal with this culture. They simply apply the existing processes (which have grown in closed culture) to JDK7, which results in what you experience…

Thanks for your comments. Unfortunately, it seems I’ve also disappeared from Planet JDK (I’m sure that list is much shorter than it was) though Mario has made a reappearance.

Mark, I think you’re right about the chicken-and-egg situation. We need a lot more outside interest (I guess external participation is not even at 25% of the level of Sun participation at present) but this won’t happen while it’s such a depressing affair.

Roman, sadly you confirm my own gut feeling about this endemic culture. That’s what I was indirectly referring to with ‘Sun hackers working in the open’. It’s not a community, but Sun engineers doing what they always did, but with the windows open so we can observe. I know Mark Reinhold is aware of this, and others as well, but I feel they just aren’t being given the resources or support from higher level management to change things for the better. I fear they got the good press from ‘open sourcing Java’ and now they feel it’s a case of ‘job done’.

Weijun

As for the code review part, sorry for not being public enough. In my case, sometimes it’s because of laziness, the bug report was originally sent to a Sun-internal mail list, and when my code review is ready, I simply reply-all that mail. Sometimes I only send the request to a single person because I think (s)he’s the one who can review it most quickly.

Weijun, thanks for your comments. I know it’s less intentional than it is a matter of existing practice. We’ve already seen good progress in making things more open and I hope this will continue.

Thanks very much for the link. I’d forgotten about the feed, and also about tagging my own contributions, due to delays I had initially getting access. I’ve subscribed to the feed and will try and mark up my own patches better.