Topics

Featured in Development

Alex Bradbury gives an overview of the status and development of RISC-V as it relates to modern operating systems, highlighting major research strands, controversies, and opportunities to get involved.

Featured in Architecture & Design

Will Jones talks about how Habito, the leading digital mortgage broker, benefited from using Haskell, some of the wins and trade-offs that have brought it to where it is today and where it's going next. He also talks about why functional programming is beneficial for large projects, and how it helps especially with migrating the data store.

Featured in AI, ML & Data Engineering

Katharine Jarmul discusses research related to fair-and-private ML algorithms and privacy-preserving models, showing that caring about privacy can help ensure a better model overall and support ethics.

Featured in Culture & Methods

This personal experience report shows that political in-house games and bad corporate culture are not only annoying and a waste of time, but also harm a lot of initiatives for improvement. Whenever we become aware of the blame game, we should address it! DevOps wants to deliver high quality. The willingness to make things better - products, processes, collaboration, and more - is vital.

Featured in DevOps

Service mesh architectures enable a control and observability loop. At the moment, service mesh implementations vary in regard to API and technology, and this shows no signs of slowing down. Building on top of volatile APIs can be hazardous. Here we suggest to use a simplified, workflow-friendly API to shield organization platform code from specific service-mesh implementation details.

IBM and Red Hat to Vote "No" on Java Modules (Jigsaw)

InfoQ has previously reported on the developing situation regarding JSR 376 - the Java Platform Module System, commonly called "Project Jigsaw". Now, in a highly unusual move, IBM and Red Hat have both publicly announced that they will vote "no" on Jigsaw in its current form.

This project is an attempt to provide a standardised module system for Java, and after having been initially scheduled for Java 7, it was moved to Java 8 after Mark Reinhold announced Plan B. In July 2012, Oracle further delayed the project, by moving it to Java 9. Even this schedule was subject to slippage, due to the complexity of Jigsaw and the need for extensive testing.

The scope of Jigsaw is impressive, intending not only to modularise the monolithic Java runtime, but to also enforce strong encapsulation, so that Java code is forced to only access platform libraries via their public interfaces. Modular code may not simply reach into the internals of another module.

Not only that, but Oracle intends for the module system to be used by application code as well as the JDK itself. To this end, the JSR 376 Expert Group (the EG) has been discussing all aspects of application-level modularity as well as the JDK.

Outside of the standardisation process, there are already two main module implementations that have some overlap with the scope of Jigsaw: JBoss Modules, from Red Hat, and OSGi, which is looked after by a consortium of vendors, but includes IBM.

Both of these vendors are members of the 25-member Java Community Process Executive Committee, which is the body that must approve all new Java standards, including those which make up the "platform releases", i.e. major Java versions. Such releases must be approved by a 2/3 majority.

IBM is also voting "no" which reflects our position that the JSR is not ready at this time to move beyond the Public Review stage and proceed to Proposed Final Draft.

The JSR 376 Expert Group and the public have raised a number of reasonable issues and concerns with the current public review draft of the specification that warrant further discussion and resolution. We advocate work continuing amongst all members of the Expert Group to address the issues documented on the mailing lists.

IBM would like to see closer consensus amongst the entire Expert Group before this specification proceeds to the next step.

As it stands, Red Hat will not vote for the approval of this public review draft of JPMS as it is not in the best interest of the Java community.

At issue is the difficulty of making the existing modules systems supplied by IBM and Red Hat interoperate cleanly with Jigsaw. However, neither system is dominant in the market, and whilst painful for those affected, the majority of Java projects would not be affected by these incompatibilities.

However, the problems with Jigsaw extend further than just OSGi and JBoss modules. The dominant code packaging and distribution system in the Java world is Maven. Jigsaw does not provide a clean upgrade path for Maven-based projects to move from the packaging of code into JAR files to modules.

The detailed concerns are presented in a joint document authored by both Red Hat and external Maven experts. The key Maven concern is known as "Automatic Modules", and for many in the community the situation as regards Maven is potentially far worse than the compatibility problems with either of the extant vendor module systems.

Martijn Verburg, leader of the London Java Community (and also a voting member on the JCP Executive Committee) told InfoQ that:

It's unusual for an EC member to state their voting presence this early and in such a public manner. It is indicative of the serious concerns that the major players in the Java ecosystem have about JPMS and the de-facto Jigsaw implementation. Although the module specification and the implementation to date provides Java with some major security and compact runtime improvements, it has not (yet) bridged the gaps with other common module systems and build tools that sit upon it, rightly or wrongly raising fears of a Python2/3 style split for the Java platform.

For there to be such open disagreement and outright hostility to a new Java version so late in the development cycle is completely unprecedented. Whilst this vote is only the Public Review Ballot and not the final ratification ballot, this is a clear warning shot.

Oracle's published timeline for the delivery of Java 9 appears to be on a collision course with the expressed view of other vendors and the community. It remains to be seen whether they will alter direction, potentially delaying Java 9 yet again or to force through their vision of modular Java, even over the objections of major vendors and community participants.

Perfect is the enemy of good

Your message is awaiting moderation. Thank you for participating in the discussion.

Jigsaw is an engineering solution to an incredibly complicated and largely intractable problem. Both IBM and RedHat are backed into a corner on this one; it is arguably natural for each of them to oppose it. Hopefully Jigsaw passes, and we see (3 years down the road) that it was a good thing for Maven, OSGi, and for other module projects on top of Java.

In my own cynical/jaded view, the fact that Maven has a hard time with Jigsaw is not surprising, and must not be used as an excuse to reject Jigsaw. (Perhaps it should be used as an excuse to fix Maven?)

Re: Perfect is the enemy of good

Your message is awaiting moderation. Thank you for participating in the discussion.

Hey Cameron,

Yeah we could just hope that Jigsaw turns out okay in the end. Or we could actually fix the problems with it now, before they are released into Java and become impossible to fix due to backwards compatibility constraints. Bear in mind that a No vote at the current stage -- if it achieves a majority -- does not kill the JSR, it simply sends it back to the expert group for further consideration.

What in your view *should* be used as an "excuse" to reject Jigsaw? Should the Java community just accept it no matter how bad it is?

Should be no surprise...

Your message is awaiting moderation. Thank you for participating in the discussion.

This mess shouldn't be a surprise to anyone. Those in the industry with deep experience with Modularity have been flagging issues with Jigsaw for quite some time - e.g. aniszczyk.org/2009/11/23/jigsaw-versioning-is-r... - and the JSR has ignored this industry experience.

Re: Perfect is the enemy of good

Your message is awaiting moderation. Thank you for participating in the discussion.

Neil - This isn't the type of thing that we'll be able to resolve in an online discussion, and the challenge is far greater than a technical issue. The problem is that no matter what, the result of a project like this will be flawed, because it is being introduced into an existing ecosystem, it makes assumptions of its own, and it cannot achieve all of the goals that its end users would demand of it.

Bear with my twisted logic a bit further: The time to have agreed on the gating criteria for Jigsaw is over 10 years ago, and Sun should have built that consensus proactively from the beginning with the various interested parties, including OSGi, the various JCP members with a potential stake in this, etc.

There are many reasons why this did not happen, including some strong personalities that would have attempted to block it (far be it for me to name any names). So instead, fast forward 10+ years, and we have a devil's choice to make.

So either the whole thing gets accepted, or the whole thing gets rejected. Those are the choices that the vote allows for. So my $.02 is to evaluate whether the project makes Java better or worse, and to factor in time periods (how long it will take to put OSGi and Maven and Humpty Dumpty back together, for example), costs (how many apps it might break, and what it will take to fix them), and benefits. On the flip side, what is the cost of not having Jigsaw go in, and my guess is that carries a fairly heavy cost (based on the investments made over the past several years by the Java team with the assumption that Jigsaw would go in.)

I don't have the same vantage point as you do, so the way that we each weigh those costs and benefits will differ. You have a fairly unique vantage point from your long involvement with OSGi, but you have to remember that not a lot of people have seen what you've seen or been told what you've been told or heard what you've heard. I respect your opinion, even if my own calculations come out completely differently.

In the end, my vote (if I had one) would be based on Jigsaw's readiness and a judgment on whether it is fit for purpose, versus the likely costs of disruption that it will engender. For me, and from my limited knowledge, that would put me on the "yes" side.

Re: Perfect is the enemy of good

Your message is awaiting moderation. Thank you for participating in the discussion.

Thanks Cameron for your considered reply. I do agree that it's a devil's choice, but to me your cost calculation smells like a sunk-cost fallacy.

On the one side, ALL users of Java are being asked to bear what I regard as the heavy cost of dealing with a broken module system. I'm not just talking about OSGi users or Maven users, nobody in the Java community will be unaffected. The existing classpath is flawed too, but we have 20 years of experience dealing with it, along with techniques (such as a classloaders, OSGi, JBoss modules, etc) that make it manageable.

On the other side, you have the admittedly large investment made by the Java team at Oracle. But forgive me if I don't cry into my beer over Oracle making a bad investment.

I'll reiterate: a no vote at this stage doesn't kill Jigsaw, it just sends it back to the expert group to try to do better. And that's assuming the No vote is a majority — so far we have heard from Red Hat and IBM, only 2 out of 23 EC members. Therefore I don't have any real expectation of Jigsaw failing to pass the ballot.

Re: Perfect is the enemy of good

Your message is awaiting moderation. Thank you for participating in the discussion.

I question the heavy costs or even the utility of not having JigSaw go into 9. Anyone that cares about modularity is most likely using OSGi and those that do not will only neuter the modularity system.

No matter, what is of a bigger concern is that I've not seen any concern about breaking the tooling that we've built up around Java over the years. That agents will not work as is.. is a huge concern because that alone will knock out most of the diagnostic and monitoring tooling that exists in the Java eco system with the exception of JMC/JFR. Add in the arrogance of simply passing the costs to the Java community for a feature that few want and need....

As much I respect the amount of good honest effort that has gone into this project that has taken it farther than I imagined anyone could on a 20+ year old code base, I have to say that it's simply not ready. I am by no means a maven fan, that said, it needs to work as is before JigSaw ships. Add in the arrogance of simply passing the costs to the Java community for a feature that few want and need.... if I had a vote, I fear it must be no at this POT.

Re: Perfect is the enemy of good

Your message is awaiting moderation. Thank you for participating in the discussion.

Update, some information that Oracle engineering is indeed making efforts to ensure that agents do work in 9. HotSpot engineers are looking for input and test cases to help them weed out as many of the issues they can. Like I said, I believe this is an honest effort to get it done, it is my belief it needs more time.

Re: Perfect is the enemy of good

Your message is awaiting moderation. Thank you for participating in the discussion.

Kirk--

Would you hold the release of 9 until Jigsaw is ready? Or release 9 without modules, which would make the second release (fourth, if you include Java 6, 7 and 8 stemming all the way back to JSR 277) without them? Or would you pitch the effort entirely and just keep Java as it is?

I don't have a real dog in this fight--I gave up on worrying about Java modules when JSR 277 failed, and I could see the dynamics in that fight and saw no real factors coming that would ever change them. But I'm curious to know what IBM and Red Hat are proposing in place of Jigsaw. If not "this" Jigsaw, then what?

Re: Perfect is the enemy of good

Your message is awaiting moderation. Thank you for participating in the discussion.

Hi Ted,

The work that has gone into the module system is needed and is the right thing to do. The questions at hand are; are they ready for prime time and how much pain will the inflict on the community. IMHO we need the modules in at least 1 release before we hit the Java community with them full force.

My experience with componentization predates Java. You may recall Envy in VisusalAge for Java which was an unsuccessful port Envy for Smalltalk. The Smalltalk version was brilliant and represented one of *the* best source code control/repositories/component models I've yet to work with. We can talk about why the port to Java failed but more interesting is, what were the problems using Envy in Smalltalk? I don't want to clutter up this posting with a Smalltalk rant so I'll end with, there is history here that we could learn from but it seems that it's being ignored.

Re: Perfect is the enemy of good

Your message is awaiting moderation. Thank you for participating in the discussion.

Hi Ted, I know you asked Kirk but this is a public forum so I will answer you anyway.

In my opinion, Jigsaw is badly needed for modularising the JDK itself. Both for the ability to customise and strip down the runtime, and also for the security benefits of restricting access to JDK internals from application code. However the design of Jigsaw is just wrong for application modularity, because the needs of the JDK are very different from the needs of applications, libraries and the Java ecosystem at large.

Over the last couple of years, features have been incrementally added to Jigsaw to solve edge cases and application-level concerns that had not been anticipated in the original design. This has made Jigsaw highly complex and has created the significant delay in getting Java 9 out of the door.

So I would like to see Java 9 released with its modular JDK only. This could have been done at least a year ago if that had been the focus of the release. In future releases we could look at whether Jigsaw's secure isolation facility could be offered to Java developers in a way that doesn't require them to port their applications wholesale to this new module system -- which is doubly painful for users of existing module systems like OSGi.

Re: Perfect is the enemy of good

Your message is awaiting moderation. Thank you for participating in the discussion.

Hi Ted,

I'm David Lloyd, the JSR 376 EG representative for Red Hat.

We aren't actually proposing anything in place of Jigsaw. The main stumbling block we have encountered is that we have raised many issues, some of which we feel present a severe enough quality problem that the community will suffer (directly or indirectly) as a result.

Jigsaw does not represent what we (or others) have hoped it would be, but it's not insanely far off the mark either: there are maybe a half-dozen issues which push it over the line for us. Obviously I can't speak for IBM or anyone else, but from our perspective, it seems like it shouldn't take too much to move things to a place we would find acceptable (if not ideal).

Conceptually, Jigsaw is largely what we feel a Java module system would have to be. The bulk of the issues that we are concerned about relate to the actual development and deployment experience and many are derived directly from our experiences with deploying JBoss Modules, our module system (which I had originally hoped could be sunsetted after Java 9, but it looks like that is not going to be possible for us).

As for mitigation, opinions vary widely. I've heard good arguments for delaying Java 9 a bit longer, or for pulling Jigsaw out completely. Personally I think a few weeks delay to discuss, introduce, and test the changes required to bring this up to a minimum acceptable state is probably the best (and most realistic) option for everyone. However that of course is contingent on EG consensus (!) and a few other assumptions as well which may not actually hold up to reality.

It's also possible that the release will move ahead without regard to the PR vote, and it's also possible that it will turn out that none of the issues that the EG has raised will actually cause a problem in practice and that we're all overly concerned over nothing. But, given the general agreement of many EG members (and now some EC members), I am reassured that the issues that have been raised are more than just baseless worries or edge cases.

Re: Perfect is the enemy of good

Your message is awaiting moderation. Thank you for participating in the discussion.

Hi Cameron,

You mention your standards for a "yes" vote as being "based on Jigsaw's readiness and a judgment on whether it is fit for purpose, versus the likely costs of disruption that it will engender". This is essentially the same standard we have used. In my estimation, and that of several other experts, the costs of disruption are still too high, and the implementation itself is also not quite ready yet. But (at least for Red Hat), the bar is not too high to reach.

As Neil pointed out earlier, a "no" vote just means that we bring it back to the EG where we can collaborate to fix the most critical issues. I still believe this is possible and represents the best possible outcome for the Java community.

Re: Perfect is the enemy of good

Your message is awaiting moderation. Thank you for participating in the discussion.

That seems a reasonable answer; thanks, Neil. Appreciate it. I'm not sure how much or how little I agree with what you're saying--I've been too removed from all of this to really be able to weigh in with any sort of expertise--but your arguments at least have reasonable rationale, to me. I'll be curious to see how this all goes. Thanks again.

Re: Perfect is the enemy of good

Your message is awaiting moderation. Thank you for participating in the discussion.

That seems reasonable, and I'm happy to hear that everybody seems to agree that a module system is necessary and desirable--that wasn't obviously clear to me from reading the dissenting opinions. (And for the record, I'm a fan of dissenting opinion--that helps the discussion get better in the long run.)

Re: Perfect is the enemy of good

Your message is awaiting moderation. Thank you for participating in the discussion.

There's always that one Smalltalk guy at the back of the room.... ;-)

I vaguely recall Envy, and I was not impressed. Maybe sometime at a conference we can catch up and talk about what made Envy so interesting and compelling in this particular situation; I never really saw Smalltalk as having a great packaging/modularity solution, so I'm not sure what it would look like or behave like.

Re: Perfect is the enemy of good

Your message is awaiting moderation. Thank you for participating in the discussion.

Hi David,

I agree with your posting. I don't believe JigSaw far off the mark however I do see this as a release that will inflict unnecessary pain. We see a softer migration path as well as a wee bit more time to sort through the issues.

Re: Perfect is the enemy of good

Your message is awaiting moderation. Thank you for participating in the discussion.

I'm not sure what you had to compare it to but at the time I didn't see anything that worked any where near the level of Envy. It's tight integration into Smalltalk worked very well but that style of tight integration simply didn't translate in Java. With Java being file based (no image) and all the types and that is was attached to VisualAge which forced a peculiar workflow on developers, yes less than impressive which is why it simply failed. The Smalltalk integration was so tight it didn't even feel like you were using a SCCS. The component model was a bit simplistic and suffered because of that but there wasn't a viable alternative. In fact, there weren't any alternatives.

Re: Perfect is the enemy of good

Your message is awaiting moderation. Thank you for participating in the discussion.

>>it was a good thing for MavenThis is absolutely not truth. I am in ASF Maven and I know for certain that Oracle wanted Jigsaw to substitute Maven. It was obvious to me that they did not know Maven, Oracle used Ant and Makefiles only, and later on they understood that Maven is doing a lot and cannot be substituted. Certainly they changed their mind and stopped to be a competitor for Maven however mixing Jigsaw and Maven is nonsense because both have modularity principles and thus writing two similar but still different meta file is an overhead in maintenance.You all should understand that Jigsaw is for Oracle and JDK and not for you. Oracle wanted to solve their problem and not the API is shorter and they may pay less for their U.S. patents.