I've starting investigating this issue. The most obvious thing to notice is that the icedtea6-plugin doesn't create the named pipes in ~/icedteaplugin/ as it should.

That would explain the subsequent failure of the assertion for the gio write 'channel != NULL'. Obviously the channel is NULL and is presumably the pipe firefox should be writing to to communicate back to the plugin.

This is a high-importance bug since any applet embedded in a web page can cause Firefox to spin the CPU at 100% until the entire Firefox process is killed.

The user has no control over the launching of a Java applet nor warning that a page contains one.

Now that firefox-3.5 takes over the /usr/bin/firefox symbolic link this becomes more invasive since any desktop shortcuts or links embedded in application configurations that expect to launch a working firefox-3.0 will launch firefox-3.5 instead.

Actual Results:
CPU core runs at 100% and can only be stopped if the browser process is sent the SIGKILL signal - SIGTERM/closing the application doesn't remove the process from memory or stop the CPU spinning.

This matches the IID requested by IcedTeaPlugin, "9DA0B650-D07E-4617-*"

I looked at the differences in the interface descriptions of the two versions. It seems that apart from all the additional comments, the 1.9.1 interface has an additional member:

readonly attribute boolean isRunning;

Can an XPCOM expert confirm I've interpreted this information correctly?

If it is correct it has implications for the IcedTea/OpenJDK builds since to support both versions of xulrunner there would need to be two versions of the IcedTea Plugin built, one against the xulrunner 1.9.0 headers and the other against the 1.9.1 headers.

Would patching IcedTeaPlugin with additional code to detect the interface version at run-time and choosing the IID to pass based on that be sufficient to work around this issue, allowing one IcedTea/OpenJDK build to service both xulrunner versions?

I've now confirmed the problem is the use of the static build-time IID for nsIProcess in the IcedTeaPlugin StartAppletviewer() method, combined with it including the headers from xulrunner 1.9.0, resulting in the wrong IID being requested when running with xulrunner 1.9.1.

I've created a patch on IcedTeaPlugin that requests the run-time IID and requests that. Details are in the Ubuntu bug report:

The bug referred to in my comment #8 and linked against Mozilla Firefox.

I've got a relatively simply patch in the works, although its a slight kludge.

In IcedTeaPluginFactory::StartAppletViewer() it will check the run-time xulrunner platform version and choose the IID to request (from the one provided by the 1.9.0 headers, or the explicitly coded 1.9.1 IID).

It should make the patch about 8 lines if everything I've read up on, and checked with the xulrunner dev's on IRC, proves correct.

In summary, The Jaunty version of openjdk-6 IcedTeaPlugin is built against the xulrunner 1.9.0.x dev headers (which Firefox 3.0.x uses). IcedTeaPlugin creates an instance of nsIProcess using the Interface ID (IID) declared in those headers. The nsIProcess object then loads and manages the JVM in another process.

Firefox 3.5 uses xulrunner 1.9.1.x, which has 'unfrozen' the nsIProcess interface definition and consequently changed the IID. Therefore when xulrunner 1.9.1 is asked to create the nsIProcess object by IcedTeaPlugin it fails since the old xulrunner 1.9.0 IID doesn't exist.

I've patched IcedTeaPlugin.cc to query the IID of nsIProcess at run-time rather than use the build-time definition. I've tested the same IcedTeaPlugin.so with Firefox 3.0 and 3.5 and both work correctly, starting the applet.

I've found there is a subsequent exception once the Sun Java test applet is running which needs a separate investigation:

However, other sophisticated Java applets are working fine on Firefox 3.5 now.

I will attach a patch and debdiff with the fix and publish a fixed version of openjdk-6 to my PPA.

Because it takes so long to build openjdk and its a lot of overkill for building just IcedTeaPlugin I'll also build just the IcedTeaPlugin.so for i386 and amd64 and host them where they can easily be tested by backing up the current shared object:

The patch is intended as a specific patch for the Ubuntu Jaunty (and related Debian) openjdk package since it is supposed to support the use of the IcedTeaPlugin in both Firefox-3.0 and Firefox-3.5, but being built against the xulrunner 1.9.0 headers, fails in respect of the nsIProcess IID as I describe.

In later packages of openjdk (Karmic, etc) this issue should no longer be a problem since the default Firefox will be 3.5 and xulrunner 1.9.1+, and the IcedTea/OpenJDK packages are similarly up to date and don't make use of xulrunner 1.9.0 headers.

I just noticed that openjdk 6b16-1.6.1-1ubuntu3 freezes Firefox 3.5.4 (and 3.5.5 from the ppa) in Karmic in a reproducible manner whenever I try to open a site requiring java. The only hint I was able to find is the following line in .xsession-errors:

I am using Jaunty 32-bit with Firefox 3.5 (3.5.7+nobinonly-0ubuntu0.9.04.1) from the Ubuntu Universe repository and OpenJDK/IcedTea (6b16-1.6.1-0jaunty1) from the OpenJDK PPA (https://launchpad.net/~openjdk/+archive/ppa). I have never had Sun JDK or GCJ installed on this system.

Prior to installing OpenJDK 1.6.x from the OpenJDK PPA (when I was using OpenJDK 1.4.x that came with the official Jaunty repositories) when I loaded a Java applet, Firefox would hang at 100% CPU and nothing but a grey box would appear in place of the applet. I would have to manually kill the Firefox process.

Now that I am using OpenJDK 1.6.x, Firefox does not hang upon attempting to load the applet, but still only shows the grey box. I can still do all other Firefox operations normally. I still get error messages on the terminal identical to those shown in this bug report. When I close Firefox, the Firefox process hangs, and I have to manually kill it before starting a new Firefox instance.

So from what I can see, the OpenJDK/IcedTea 1.6.x packages in the OpenJDK PPA do nothing for this bug except delay the hang until you attempt to close the Firefox instance. Can someone confirm whether simply adding the OpenJDK PPA and dist-upgrading to OpenJDK 1.6.x is enough to fix this? Do I have to do any extra voodoo like first remove and purge all of Firefox/OpenJDK/world? Do I have to manually delete any directories or symlinks? I have not done any manual mucking around with the Firefox or OpenJDK stuff, and was hoping that simply letting apt do the upgrade would be sufficient, but apparently it's not.

Does this mean the bug was fixed in Karmic, but not in Jaunty (for which it was reported)?
I installed openjdk-6-jre (6b14-1.4.1-0ubuntu12) and icedtea6-plugin (6b14-1.4.1-0ubuntu12) from the current repository but there's still no java in Firefox 3.5/3.6 (in Firefox 3.0.8 it works.)
Karmic does not like my notebook, that's why I have to stick with Jaunty.
Are there any dependencies in Jaunty that prevent the fixed openjdk from being incorporated into the repository?

* Update IcedTea6 to the icedtea6-1.8 release.
* Fix builds on Ubuntu/dapper and Debian/lenny.
* On hppa, configure --without-rhino --disable-plugin.
* Fix Hitachi SH configury. Closes: #575346.
* Start a window manager when running the tests. Prefer metacity,
as more tests pass with it.
* Let XToolkit.isTraySupported() return true, if Compiz is running.
Works around sun#6438179. LP: #300948.
* Make <java_home>/jre/lib/security/nss.cfg a config file.
* Fail in the configuration of the packages, if /proc is not mounted.
java currently uses tricks to find its own shared libraries depending
on the path of the binary. Will be changed in OpenJDK7. Closes: #576453.
* Fix PR icedtea/469, testsuite failures with the NSS based security
provider. LP: #556549.
* Do not pass LD_LIBRARY_PATH from the plugin to the java process.
While libnss3.so gets loaded from /usr/lib, the dependent libraries
are loaded from MOZILLA_FIVE_HOME (See #561216 for the wrong firefox
config). LP: ...