JDK-7020709 : regression: cannot run filemaker application due to java.lang.ClassCircularityError

Type:Bug

Component:deploy

Sub-Component:deployment_toolkit

Affected Version:6u24

Priority:P2

Status:Closed

Resolution:Fixed

OS:generic

CPU:generic

Submitted:2011-02-18

Updated:2012-09-14

Resolved:2011-05-31

Versions (Unresolved/Resolved/Fixed)

The Version table provides details related to the release that this issue/RFE will be addressed.

Unresolved : Release in which this issue/RFE will be addressed.Resolved: Release in which this issue/RFE has been resolved.Fixed : Release in which this issue/RFE has been fixed. The release containing this fix may be available for download as an Early Access Release or a General Availability Release.

Customer report (Filemaker): Java 6u24 was released yesterday and it created a show stopper problem in our product for customers who upgraded. It appears that Sun/Oracle put a lot of opaque security fixes in that release and I believe that it may have impacted the class loading of our Java Web Start Application - specifically in the httpclient library.
Stack trace is below.
Caused by: java.lang.ClassCircularityError: com/oaklandsw/util/c
at com.oaklandsw.http.Handler.openConnection(Handler.java:19)
at java.net.URL.openConnection(Unknown Source)
at sun.net.www.protocol.jar.JarURLConnection.<init>(Unknown Source)
at com.sun.jnlp.JNLPCachedJarURLConnection.<init>(Unknown Source)
at com.sun.javaws.net.protocol.jar.Handler.openConnection(Unknown Source)
at java.net.URL.openConnection(Unknown Source)
at com.sun.deploy.util.URLUtil.checkTargetURL(Unknown Source)
at com.sun.deploy.security.DeployURLClassPath$JarLoader.checkResource(Unknown Source)
at com.sun.deploy.security.DeployURLClassPath$JarLoader.getResource(Unknown Source)
at com.sun.deploy.security.DeployURLClassPath.getResource(Unknown Source)
at java.net.URLClassLoader$1.run(Unknown Source)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(Unknown Source)
at com.sun.jnlp.JNLPClassLoader.findClass(Unknown Source)
at java.lang.ClassLoader.loadClass(Unknown Source)
at java.lang.ClassLoader.loadClass(Unknown Source)
at java.lang.ClassLoader.loadClass(Unknown Source)
at com.oaklandsw.http.Handler.openConnection(Handler.java:19)
at java.net.URL.openConnection(Unknown Source)
at sun.net.www.protocol.jar.JarURLConnection.<init>(Unknown Source)
at com.sun.jnlp.JNLPCachedJarURLConnection.<init>(Unknown Source)
at com.sun.javaws.net.protocol.jar.Handler.openConnection(Unknown Source)
at java.net.URL.openConnection(Unknown Source)
at com.sun.deploy.util.URLUtil.checkTargetURL(Unknown Source)
at com.sun.deploy.security.DeployURLClassPath$JarLoader.checkResource(Unknown Source)
at com.sun.deploy.security.DeployURLClassPath$JarLoader.getResource(Unknown Source)
at com.sun.deploy.security.DeployURLClassPath.getResource(Unknown Source)
at java.net.URLClassLoader$1.run(Unknown Source)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(Unknown Source)
at com.sun.jnlp.JNLPClassLoader.findClass(Unknown Source)
at java.lang.ClassLoader.loadClass(Unknown Source)
at java.lang.ClassLoader.loadClass(Unknown Source)
at java.lang.ClassLoader.loadClass(Unknown Source)
at com.filemaker.adminconsole.ui.clients.AdminConsoleJnlpLauncher.<init>(AdminConsoleJnlpLauncher.java:217)
at com.filemaker.adminconsole.ui.clients.AdminConsoleJnlpLauncher.main(AdminConsoleJnlpLauncher.java:106)
Reviewed with Roger, OK to defer for 7. Waiting on security review, will putback for 6 update and forward port.

EVALUATION
In 6u24, when we fix the bug:
6954743 "Applet codebase parameter permits web to file system linking"
We introduced a method called checkTargetURL to perform the security check, based on permission objects obtained from URLConnection. (via URL.openConnection)
Problem is in the case where custom HTTP handler is installed, we might ran into ClassCircularityError. For example:
1) Application registers new HttpURLConnectionHandler
It is supposed to be called by URL.openConnection()
Note that registration is just specifying the property => no classes are loaded
2) user code wants to load something from remote jar
3) this initiate checkURL and this will require http connection => lead to attempt to load some user class from connection handler
4) handler itself (or its dependency) is needs to loaded from remote jar => checkURL is called again and will try to do openConnection again
5) this means attempt to resolve handler class again (like on step 4) => classloader detects circularity

2011-02-21

SUGGESTED FIX
Remove use of URL.openConnection() in checkTargetURL. The security check can be done by comparing the protocol used in the sourceURL and the targetURL.
http://sa.sfbay.sun.com/projects/deployment_data/7/7020709.0

2011-02-21

EVALUATION
customer provided testcase that can be used to reproduce the bug:
http://javaweb.sfbay.sun.com/~ngthomas/filemaker/httptest.jnlp
also evaluation from networking team:
The stacktrace (see the last stacktrace below, original from the customer) shows that URLUtil.checkTargetURL() may lead to a HTTP connection which leads to the problem. That is,
1) Try to load a class
2) findClass may trigger the HTTP connection
3) DeployURLClassPath$JarLoader.checkResource invokes checkTargetURL()
4) May trigger a HTTP connection
5) DeployURLClassPath$JarLoader.checkResource invokes checkTargetURL()
6) Bang!
I think this may be a general problem with the original fix for CR 6954743. You may need to ensure that circularity cannot occur.
This issue may be specific to when a user defined HTTP protocol handler is installed. A user defined protocol handler may force the loading of classes/resources from the codebase, i.e. not on the classpath. Hence the circularity problem. But I think this code needs to be carefully reexamined.