I'm getting the dreaded ClassCastException when deploying my web app due to a conflict between XercesImpl.jar bundled with JBoss and another one inside my war file:

13:00:45,353 INFO [TomcatDeployment] deploy, ctxPath=/nexj-web
13:00:45,462 ERROR [JBossContextConfig] XML error parsing: context.xml
org.jboss.xb.binding.JBossXBRuntimeException: Failed to create a new SAX parser
at org.jboss.xb.binding.UnmarshallerFactory$UnmarshallerFactoryImpl.newUnmarshaller(UnmarshallerFactory.java:100)
at org.jboss.web.tomcat.service.deployers.JBossContextConfig.processContextConfig(JBossContextConfig.java:549)
at org.jboss.web.tomcat.service.deployers.JBossContextConfig.init(JBossContextConfig.java:536)
at org.apache.catalina.startup.ContextConfig.lifecycleEvent(ContextConfig.java:279)
at org.apache.catalina.util.LifecycleSupport.fireLifecycleEvent(LifecycleSupport.java:117)
at org.apache.catalina.core.StandardContext.init(StandardContext.java:5436)
at org.apache.catalina.core.StandardContext.start(StandardContext.java:4148)
at org.jboss.web.tomcat.service.deployers.TomcatDeployment.performDeployInternal(TomcatDeployment.java:310)
at org.jboss.web.tomcat.service.deployers.TomcatDeployment.performDeploy(TomcatDeployment.java:142)
at org.jboss.web.deployers.AbstractWarDeployment.start(AbstractWarDeployment.java:461)
at org.jboss.web.deployers.WebModule.startModule(WebModule.java:118)
at org.jboss.web.deployers.WebModule.start(WebModule.java:97)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:585)
at org.jboss.mx.interceptor.ReflectedDispatcher.invoke(ReflectedDispatcher.java:157)
at org.jboss.mx.server.Invocation.dispatch(Invocation.java:96)
at org.jboss.mx.server.Invocation.invoke(Invocation.java:88)
at org.jboss.mx.server.AbstractMBeanInvoker.invoke(AbstractMBeanInvoker.java:264)
at org.jboss.mx.server.MBeanServerImpl.invoke(MBeanServerImpl.java:668)
at org.jboss.system.microcontainer.ServiceProxy.invoke(ServiceProxy.java:206)
at $Proxy38.start(Unknown Source)
at org.jboss.system.microcontainer.StartStopLifecycleAction.installAction(StartStopLifecycleAction.java:42)
at org.jboss.system.microcontainer.StartStopLifecycleAction.installAction(StartStopLifecycleAction.java:37)
at org.jboss.dependency.plugins.action.SimpleControllerContextAction.simpleInstallAction(SimpleControllerContextAction.java:62)
at org.jboss.dependency.plugins.action.AccessControllerContextAction.install(AccessControllerContextAction.java:71)
at org.jboss.dependency.plugins.AbstractControllerContextActions.install(AbstractControllerContextActions.java:51)
at org.jboss.dependency.plugins.AbstractControllerContext.install(AbstractControllerContext.java:348)
at org.jboss.system.microcontainer.ServiceControllerContext.install(ServiceControllerContext.java:286)
at org.jboss.dependency.plugins.AbstractController.install(AbstractController.java:1631)
at org.jboss.dependency.plugins.AbstractController.incrementState(AbstractController.java:934)
at org.jboss.dependency.plugins.AbstractController.resolveContexts(AbstractController.java:1082)
at org.jboss.dependency.plugins.AbstractController.resolveContexts(AbstractController.java:984)
at org.jboss.dependency.plugins.AbstractController.change(AbstractController.java:822)
at org.jboss.dependency.plugins.AbstractController.change(AbstractController.java:553)
at org.jboss.system.ServiceController.doChange(ServiceController.java:688)
at org.jboss.system.ServiceController.start(ServiceController.java:460)
at org.jboss.system.deployers.ServiceDeployer.start(ServiceDeployer.java:163)
at org.jboss.system.deployers.ServiceDeployer.deploy(ServiceDeployer.java:99)
at org.jboss.system.deployers.ServiceDeployer.deploy(ServiceDeployer.java:46)
at org.jboss.deployers.spi.deployer.helpers.AbstractSimpleRealDeployer.internalDeploy(AbstractSimpleRealDeployer.java:62)
at org.jboss.deployers.spi.deployer.helpers.AbstractRealDeployer.deploy(AbstractRealDeployer.java:50)
at org.jboss.deployers.plugins.deployers.DeployerWrapper.deploy(DeployerWrapper.java:171)
at org.jboss.deployers.plugins.deployers.DeployersImpl.doDeploy(DeployersImpl.java:1439)
at org.jboss.deployers.plugins.deployers.DeployersImpl.doInstallParentFirst(DeployersImpl.java:1157)
at org.jboss.deployers.plugins.deployers.DeployersImpl.doInstallParentFirst(DeployersImpl.java:1178)
at org.jboss.deployers.plugins.deployers.DeployersImpl.install(DeployersImpl.java:1098)
at org.jboss.dependency.plugins.AbstractControllerContext.install(AbstractControllerContext.java:348)
at org.jboss.dependency.plugins.AbstractController.install(AbstractController.java:1631)
at org.jboss.dependency.plugins.AbstractController.incrementState(AbstractController.java:934)
at org.jboss.dependency.plugins.AbstractController.resolveContexts(AbstractController.java:1082)
at org.jboss.dependency.plugins.AbstractController.resolveContexts(AbstractController.java:984)
at org.jboss.dependency.plugins.AbstractController.change(AbstractController.java:822)
at org.jboss.dependency.plugins.AbstractController.change(AbstractController.java:553)
at org.jboss.deployers.plugins.deployers.DeployersImpl.process(DeployersImpl.java:781)
at org.jboss.deployers.plugins.main.MainDeployerImpl.process(MainDeployerImpl.java:702)
at org.jboss.system.server.profileservice.repository.MainDeployerAdapter.process(MainDeployerAdapter.java:117)
at org.jboss.system.server.profileservice.repository.ProfileDeployAction.install(ProfileDeployAction.java:70)
at org.jboss.system.server.profileservice.repository.AbstractProfileAction.install(AbstractProfileAction.java:53)
at org.jboss.system.server.profileservice.repository.AbstractProfileService.install(AbstractProfileService.java:361)
at org.jboss.dependency.plugins.AbstractControllerContext.install(AbstractControllerContext.java:348)
at org.jboss.dependency.plugins.AbstractController.install(AbstractController.java:1631)
at org.jboss.dependency.plugins.AbstractController.incrementState(AbstractController.java:934)
at org.jboss.dependency.plugins.AbstractController.resolveContexts(AbstractController.java:1082)
at org.jboss.dependency.plugins.AbstractController.resolveContexts(AbstractController.java:984)
at org.jboss.dependency.plugins.AbstractController.change(AbstractController.java:822)
at org.jboss.dependency.plugins.AbstractController.change(AbstractController.java:553)
at org.jboss.system.server.profileservice.repository.AbstractProfileService.activateProfile(AbstractProfileService.java:306)
at org.jboss.system.server.profileservice.ProfileServiceBootstrap.start(ProfileServiceBootstrap.java:271)
at org.jboss.bootstrap.AbstractServerImpl.start(AbstractServerImpl.java:461)
at org.jboss.Main.boot(Main.java:221)
at org.jboss.Main$1.run(Main.java:556)
at java.lang.Thread.run(Thread.java:595)
Caused by: org.jboss.xb.binding.JBossXBException: Failed to create a new SAX parser
at org.jboss.xb.binding.parser.sax.SaxJBossXBParser.<init>(SaxJBossXBParser.java:97)
at org.jboss.xb.binding.UnmarshallerImpl.<init>(UnmarshallerImpl.java:56)
at org.jboss.xb.binding.UnmarshallerFactory$UnmarshallerFactoryImpl.newUnmarshaller(UnmarshallerFactory.java:96)
... 73 more
Caused by: java.lang.ClassCastException: org.apache.xerces.parsers.XIncludeAwareParserConfiguration
at org.apache.xerces.parsers.SAXParser.<init>(SAXParser.java:99)
at org.apache.xerces.parsers.SAXParser.<init>(SAXParser.java:84)
at org.apache.xerces.jaxp.SAXParserImpl$JAXPSAXParser.<init>(SAXParserImpl.java:364)
at org.apache.xerces.jaxp.SAXParserImpl.<init>(SAXParserImpl.java:124)
at org.apache.xerces.jaxp.SAXParserFactoryImpl.newSAXParser(SAXParserFactoryImpl.java:76)
at org.jboss.xb.binding.parser.sax.SaxJBossXBParser.<init>(SaxJBossXBParser.java:92)
... 75 more

I've traced through the deployer and Xerces code and the Exception is happening for the following reason: - the deployer sets the context classloader to be the war classloader - the deployer tries to create a parser to parse context.xml file, this is delegated to xerces - as part of initializing the parser, xerces does the following:

- digging into the createObject() call, xerces uses the context classloader to load XIncludeAwareParserConfiguration - however, XMLParserConfiguration to which XIncludeAwareParserConfiguration is cast in the above statement is loaded by the current classloader which is the deployer classloader (jboss-web, I believe) - thus the ClassCastException

Now, I can resolve this issue by following the instructions in http://www.jboss.org/community/wiki/useJBossWebClassLoaderinJBoss5. However, neither of the approaches there is acceptable since they both boil down to effectively removing the XercesImpl.jar from my war. Both of those approaches force the war classloader to delegate to parent first which essentially forces the JBoss bundled XercesImpl.jar to always be used. In fact, if I follow this approach I get another (application specific) Exception because my application relies on using the context classloader (i.e. the war classloader) to load classes from the XercesImpl.jar inside my war. Additionally, I don't really want to change the Servlet Spec classloading behaviour anyway.

I think the real problem is that the deployer sets the context classloader to the war classloader before instantiating the parser. IMHO, I think the deployer should either do this after it instantiates the parser or it should temporarily swap out the context classloader and swap in the deployer classloader in its place. Incidently, in JBoss 4.0.5, the version from which I'm trying to upgrade to 5.1.0, there is no problem. I've briefly looked at the 4.0.5 code and I believe that the tomcat deployer used to deploy the war, indeed uses the deployer classloader to instantiate the parser and not the war classloader.

As can be seen from the Exception StackTrace, there is absolutely no involvement of the application at this point so I really fail to understand why the application classloader (the war classloader) is being used by the JBoss deployer to load the Xerces classes.

I've triple checked how this works in JBoss 4 and it definitely doesn't do this. The war deployer (tomcat deployer) uses its own classloader, not the war classloader to load Xerces classes. So this looks like a change in behavior between JBoss 4 and 5.

In JBoss 5, by default the war classloader is isolated and has parent-first=false as per the servlet spec. I want to keep this behavior. I can change this with jboss-classloading.xml and get through the deployment but I then have problems later on in my app as I stated in my previous post. So this path is really not an option for me.

I'm really down to my last option here of refactoring my Xerces with my own package structure so that it wouldn't interfere with the JBoss Xerces. I would really like to avoid this if possible.

Just to understand this better - Are you saying that when the "server" is parsing some xml files within your app, then it uses the xerces jar shipped in your application instead of the one shipped in JBoss?

I think the 3rd exception there is from when it is trying to parse the context.xml file in my war. However, the 1st (and the 2nd?) entry I think are from when it is trying to parse the context.xml sitting under server/default/deploy/jbossweb.sar/.

In either case, yes it is the "server" (more specifically the war deployer) that is doing the parsing.

I tried to reproduce this issue. The error only affects the application which ships the xercesImpl.jar (or other xerces/xml related jars). Although the 3 log lines that you posted might give an impression that its affects other (default) apps too, but in reality it just fails to deploy only your application.

However, this changes the servlet spec classloading behaviour as well as causes other Exceptions in my application (which I am not posting here since they are app specific) because my app is now using the JBoss xerces and not my own. Therefore, this is not an acceptable solution.

If I configure classloading isolation like so (notice the different value for parent-first), I get the same Exceptions as posted above.

I've already explained all this in my post. The problem is that JBoss is using my web application's classloader during deployment instead of its own.

Can you just answer this one question: If you look at the Exception stacktrace, there is no sign of application specific classes there, so how come it is the xerces bundled with the application that gets used to parse the xml files?

No problem. All I was saying was that while deploying my app, the deployer throws the Exception for each of the 3 context.xml files that it tries to parse (because it is using the xerces inside my app instead of the one bundled with JBoss).

Yes, i did try this out. From what i looked at, the deployment framework starts using the version of xerces jar present in your deployment because it sets the classloader to your deployment's classloader. I think this is the correct thing to do, because each deployment is expected to be started within its own classloader to ensure that it gets access to the right resources.

Now the real issue with xerces is that its an endorsed library. i.e. it's part of the Java endorsed library directory. So the only way to override the xerces version appears to be to change the version in the JBoss endorsed directory (but that can lead to other issues). This is where i had left off in my investigation of this issue a while back.

"jaikiran" wrote:This is where i had left off in my investigation of this issue a while back.

I did not reply back with these details earlier because, i did see that there was some weird logic in getting hold of the "right" classloader in the xerces implementation (ObjectFactory and/or its related classes). So i am not sure whether its an intended behaviour or just a bug in its implementation.

"jaikiran" wrote:I think this is the correct thing to do, because each deployment is expected to be started within its own classloader to ensure that it gets access to the right resources.

I do not believe JBoss 4 does this. It uses the deployer's classloader when deploying the war, not the war's classloader. Thus the problem is avoided.

"jaikiran" wrote:So the only way to override the xerces version appears to be to change the version in the JBoss endorsed directory (but that can lead to other issues).

The application I'm working on may be installed at a client site where we have no controll over the the App Server. In addition, as you said, changing the endorsed Xerces may lead to other problems so I do not think this is a viable solution.

Are you saying that currently there is no solution to this problem? If so, I'd say this is a pretty big issue for JBoss considering this problem is not seen in JBoss 4 and thus constitutes a pretty important change in behavior.

Perhaps this should be escalated and/or be given more attention? I've seen numerous posts on the forum complaining about this and none of the workarounds (classloader isolation, removing Xerces from war) address the underlying problem of being able to use a different version of Xerces than the one in the endorsed dir.

I read about this, during the weekend, in the apache xerces project page. Turns out, this is a common issue with all application server and they have a FAQ for this specific issue which explains the details http://xerces.apache.org/xerces2-j/faq-general.html#faq-5. JBoss AS runs into this exact issue.

"rodos77" wrote:

IMHO, I think the deployer should either do this after it instantiates the parser or it should temporarily swap out the context classloader and swap in the deployer classloader in its place

The deployers use JBossXB:

at org.jboss.xb.binding.parser.sax.SaxJBossXBParser.<init>(SaxJBossXBParser.java:97)
at org.jboss.xb.binding.UnmarshallerImpl.<init>(UnmarshallerImpl.java:56)
at org.jboss.xb.binding.UnmarshallerFactory$UnmarshallerFactoryImpl.newUnmarshaller(Unmarsha
llerFactory.java:96)

So i guess its the responsibility of JBoss XB to switch the classloader.

From what i looked at, the deployment framework starts using the version of xerces jar present in your deployment because it sets the classloader to your deployment's classloader. I think this is the correct thing to do, because each deployment is expected to be started within its own classloader to ensure that it gets access to the right resources.

Switching the classloader should be done only for the xerces issue, the rest of the deployment process should continue to use the deployment's classloader.

So i guess its the responsibility of JBoss XB to switch the classloader.

By the way, switching the classloader is only going to fix the classcastexception. Switching the classloader would mean that the xerces class will end up being used from what is shipped in JBoss (or for that matter in any other application server). So it really needs to be a question asked in the xerces-J forums whether there is any practical way to overriding the xercesImpl jars in a application server environment where multiple classloaders are involved.