JBoss Web Service Client call causing VFS IO contention issue in JBoss EAP 5.0.0
Issue
- EJB clients are timingout waiting for an instance to be available. The main issue is with the IO contention within internal JBoss VFS, The part which is managing IO access to code/JAR file repository, typically being triggered by class loading calls. In that case its triggered by JBoss when calling getResourceAsStream to pull WSService definition etc. What are the other ways of avoiding this contention? should we deploy the application as deflated instead of ear file
"http-192.161.1.68-8443-17" daemon prio=3 tid=0x00ef0400 nid=0x2321 runnable [0x5c5fb000]
java.lang.Thread.State: RUNNABLE
at java.util.jar.JarFile.getBytes(JarFile.java:358)
at java.util.jar.JarFile.initializeVerifier(JarFile.java:317)
at java.util.jar.JarFile.getInputStream(JarFile.java:386)
- locked <0x76b40030> (a java.util.jar.JarFile)
at org.jboss.virtual.plugins.context.zip.ZipFileWrapper.openStream(ZipFileWrapper.java:215)
- locked <0x76b40170> (a org.jboss.virtual.plugins.context.zip.ZipFileWrapper)
at org.jboss.virtual.plugins.context.zip.ZipEntryContext.openStream(ZipEntryContext.java:1084)
at org.jboss.virtual.plugins.context.zip.ZipEntryHandler.openStream(ZipEntryHandler.java:154)
at org.jboss.virtual.VirtualFile.openStream(VirtualFile.java:241)
at org.jboss.virtual.plugins.vfs.VirtualFileURLConnection.getInputStream(VirtualFileURLConnection.java:93)
at java.net.URL.openStream(URL.java:1010)
at java.lang.ClassLoader.getResourceAsStream(ClassLoader.java:1150)
at org.jboss.wsf.spi.util.ServiceLoader.loadFromServices(ServiceLoader.java:81)
at org.jboss.wsf.spi.util.ServiceLoader.loadService(ServiceLoader.java:59)
at org.jboss.ws.core.soap.MessageFactoryImpl.createMessage(MessageFactoryImpl.java:289)
at org.jboss.ws.core.soap.SOAPMessageUnMarshallerHTTP.read(SOAPMessageUnMarshallerHTTP.java:82)
at org.jboss.remoting.transport.http.HTTPClientInvoker.readResponse(HTTPClientInvoker.java:584)
at org.jboss.remoting.transport.http.HTTPClientInvoker.useHttpURLConnection(HTTPClientInvoker.java:381)
at org.jboss.remoting.transport.http.HTTPClientInvoker.makeInvocation(HTTPClientInvoker.java:232)
at org.jboss.remoting.transport.http.HTTPClientInvoker.transport(HTTPClientInvoker.java:162)
at org.jboss.remoting.MicroRemoteClientInvoker.invoke(MicroRemoteClientInvoker.java:167)
at org.jboss.remoting.Client.invoke(Client.java:1917)
at org.jboss.remoting.Client.invoke(Client.java:768)
at org.jboss.ws.core.client.HTTPRemotingConnection.invoke(HTTPRemotingConnection.java:232)
at org.jboss.ws.core.client.SOAPProtocolConnectionHTTP.invoke(SOAPProtocolConnectionHTTP.java:71)
at org.jboss.ws.core.CommonClient.invoke(CommonClient.java:339)
at org.jboss.ws.core.jaxws.client.ClientImpl.invoke(ClientImpl.java:231)
at org.jboss.ws.core.jaxws.client.ClientProxy.invoke(ClientProxy.java:162)
at org.jboss.ws.core.jaxws.client.ClientProxy.invoke(ClientProxy.java:148)
at $Proxy245.executeCommand(Unknown Source)
- Application is responding very slow and most of the threads are blocked state and are waiting to acquire lock on a thread like below :
Thread: pool-34-thread-1 : priority:5, demon:false, threadId:290, threadState:RUNNABLE
java.util.zip.ZipFile.open(Native Method)
java.util.zip.ZipFile.(ZipFile.java:127)
java.util.jar.JarFile.(JarFile.java:135)
java.util.jar.JarFile.(JarFile.java:99)
org.jboss.virtual.plugins.context.zip.ZipFileWrapper.ensureZipFile(ZipFileWrapper.java:173)
org.jboss.virtual.plugins.context.zip.ZipFileWrapper.openStream(ZipFileWrapper.java:211)
- locked <0x11ac8b5d> (a org.jboss.virtual.plugins.context.zip.ZipFileWrapper)
org.jboss.virtual.plugins.context.zip.ZipEntryContext.openStream(ZipEntryContext.java:1084)
org.jboss.virtual.plugins.context.zip.ZipEntryHandler.openStream(ZipEntryHandler.java:154)
org.jboss.virtual.VirtualFile.openStream(VirtualFile.java:241)
org.jboss.virtual.plugins.vfs.VirtualFileURLConnection.getInputStream(VirtualFileURLConnection.java:93)
java.net.URL.openStream(URL.java:1010)
java.lang.ClassLoader.getResourceAsStream(ClassLoader.java:1194)
javax.xml.bind.ContextFinder.find(ContextFinder.java:302)
javax.xml.bind.JAXBContext.newInstance(JAXBContext.java:412)
javax.xml.bind.JAXBContext.newInstance(JAXBContext.java:375)
javax.xml.bind.JAXBContext.newInstance(JAXBContext.java:279)
com.sun.xml.ws.assembler.MetroConfigLoader.loadMetroConfig(MetroConfigLoader.java:212)
com.sun.xml.ws.assembler.MetroConfigLoader.(MetroConfigLoader.java:115)
com.sun.xml.ws.assembler.MetroConfigLoader.(MetroConfigLoader.java:105)
com.sun.xml.ws.assembler.TubelineAssemblyController.getTubeCreators(TubelineAssemblyController.java:86)
com.sun.xml.ws.assembler.TubelineAssemblerFactoryImpl$MetroTubelineAssembler.createClient(TubelineAssemblerFactoryImpl.java:114)
com.sun.xml.ws.client.Stub.createPipeline(Stub.java:228)
com.sun.xml.ws.client.Stub.(Stub.java:205)
com.sun.xml.ws.client.Stub.(Stub.java:178)
com.sun.xml.ws.client.sei.SEIStub.(SEIStub.java:85)
com.sun.xml.ws.client.WSServiceDelegate.createEndpointIFBaseProxy(WSServiceDelegate.java:608)
com.sun.xml.ws.client.WSServiceDelegate.getPort(WSServiceDelegate.java:348)
com.sun.xml.ws.client.WSServiceDelegate.getPort(WSServiceDelegate.java:330)
com.sun.xml.ws.client.WSServiceDelegate.getPort(WSServiceDelegate.java:312)
javax.xml.ws.Service.getPort(Service.java:134)
Environment
- JBoss Enterprise Application Platform (EAP)
- 5.0.0
Subscriber exclusive content
A Red Hat subscription provides unlimited access to our knowledgebase of over 48,000 articles and solutions.
Welcome! Check out the Getting Started with Red Hat page for quick tours and guides for common tasks.
