JBoss Web Service Client call causing VFS IO contention issue in JBoss EAP 5.0.0

Solution Unverified - Updated -

Issue

  • EJB clients are timingout waiting for an instance to be available.  The main issue is with the IO contention within internal JBoss VFS,  The part which is managing IO access to code/JAR file repository, typically being triggered by class loading calls.  In that case its triggered by JBoss when calling getResourceAsStream to pull WSService definition etc.  What are the other ways of avoiding this contention? should we deploy the application as deflated instead of ear file
        "http-192.161.1.68-8443-17" daemon prio=3 tid=0x00ef0400 nid=0x2321 runnable [0x5c5fb000]
           java.lang.Thread.State: RUNNABLE
          at java.util.jar.JarFile.getBytes(JarFile.java:358)
          at java.util.jar.JarFile.initializeVerifier(JarFile.java:317)
          at java.util.jar.JarFile.getInputStream(JarFile.java:386)
          - locked <0x76b40030> (a java.util.jar.JarFile)
          at org.jboss.virtual.plugins.context.zip.ZipFileWrapper.openStream(ZipFileWrapper.java:215)
          - locked <0x76b40170> (a org.jboss.virtual.plugins.context.zip.ZipFileWrapper)
          at org.jboss.virtual.plugins.context.zip.ZipEntryContext.openStream(ZipEntryContext.java:1084)
          at org.jboss.virtual.plugins.context.zip.ZipEntryHandler.openStream(ZipEntryHandler.java:154)
          at org.jboss.virtual.VirtualFile.openStream(VirtualFile.java:241)
          at org.jboss.virtual.plugins.vfs.VirtualFileURLConnection.getInputStream(VirtualFileURLConnection.java:93)
          at java.net.URL.openStream(URL.java:1010)
          at java.lang.ClassLoader.getResourceAsStream(ClassLoader.java:1150)
          at org.jboss.wsf.spi.util.ServiceLoader.loadFromServices(ServiceLoader.java:81)
          at org.jboss.wsf.spi.util.ServiceLoader.loadService(ServiceLoader.java:59)
          at org.jboss.ws.core.soap.MessageFactoryImpl.createMessage(MessageFactoryImpl.java:289)
          at org.jboss.ws.core.soap.SOAPMessageUnMarshallerHTTP.read(SOAPMessageUnMarshallerHTTP.java:82)
          at org.jboss.remoting.transport.http.HTTPClientInvoker.readResponse(HTTPClientInvoker.java:584)
          at org.jboss.remoting.transport.http.HTTPClientInvoker.useHttpURLConnection(HTTPClientInvoker.java:381)
          at org.jboss.remoting.transport.http.HTTPClientInvoker.makeInvocation(HTTPClientInvoker.java:232)
          at org.jboss.remoting.transport.http.HTTPClientInvoker.transport(HTTPClientInvoker.java:162)
          at org.jboss.remoting.MicroRemoteClientInvoker.invoke(MicroRemoteClientInvoker.java:167)
          at org.jboss.remoting.Client.invoke(Client.java:1917)
          at org.jboss.remoting.Client.invoke(Client.java:768)
          at org.jboss.ws.core.client.HTTPRemotingConnection.invoke(HTTPRemotingConnection.java:232)
          at org.jboss.ws.core.client.SOAPProtocolConnectionHTTP.invoke(SOAPProtocolConnectionHTTP.java:71)
          at org.jboss.ws.core.CommonClient.invoke(CommonClient.java:339)
          at org.jboss.ws.core.jaxws.client.ClientImpl.invoke(ClientImpl.java:231)
          at org.jboss.ws.core.jaxws.client.ClientProxy.invoke(ClientProxy.java:162)
          at org.jboss.ws.core.jaxws.client.ClientProxy.invoke(ClientProxy.java:148)
          at $Proxy245.executeCommand(Unknown Source)
  • Application is responding very slow and most of the threads are blocked state and are waiting to acquire lock on a thread like below :
            Thread: pool-34-thread-1 : priority:5, demon:false, threadId:290, threadState:RUNNABLE

                java.util.zip.ZipFile.open(Native Method)
                java.util.zip.ZipFile.(ZipFile.java:127)
                java.util.jar.JarFile.(JarFile.java:135)
                java.util.jar.JarFile.(JarFile.java:99)
                org.jboss.virtual.plugins.context.zip.ZipFileWrapper.ensureZipFile(ZipFileWrapper.java:173)
                org.jboss.virtual.plugins.context.zip.ZipFileWrapper.openStream(ZipFileWrapper.java:211)
                - locked <0x11ac8b5d> (a org.jboss.virtual.plugins.context.zip.ZipFileWrapper)
                org.jboss.virtual.plugins.context.zip.ZipEntryContext.openStream(ZipEntryContext.java:1084)
                org.jboss.virtual.plugins.context.zip.ZipEntryHandler.openStream(ZipEntryHandler.java:154)
                org.jboss.virtual.VirtualFile.openStream(VirtualFile.java:241)
                org.jboss.virtual.plugins.vfs.VirtualFileURLConnection.getInputStream(VirtualFileURLConnection.java:93)
                java.net.URL.openStream(URL.java:1010)
                java.lang.ClassLoader.getResourceAsStream(ClassLoader.java:1194)
                javax.xml.bind.ContextFinder.find(ContextFinder.java:302)
                javax.xml.bind.JAXBContext.newInstance(JAXBContext.java:412)
                javax.xml.bind.JAXBContext.newInstance(JAXBContext.java:375)
                javax.xml.bind.JAXBContext.newInstance(JAXBContext.java:279)
                com.sun.xml.ws.assembler.MetroConfigLoader.loadMetroConfig(MetroConfigLoader.java:212)
                com.sun.xml.ws.assembler.MetroConfigLoader.(MetroConfigLoader.java:115)
                com.sun.xml.ws.assembler.MetroConfigLoader.(MetroConfigLoader.java:105)
                com.sun.xml.ws.assembler.TubelineAssemblyController.getTubeCreators(TubelineAssemblyController.java:86)
                com.sun.xml.ws.assembler.TubelineAssemblerFactoryImpl$MetroTubelineAssembler.createClient(TubelineAssemblerFactoryImpl.java:114)
                com.sun.xml.ws.client.Stub.createPipeline(Stub.java:228)
                com.sun.xml.ws.client.Stub.(Stub.java:205)
                com.sun.xml.ws.client.Stub.(Stub.java:178)
                com.sun.xml.ws.client.sei.SEIStub.(SEIStub.java:85)
                com.sun.xml.ws.client.WSServiceDelegate.createEndpointIFBaseProxy(WSServiceDelegate.java:608)
                com.sun.xml.ws.client.WSServiceDelegate.getPort(WSServiceDelegate.java:348)
                com.sun.xml.ws.client.WSServiceDelegate.getPort(WSServiceDelegate.java:330)
                com.sun.xml.ws.client.WSServiceDelegate.getPort(WSServiceDelegate.java:312)
                javax.xml.ws.Service.getPort(Service.java:134)

Environment

  • JBoss Enterprise Application Platform (EAP)
    • 5.0.0

Subscriber exclusive content

A Red Hat subscription provides unlimited access to our knowledgebase, tools, and much more.

Current Customers and Partners

Log in for full access

Log In

New to Red Hat?

Learn more about Red Hat subscriptions

Using a Red Hat product through a public cloud?

How to access this content