Session blocked when using distributable applications and cross-context forward/include in EAP 7.1

Solution Verified - Updated -

Issue

  • When using distributable applications (session replication using <distributable/> in the web.xml) and doing cross-context access between them (one application includes or forwards to a resource from another application) the application gets blocked. The resulting exception is similar to:

    ERROR [io.undertow.request] (default task-11) UT005023: Exception handling request to /webapp1/servlet1: java.lang.RuntimeException: java.lang.RuntimeException: org.infinispan.util.concurrent.TimeoutException: ISPN000299: Unable to acquire lock after 15 seconds for key SessionCreationMetaDataKey(x9ah3VdYkbEentlDFomBMhRR_eilSE7UzlgPtVDI) and requestor GlobalTransaction:<null>:22:local. Lock is held by GlobalTransaction:<null>:14:local
            at io.undertow.servlet.spec.ServletContextImpl.invokeAction(ServletContextImpl.java:934)
            at io.undertow.servlet.spec.RequestDispatcherImpl.setupIncludeImpl(RequestDispatcherImpl.java:299)
            at io.undertow.servlet.spec.RequestDispatcherImpl.include(RequestDispatcherImpl.java:277)
            at com.example.Servlet1.service(Servlet1.java:48)
            at javax.servlet.http.HttpServlet.service(HttpServlet.java:790)
            ...
    Caused by: java.lang.RuntimeException: org.infinispan.util.concurrent.TimeoutException: ISPN000299: Unable to acquire lock after 15 seconds for key SessionCreationMetaDataKey(x9ah3VdYkbEentlDFomBMhRR_eilSE7UzlgPtVDI) and requestor GlobalTransaction:<null>:22:local. Lock is held by GlobalTransaction:<null>:14:local
            at io.undertow.servlet.spec.RequestDispatcherImpl.includeImpl(RequestDispatcherImpl.java:382)
            at io.undertow.servlet.spec.RequestDispatcherImpl.access$300(RequestDispatcherImpl.java:54)
            at io.undertow.servlet.spec.RequestDispatcherImpl$4.call(RequestDispatcherImpl.java:302)
            at io.undertow.servlet.spec.RequestDispatcherImpl$4.call(RequestDispatcherImpl.java:299)
            ...
    Caused by: org.infinispan.util.concurrent.TimeoutException: ISPN000299: Unable to acquire lock after 15 seconds for key SessionCreationMetaDataKey(x9ah3VdYkbEentlDFomBMhRR_eilSE7UzlgPtVDI) and requestor GlobalTransaction:<null>:22:local. Lock is held by GlobalTransaction:<null>:14:local
            at org.infinispan.util.concurrent.locks.impl.DefaultLockManager$KeyAwareExtendedLockPromise.lock(DefaultLockManager.java:239)
            at org.infinispan.interceptors.locking.AbstractLockingInterceptor.lockAndRecord(AbstractLockingInterceptor.java:193)
            at org.infinispan.interceptors.locking.AbstractTxLockingInterceptor.checkPendingAndLockKey(AbstractTxLockingInterceptor.java:193)
            at org.infinispan.interceptors.locking.AbstractTxLockingInterceptor.lockOrRegisterBackupLock(AbstractTxLockingInterceptor.java:116)
            at org.infinispan.interceptors.locking.PessimisticLockingInterceptor.visitDataReadCommand(PessimisticLockingInterceptor.java:71)
            at org.infinispan.interceptors.locking.AbstractLockingInterceptor.visitGetKeyValueCommand(AbstractLockingInterceptor.java:80)
            at org.infinispan.commands.read.GetKeyValueCommand.acceptVisitor(GetKeyValueCommand.java:43)
            at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:99)
            at org.infinispan.interceptors.TxInterceptor.enlistReadAndInvokeNext(TxInterceptor.java:346)
            at org.infinispan.interceptors.TxInterceptor.visitGetKeyValueCommand(TxInterceptor.java:331)
            at org.infinispan.commands.read.GetKeyValueCommand.acceptVisitor(GetKeyValueCommand.java:43)
            at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:99)
            at org.infinispan.interceptors.InvocationContextInterceptor.handleAll(InvocationContextInterceptor.java:110)
            at org.infinispan.interceptors.InvocationContextInterceptor.handleDefault(InvocationContextInterceptor.java:79)
            at org.infinispan.commands.AbstractVisitor.visitGetKeyValueCommand(AbstractVisitor.java:85)
            at org.infinispan.commands.read.GetKeyValueCommand.acceptVisitor(GetKeyValueCommand.java:43)
            at org.infinispan.interceptors.InterceptorChain.invoke(InterceptorChain.java:335)
            at org.infinispan.cache.impl.CacheImpl.get(CacheImpl.java:414)
            at org.infinispan.cache.impl.DecoratedCache.get(DecoratedCache.java:443)
            at org.infinispan.cache.impl.AbstractDelegatingCache.get(AbstractDelegatingCache.java:287)
            at org.wildfly.clustering.web.infinispan.session.InfinispanSessionMetaDataFactory.getValue(InfinispanSessionMetaDataFactory.java:74)
            at org.wildfly.clustering.web.infinispan.session.InfinispanSessionMetaDataFactory.findValue(InfinispanSessionMetaDataFactory.java:64)
            at org.wildfly.clustering.web.infinispan.session.InfinispanSessionMetaDataFactory.findValue(InfinispanSessionMetaDataFactory.java:36)
            at org.wildfly.clustering.web.infinispan.session.InfinispanSessionFactory.findValue(InfinispanSessionFactory.java:60)
            at org.wildfly.clustering.web.infinispan.session.InfinispanSessionFactory.findValue(InfinispanSessionFactory.java:38)
            at org.wildfly.clustering.web.infinispan.session.InfinispanSessionManager.findSession(InfinispanSessionManager.java:249)
            at org.wildfly.clustering.web.undertow.session.DistributableSessionManager.getSession(DistributableSessionManager.java:209)
            at io.undertow.servlet.spec.ServletContextImpl.getSession(ServletContextImpl.java:773)
            at io.undertow.servlet.spec.HttpServletRequestImpl.getSession(HttpServletRequestImpl.java:368)
            at io.undertow.servlet.spec.HttpServletRequestImpl.getSession(HttpServletRequestImpl.java:373)
            at com.example.Servlet2.service(Servlet2.java:32)
            at javax.servlet.http.HttpServlet.service(HttpServlet.java:790)
            at io.undertow.servlet.handlers.ServletHandler.handleRequest(ServletHandler.java:74)
            ...
    
  • Applications do not undeploy/stop, they are blocked in a similar stack to the one commented in solution: EAP 7.1 shutdown blocked by DistributableSessionManager.stop.

    "ServerService Thread Pool -- 107" #2334 prio=5 os_prio=0 tid=0x000000001aee0000 nid=0x53e8 waiting on condition [0x00007fad3b6f0000]
       java.lang.Thread.State: WAITING (parking)
        at sun.misc.Unsafe.park(Native Method)
        - parking to wait for  <0x00000006dc2b7ae8> (a java.util.concurrent.locks.StampedLock)
        at java.util.concurrent.locks.StampedLock.acquireWrite(StampedLock.java:1119)
        at java.util.concurrent.locks.StampedLock.writeLockInterruptibly(StampedLock.java:413)
        at org.wildfly.clustering.web.undertow.session.DistributableSessionManager.stop(DistributableSessionManager.java:90)
        - locked <0x00000006dc2b7ac0> (a org.wildfly.clustering.web.undertow.session.DistributableSessionManager)
        ...
    

Environment

  • JBoss Enterprise Application Platform (EAP)
    • 7

Subscriber exclusive content

A Red Hat subscription provides unlimited access to our knowledgebase, tools, and much more.

Current Customers and Partners

Log in for full access

Log In