Session blocked when using distributable applications and cross-context forward/include in EAP 7.1

Solution Verified - Updated -

Issue

  • When using distributable applications (session replication using <distributable/> in the web.xml) and doing cross-context access between them (one application includes or forwards to a resource from another application) the application gets blocked. The resulting exception is similar to:

    ERROR [io.undertow.request] (default task-11) UT005023: Exception handling request to /webapp1/servlet1: java.lang.RuntimeException: java.lang.RuntimeException: org.infinispan.util.concurrent.TimeoutException: ISPN000299: Unable to acquire lock after 15 seconds for key SessionCreationMetaDataKey(x9ah3VdYkbEentlDFomBMhRR_eilSE7UzlgPtVDI) and requestor GlobalTransaction:<null>:22:local. Lock is held by GlobalTransaction:<null>:14:local
            at io.undertow.servlet.spec.ServletContextImpl.invokeAction(ServletContextImpl.java:934)
            at io.undertow.servlet.spec.RequestDispatcherImpl.setupIncludeImpl(RequestDispatcherImpl.java:299)
            at io.undertow.servlet.spec.RequestDispatcherImpl.include(RequestDispatcherImpl.java:277)
            at com.example.Servlet1.service(Servlet1.java:48)
            at javax.servlet.http.HttpServlet.service(HttpServlet.java:790)
            ...
    Caused by: java.lang.RuntimeException: org.infinispan.util.concurrent.TimeoutException: ISPN000299: Unable to acquire lock after 15 seconds for key SessionCreationMetaDataKey(x9ah3VdYkbEentlDFomBMhRR_eilSE7UzlgPtVDI) and requestor GlobalTransaction:<null>:22:local. Lock is held by GlobalTransaction:<null>:14:local
            at io.undertow.servlet.spec.RequestDispatcherImpl.includeImpl(RequestDispatcherImpl.java:382)
            at io.undertow.servlet.spec.RequestDispatcherImpl.access$300(RequestDispatcherImpl.java:54)
            at io.undertow.servlet.spec.RequestDispatcherImpl$4.call(RequestDispatcherImpl.java:302)
            at io.undertow.servlet.spec.RequestDispatcherImpl$4.call(RequestDispatcherImpl.java:299)
            ...
    Caused by: org.infinispan.util.concurrent.TimeoutException: ISPN000299: Unable to acquire lock after 15 seconds for key SessionCreationMetaDataKey(x9ah3VdYkbEentlDFomBMhRR_eilSE7UzlgPtVDI) and requestor GlobalTransaction:<null>:22:local. Lock is held by GlobalTransaction:<null>:14:local
            at org.infinispan.util.concurrent.locks.impl.DefaultLockManager$KeyAwareExtendedLockPromise.lock(DefaultLockManager.java:239)
            at org.infinispan.interceptors.locking.AbstractLockingInterceptor.lockAndRecord(AbstractLockingInterceptor.java:193)
            at org.infinispan.interceptors.locking.AbstractTxLockingInterceptor.checkPendingAndLockKey(AbstractTxLockingInterceptor.java:193)
            at org.infinispan.interceptors.locking.AbstractTxLockingInterceptor.lockOrRegisterBackupLock(AbstractTxLockingInterceptor.java:116)
            at org.infinispan.interceptors.locking.PessimisticLockingInterceptor.visitDataReadCommand(PessimisticLockingInterceptor.java:71)
            at org.infinispan.interceptors.locking.AbstractLockingInterceptor.visitGetKeyValueCommand(AbstractLockingInterceptor.java:80)
            at org.infinispan.commands.read.GetKeyValueCommand.acceptVisitor(GetKeyValueCommand.java:43)
            at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:99)
            at org.infinispan.interceptors.TxInterceptor.enlistReadAndInvokeNext(TxInterceptor.java:346)
            at org.infinispan.interceptors.TxInterceptor.visitGetKeyValueCommand(TxInterceptor.java:331)
            at org.infinispan.commands.read.GetKeyValueCommand.acceptVisitor(GetKeyValueCommand.java:43)
            at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:99)
            at org.infinispan.interceptors.InvocationContextInterceptor.handleAll(InvocationContextInterceptor.java:110)
            at org.infinispan.interceptors.InvocationContextInterceptor.handleDefault(InvocationContextInterceptor.java:79)
            at org.infinispan.commands.AbstractVisitor.visitGetKeyValueCommand(AbstractVisitor.java:85)
            at org.infinispan.commands.read.GetKeyValueCommand.acceptVisitor(GetKeyValueCommand.java:43)
            at org.infinispan.interceptors.InterceptorChain.invoke(InterceptorChain.java:335)
            at org.infinispan.cache.impl.CacheImpl.get(CacheImpl.java:414)
            at org.infinispan.cache.impl.DecoratedCache.get(DecoratedCache.java:443)
            at org.infinispan.cache.impl.AbstractDelegatingCache.get(AbstractDelegatingCache.java:287)
            at org.wildfly.clustering.web.infinispan.session.InfinispanSessionMetaDataFactory.getValue(InfinispanSessionMetaDataFactory.java:74)
            at org.wildfly.clustering.web.infinispan.session.InfinispanSessionMetaDataFactory.findValue(InfinispanSessionMetaDataFactory.java:64)
            at org.wildfly.clustering.web.infinispan.session.InfinispanSessionMetaDataFactory.findValue(InfinispanSessionMetaDataFactory.java:36)
            at org.wildfly.clustering.web.infinispan.session.InfinispanSessionFactory.findValue(InfinispanSessionFactory.java:60)
            at org.wildfly.clustering.web.infinispan.session.InfinispanSessionFactory.findValue(InfinispanSessionFactory.java:38)
            at org.wildfly.clustering.web.infinispan.session.InfinispanSessionManager.findSession(InfinispanSessionManager.java:249)
            at org.wildfly.clustering.web.undertow.session.DistributableSessionManager.getSession(DistributableSessionManager.java:209)
            at io.undertow.servlet.spec.ServletContextImpl.getSession(ServletContextImpl.java:773)
            at io.undertow.servlet.spec.HttpServletRequestImpl.getSession(HttpServletRequestImpl.java:368)
            at io.undertow.servlet.spec.HttpServletRequestImpl.getSession(HttpServletRequestImpl.java:373)
            at com.example.Servlet2.service(Servlet2.java:32)
            at javax.servlet.http.HttpServlet.service(HttpServlet.java:790)
            at io.undertow.servlet.handlers.ServletHandler.handleRequest(ServletHandler.java:74)
            ...
    
  • Applications do not undeploy/stop, they are blocked in a similar stack to the one commented in solution: EAP 7.1 shutdown blocked by DistributableSessionManager.stop.

    "ServerService Thread Pool -- 107" #2334 prio=5 os_prio=0 tid=0x000000001aee0000 nid=0x53e8 waiting on condition [0x00007fad3b6f0000]
       java.lang.Thread.State: WAITING (parking)
        at sun.misc.Unsafe.park(Native Method)
        - parking to wait for  <0x00000006dc2b7ae8> (a java.util.concurrent.locks.StampedLock)
        at java.util.concurrent.locks.StampedLock.acquireWrite(StampedLock.java:1119)
        at java.util.concurrent.locks.StampedLock.writeLockInterruptibly(StampedLock.java:413)
        at org.wildfly.clustering.web.undertow.session.DistributableSessionManager.stop(DistributableSessionManager.java:90)
        - locked <0x00000006dc2b7ac0> (a org.wildfly.clustering.web.undertow.session.DistributableSessionManager)
        ...
    

Environment

  • JBoss Enterprise Application Platform (EAP)
    • 7

Subscriber exclusive content

A Red Hat subscription provides unlimited access to our knowledgebase, tools, and much more.

Current Customers and Partners

Log in for full access

Log In

New to Red Hat?

Learn more about Red Hat subscriptions

Using a Red Hat product through a public cloud?

How to access this content