Issue

• When using distributable applications (session replication using <distributable/> in the web.xml) and doing cross-context access between them (one application includes or forwards to a resource from another application) the application gets blocked. The resulting exception is similar to:

  ERROR [io.undertow.request] (default task-11) UT005023: Exception handling request to /webapp1/servlet1: java.lang.RuntimeException: java.lang.RuntimeException: org.infinispan.util.concurrent.TimeoutException: ISPN000299: Unable to acquire lock after 15 seconds for key SessionCreationMetaDataKey(x9ah3VdYkbEentlDFomBMhRR_eilSE7UzlgPtVDI) and requestor GlobalTransaction:<null>:22:local. Lock is held by GlobalTransaction:<null>:14:local
          at io.undertow.servlet.spec.ServletContextImpl.invokeAction(ServletContextImpl.java:934)
          at io.undertow.servlet.spec.RequestDispatcherImpl.setupIncludeImpl(RequestDispatcherImpl.java:299)
          at io.undertow.servlet.spec.RequestDispatcherImpl.include(RequestDispatcherImpl.java:277)
          at com.example.Servlet1.service(Servlet1.java:48)
          at javax.servlet.http.HttpServlet.service(HttpServlet.java:790)
          ...
  Caused by: java.lang.RuntimeException: org.infinispan.util.concurrent.TimeoutException: ISPN000299: Unable to acquire lock after 15 seconds for key SessionCreationMetaDataKey(x9ah3VdYkbEentlDFomBMhRR_eilSE7UzlgPtVDI) and requestor GlobalTransaction:<null>:22:local. Lock is held by GlobalTransaction:<null>:14:local
          at io.undertow.servlet.spec.RequestDispatcherImpl.includeImpl(RequestDispatcherImpl.java:382)
          at io.undertow.servlet.spec.RequestDispatcherImpl.access$300(RequestDispatcherImpl.java:54)
          at io.undertow.servlet.spec.RequestDispatcherImpl$4.call(RequestDispatcherImpl.java:302)
          at io.undertow.servlet.spec.RequestDispatcherImpl$4.call(RequestDispatcherImpl.java:299)
          ...
  Caused by: org.infinispan.util.concurrent.TimeoutException: ISPN000299: Unable to acquire lock after 15 seconds for key SessionCreationMetaDataKey(x9ah3VdYkbEentlDFomBMhRR_eilSE7UzlgPtVDI) and requestor GlobalTransaction:<null>:22:local. Lock is held by GlobalTransaction:<null>:14:local
          at org.infinispan.util.concurrent.locks.impl.DefaultLockManager$KeyAwareExtendedLockPromise.lock(DefaultLockManager.java:239)
          at org.infinispan.interceptors.locking.AbstractLockingInterceptor.lockAndRecord(AbstractLockingInterceptor.java:193)
          at org.infinispan.interceptors.locking.AbstractTxLockingInterceptor.checkPendingAndLockKey(AbstractTxLockingInterceptor.java:193)
          at org.infinispan.interceptors.locking.AbstractTxLockingInterceptor.lockOrRegisterBackupLock(AbstractTxLockingInterceptor.java:116)
          at org.infinispan.interceptors.locking.PessimisticLockingInterceptor.visitDataReadCommand(PessimisticLockingInterceptor.java:71)
          at org.infinispan.interceptors.locking.AbstractLockingInterceptor.visitGetKeyValueCommand(AbstractLockingInterceptor.java:80)
          at org.infinispan.commands.read.GetKeyValueCommand.acceptVisitor(GetKeyValueCommand.java:43)
          at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:99)
          at org.infinispan.interceptors.TxInterceptor.enlistReadAndInvokeNext(TxInterceptor.java:346)
          at org.infinispan.interceptors.TxInterceptor.visitGetKeyValueCommand(TxInterceptor.java:331)
          at org.infinispan.commands.read.GetKeyValueCommand.acceptVisitor(GetKeyValueCommand.java:43)
          at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:99)
          at org.infinispan.interceptors.InvocationContextInterceptor.handleAll(InvocationContextInterceptor.java:110)
          at org.infinispan.interceptors.InvocationContextInterceptor.handleDefault(InvocationContextInterceptor.java:79)
          at org.infinispan.commands.AbstractVisitor.visitGetKeyValueCommand(AbstractVisitor.java:85)
          at org.infinispan.commands.read.GetKeyValueCommand.acceptVisitor(GetKeyValueCommand.java:43)
          at org.infinispan.interceptors.InterceptorChain.invoke(InterceptorChain.java:335)
          at org.infinispan.cache.impl.CacheImpl.get(CacheImpl.java:414)
          at org.infinispan.cache.impl.DecoratedCache.get(DecoratedCache.java:443)
          at org.infinispan.cache.impl.AbstractDelegatingCache.get(AbstractDelegatingCache.java:287)
          at org.wildfly.clustering.web.infinispan.session.InfinispanSessionMetaDataFactory.getValue(InfinispanSessionMetaDataFactory.java:74)
          at org.wildfly.clustering.web.infinispan.session.InfinispanSessionMetaDataFactory.findValue(InfinispanSessionMetaDataFactory.java:64)
          at org.wildfly.clustering.web.infinispan.session.InfinispanSessionMetaDataFactory.findValue(InfinispanSessionMetaDataFactory.java:36)
          at org.wildfly.clustering.web.infinispan.session.InfinispanSessionFactory.findValue(InfinispanSessionFactory.java:60)
          at org.wildfly.clustering.web.infinispan.session.InfinispanSessionFactory.findValue(InfinispanSessionFactory.java:38)
          at org.wildfly.clustering.web.infinispan.session.InfinispanSessionManager.findSession(InfinispanSessionManager.java:249)
          at org.wildfly.clustering.web.undertow.session.DistributableSessionManager.getSession(DistributableSessionManager.java:209)
          at io.undertow.servlet.spec.ServletContextImpl.getSession(ServletContextImpl.java:773)
          at io.undertow.servlet.spec.HttpServletRequestImpl.getSession(HttpServletRequestImpl.java:368)
          at io.undertow.servlet.spec.HttpServletRequestImpl.getSession(HttpServletRequestImpl.java:373)
          at com.example.Servlet2.service(Servlet2.java:32)
          at javax.servlet.http.HttpServlet.service(HttpServlet.java:790)
          at io.undertow.servlet.handlers.ServletHandler.handleRequest(ServletHandler.java:74)
          ...
  

• Applications do not undeploy/stop, they are blocked in a similar stack to the one commented in solution: EAP 7.1 shutdown blocked by DistributableSessionManager.stop.

  "ServerService Thread Pool -- 107" #2334 prio=5 os_prio=0 tid=0x000000001aee0000 nid=0x53e8 waiting on condition [0x00007fad3b6f0000]
     java.lang.Thread.State: WAITING (parking)
      at sun.misc.Unsafe.park(Native Method)
      - parking to wait for  <0x00000006dc2b7ae8> (a java.util.concurrent.locks.StampedLock)
      at java.util.concurrent.locks.StampedLock.acquireWrite(StampedLock.java:1119)
      at java.util.concurrent.locks.StampedLock.writeLockInterruptibly(StampedLock.java:413)
      at org.wildfly.clustering.web.undertow.session.DistributableSessionManager.stop(DistributableSessionManager.java:90)
      - locked <0x00000006dc2b7ac0> (a org.wildfly.clustering.web.undertow.session.DistributableSessionManager)
      ...