JBoss EAP 7 does not process requests for a session in parallel
Environment
- JBoss Enterprise Application Platform (EAP)
- 7.x
Issue
- When a SOAP call is called while an async servlet call is hanging, the SOAP call fails to aquire a lock on the HTTP session.
-
Errors when a request times out waiting on a session lock:
ERROR [org.infinispan.interceptors.InvocationContextInterceptor] (default task-79) ISPN000136: Error executing command GetKeyValueCommand, writing keys []: org.infinispan.util.concurrent.TimeoutException: ISPN000299: Unable to acquire lock after 15 seconds for key ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmn and requestor GlobalTransaction::45:local. Lock is held by GlobalTransaction::43:local
-
Getting exception while accessing session
org.infinispan.util.concurrent.TimeoutException: ISPN000299: Unable to acquire lock after 15 seconds for key SessionCreationMetaDataKey(ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmn) and requestor GlobalTx:node1:1001. Lock is held by GlobalTx:node1:1000
-
JBoss EAP 7 only processes a single request for a given session at a time.
Resolution
You can allow concurrent access by modifying the web
session replication cache configuration to remove <locking>
and <transaction>
completely. It is also necessary to remove the "mode" attribute to use the default setting of SYNC in EAP 7.1+.
-
For
ha
based configurations (standalone-ha.xml
/standalone-full-ha.xml
):<cache-container name="web" default-cache="dist" module="org.wildfly.clustering.web.infinispan"> <transport lock-timeout="60000"/> <distributed-cache name="dist" mode="ASYNC" l1-lifespan="0" owners="2"> <!-- IN 7.1+ REMOVE mode="..." IF PRESENT --> <locking isolation="REPEATABLE_READ"/> <!-- REMOVE THIS LINE --> <transaction mode="BATCH"/> <!-- REMOVE THIS LINE --> <file-store/> </distributed-cache> </cache-container>
You can execute the following CLI:
/subsystem=infinispan/cache-container=web/distributed-cache=dist/component=locking:remove /subsystem=infinispan/cache-container=web/distributed-cache=dist/component=transaction:remove
If you are using a
<replicated-cache>
or<local-cache>
instead of<distributed-cache>
for web sessions, you'll still remove the same<locking>
and<transaction>
tags from the corresponding cache configuration, and in the CLI command change/distributed-cache=dist/
to the appropriate type and cache name.
NOTE :
- However there were bugs that prevent the above configuration from working in EAP 7.0.3 or before. It has been fixed in EAP 7.0.4 (JBEAP-6752) and 7.1.0 (JBEAP-4128) or later, so you can configure the above setting in the newer versions.
- If replicated-cache (non-transactional + without locking), Fine (ATTRIBUTE) replication-granularity, and removing locking/transaction are set in JBoss EAP 7.3.4 or earlier, there is a known memory leak issue JBEAP-20396, fixed in JBoss EAP 7.3.5.
- Other known issue when locking/transaction are removed, fixed in JBoss EAP 7.2.8 and 7.3.1: Session attribute lost issue with the ATTRIBUTE replication-granularity + non-BATCH cache in a failover scenario in JBoss EAP 7
Root Cause
- The servlet spec requires that requests for a session are only processed on a single node at once.
- EAP 7.0 implements this by only processing a single request for a session at once cluster-wide.
Related article
This solution is part of Red Hat’s fast-track publication program, providing a huge library of solutions that Red Hat engineers have created while supporting our customers. To give you the knowledge you need the instant it becomes available, these articles may be presented in a raw and unedited form.
Comments