ISPN000136: Error executing command PutKeyValueCommand on Cache

Solution Verified - Updated -

Issue

  • Multiple Nodes trying to update the same key at the same time. This can cause deadlocks, as each one locks the
    key locally and then tries to replicate it. But the replication cannot acquire the lock on the remote node
    that the other writer is holding.
  • DataGrid GUI is completely frozen when the locking issue occurs.
WARN  org.infinispan.HOTROD.handleException - ISPN004098: Closing connection [id: XXXX, L:/y.y.y.y:61375 - R:/x.xx.x.x:11222] due to transport errorjava.net.SocketTimeoutException: PutAllOperation{user-cache, flags=6, connection=/x.x.x.x:11222} timed out after 60000 ms
    at org.infinispan.client.hotrod.impl.operations.HotRodOperation.run(HotRodOperation.java:182)
    at io.netty.util.concurrent.PromiseTask.runTask(PromiseTask.java:98)
ERROR (non-blocking-thread-node-3-p2-t20) [org.infinispan.interceptors.impl.InvocationContextInterceptor] ISPN000136: Error executing command LockControlCommand on Cache 'XXX', writing keys [] org.infinispan.util.concurrent.TimeoutException: ISPN000299: Unable to acquire lock after 60 seconds for key WrappedByteArray[\J0F\2\3\3\4\1\1\4\0\7\4\4\6\8\9\2 (17 bytes)] and requestor GlobalTransaction{id=1690332, addr=node-1, remote=true, xid=null, internalId=-1}. Lock is held by GlobalTransaction{id=12345, addr=node-3, remote=false, xid=null, internalId=-1}

Environment

  • Red Hat Data Grid
    • 8.3.1

Subscriber exclusive content

A Red Hat subscription provides unlimited access to our knowledgebase, tools, and much more.

Current Customers and Partners

Log in for full access

Log In

New to Red Hat?

Learn more about Red Hat subscriptions

Using a Red Hat product through a public cloud?

How to access this content