A RHDG cluster seems blocked an not longer resposive if expiration with max-idle is used
Issue
- A DG cluster will stuck and clients will not get any response, the grid is back to work if we kill one node which is suspected as root cause
- Data Grid is completely unresponsive and there are blocked threads shown whithin the thread dumps like followed
"HotRod-hotrod-internalServerWorker-4-12" #279 prio=5 os_prio=0 tid=0x00007f35980b8800 nid=0xbbb waiting for monitor entry [0x00007f356cfb4000]
java.lang.Thread.State: BLOCKED (on object monitor)
at org.infinispan.commons.util.concurrent.jdk8backported.EquivalentConcurrentHashMapV8.compute(EquivalentConcurrentHashMapV8.java:1910)
waiting to lock <0x0000000742cca028> (a org.infinispan.commons.util.concurrent.jdk8backported.EquivalentConcurrentHashMapV8$Node)
at org.infinispan.container.DefaultDataContainer.compute(DefaultDataContainer.java:335)
at org.infinispan.expiration.impl.ExpirationManagerImpl.handleInMemoryExpiration(ExpirationManagerImpl.java:135)
at org.infinispan.expiration.impl.ClusterExpirationManager.handleInMemoryExpiration(ClusterExpirationManager.java:152)
locked <0x0000000742cc9898> (a org.infinispan.container.entries.metadata.MetadataTransientCacheEntry)
at org.infinispan.container.DefaultDataContainer.get(DefaultDataContainer.java:201)
"pool-7-thread-1" #158 prio=5 os_prio=0 tid=0x00007f35c816c000 nid=0xb35 waiting for monitor entry [0x00007f3575238000]
java.lang.Thread.State: BLOCKED (on object monitor)
at org.infinispan.expiration.impl.ExpirationManagerImpl.lambda$handleInMemoryExpiration$0(ExpirationManagerImpl.java:137)
waiting to lock <0x0000000742cc9898> (a org.infinispan.container.entries.metadata.MetadataTransientCacheEntry)
at org.infinispan.expiration.impl.ExpirationManagerImpl$$Lambda$374/1904915245.compute(Unknown Source)
at org.infinispan.container.DefaultDataContainer.lambda$compute$3(DefaultDataContainer.java:336)
at org.infinispan.container.DefaultDataContainer$$Lambda$375/1917382785.apply(Unknown Source)
at org.infinispan.commons.util.concurrent.jdk8backported.EquivalentConcurrentHashMapV8.compute(EquivalentConcurrentHashMapV8.java:1919)
locked <0x0000000742cca028> (a org.infinispan.commons.util.concurrent.jdk8backported.EquivalentConcurrentHashMapV8$Node)
at org.infinispan.container.DefaultDataContainer.compute(DefaultDataContainer.java:335)
at org.infinispan.expiration.impl.ExpirationManagerImpl.handleInMemoryExpiration(ExpirationManagerImpl.java:135)
at org.infinispan.expiration.impl.ClusterExpirationManager.processExpiration(ClusterExpirationManager.java:82)
Environment
- Red Hat Data Grid (RHDG)
- 7
Subscriber exclusive content
A Red Hat subscription provides unlimited access to our knowledgebase, tools, and much more.