Data Grid in OCP 4 pod gets ___protobuf_metadata on cluster without state
Issue
Customer has one pod and cannot spawn another, because it gives the follow issue and goes on crashback loop:
18:44:07,871 WARN (jgroups-22,iss-prd-infinispan-0-14970) [org.infinispan.CLUSTER] ISPN000071: Caught exception when handling command TopologyJoinCommand{cacheName='___protobuf_metadata', origin=iss-prd-infinispan-1-59998, joinInfo=CacheJoinInfo{consistentHashFactory=org.infinispan.distribution.ch.impl.SyncReplicatedConsistentHashFactory@5f17220d, numSegments=256, numOwners=2, timeout=240000, cacheMode=REPL_SYNC, persistentUUID=203c3e23-829d-40af-a160-23078d9b58a0, persistentStateChecksum=Optional[601209598]}, viewId=3} org.infinispan.topology.CacheJoinException: ISPN000408: Node a-node-????? with persistent state attempting to join cache ___protobuf_metadata on cluster without state
at org.infinispan.topology.ClusterCacheStatus.addMember(ClusterCacheStatus.java:237)
at org.infinispan.topology.ClusterCacheStatus.doJoin(ClusterCacheStatus.java:703)
at org.infinispan.topology.ClusterTopologyManagerImpl.lambda$handleJoin$3(ClusterTopologyManagerImpl.java:233)
at java.base/java.util.concurrent.CompletableFuture.uniApplyNow(CompletableFuture.java:680)
at java.base/java.util.concurrent.CompletableFuture.uniApplyStage(CompletableFuture.java:658)
at java.base/java.util.concurrent.CompletableFuture.thenApply(CompletableFuture.java:2094)
at java.base/java.util.concurrent.CompletableFuture.thenApply(CompletableFuture.java:143)
at org.infinispan.topology.ClusterTopologyManagerImpl.lambda$handleJoin$4(ClusterTopologyManagerImpl.java:233)
at java.base/java.util.concurrent.CompletableFuture.uniComposeStage(CompletableFuture.java:1106)
at java.base/java.util.concurrent.CompletableFuture.thenCompose(CompletableFuture.java:2235)
at java.base/java.util.concurrent.CompletableFuture.thenCompose(CompletableFuture.java:143)
at org.infinispan.topology.ClusterTopologyManagerImpl.handleJoin(ClusterTopologyManagerImpl.java:225)
at org.infinispan.commands.topology.CacheJoinCommand.invokeAsync(CacheJoinCommand.java:42)
at org.infinispan.remoting.inboundhandler.GlobalInboundInvocationHandler$ReplicableCommandRunner.run(GlobalInboundInvocationHandler.java:252)
at org.infinispan.remoting.inboundhandler.GlobalInboundInvocationHandler.handleReplicableCommand(GlobalInboundInvocationHandler.java:174)
at org.infinispan.remoting.inboundhandler.GlobalInboundInvocationHandler.handleFromCluster(GlobalInboundInvocationHandler.java:113)
at org.infinispan.remoting.transport.jgroups.JGroupsTransport.processRequest(JGroupsTransport.java:1412)
at org.infinispan.remoting.transport.jgroups.JGroupsTransport.processMessage(JGroupsTransport.java:1335)
at org.infinispan.remoting.transport.jgroups.JGroupsTransport.access$400(JGroupsTransport.java:135)
at org.infinispan.remoting.transport.jgroups.JGroupsTransport$ChannelCallbacks.up(JGroupsTransport.java:1490)
at org.jgroups.JChannel.up(JChannel.java:784)
at org.jgroups.stack.ProtocolStack.up(ProtocolStack.java:913)
at org.jgroups.protocols.FRAG3.up(FRAG3.java:165)
Environment
- Red hat OpenShift Container Platform (OCP)
- 4.x
- Red Hat Data Grid (RHDG)
- 8.x
Subscriber exclusive content
A Red Hat subscription provides unlimited access to our knowledgebase, tools, and much more.