Replication timeout shows on target cluster during synchronize-data for rolling upgrade

Solution Unverified - Updated -

Issue

  • When invoking synchronize-data to replicate cache entries from source cluster to target cluster by rolling upgrade, Replication timeout shows on target cluster.
  • Then synchronized-data copied a partial data only after finishing the synchronization command.
 WARN  [org.infinispan.remoting.inboundhandler.NonTotalOrderPerCacheInboundInvocationHandler] (remote-thread--p2-t111) ISPN000071: Caught exception when handling command DistributedExecuteCommand [cache=default (flags=0), site-id=site, rack-id=rack, machine-id=machine), keys=[], callable=org.infinispan.persistence.remote.upgrade.MigrationTask@fxefce1e]: org.infinispan.client.hotrod.exceptions.HotRodClientException:Request for messageId=123 returned server error (status=0x86): org.infinispan.util.concurrent.TimeoutException: Replication timeout for src_node2 (flags=0), site-id=site, rack-id=rack, machine-id=machine)
    at org.infinispan.client.hotrod.impl.protocol.Codec20.checkForErrorsInResponseStatus(Codec20.java:421)
    at org.infinispan.client.hotrod.impl.protocol.Codec20.readPartialHeader(Codec20.java:168)
    at org.infinispan.client.hotrod.impl.protocol.Codec20.readHeader(Codec20.java:154)
    at org.infinispan.client.hotrod.impl.operations.HotRodOperation.readHeaderAndValidate(HotRodOperation.java:60)
    at org.infinispan.client.hotrod.impl.operations.IterationNextOperation.execute(IterationNextOperation.java:52)
    at org.infinispan.client.hotrod.impl.iteration.RemoteCloseableIterator.fetch(RemoteCloseableIterator.java:104)
    at org.infinispan.client.hotrod.impl.iteration.RemoteCloseableIterator.hasNext(RemoteCloseableIterator.java:88)
    at org.infinispan.persistence.remote.upgrade.MigrationTask.migrateEntriesWithMetadata(MigrationTask.java:108)
    at org.infinispan.persistence.remote.upgrade.MigrationTask.call(MigrationTask.java:93)
    at org.infinispan.persistence.remote.upgrade.MigrationTask.call(MigrationTask.java:49)
    at org.infinispan.commands.read.DistributedExecuteCommand.invokeAsync(DistributedExecuteCommand.java:110)
    at org.infinispan.remoting.inboundhandler.BasePerCacheInboundInvocationHandler.invokeCommand(BasePerCacheInboundInvocationHandler.java:95)
    at org.infinispan.remoting.inboundhandler.BaseBlockingRunnable.invoke(BaseBlockingRunnable.java:91)
    at org.infinispan.remoting.inboundhandler.BaseBlockingRunnable.runAsync(BaseBlockingRunnable.java:69)
    at org.infinispan.remoting.inboundhandler.BaseBlockingRunnable.run(BaseBlockingRunnable.java:41)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
    at java.lang.Thread.run(Thread.java:748)

Environment

  • Red Hat Data Grid (RHDG)
    • 7.x

Subscriber exclusive content

A Red Hat subscription provides unlimited access to our knowledgebase, tools, and much more.

Current Customers and Partners

Log in for full access

Log In

New to Red Hat?

Learn more about Red Hat subscriptions

Using a Red Hat product through a public cloud?

How to access this content