Initial infinispan transfer error when starting a new node after upgrade

Solution Unverified - Updated -

Issue

  • The first server starts OK but when booting another member of the cluster the first node throws the following exception:

    ERROR [org.infinispan.statetransfer.OutboundTransferTask] (transport-thread--p13-t15) Failed to send entries to node member02: ISPN000217: Received exception from member02, see cause for remote stack trace: org.infinispan.remoting.RemoteException: ISPN000217: Received exception from member02, see cause for remote stack trace
        at org.infinispan.remoting.transport.ResponseCollectors.wrapRemoteException(ResponseCollectors.java:27)
        at org.infinispan.remoting.transport.ValidSingleResponseCollector.withException(ValidSingleResponseCollector.java:37)
        at org.infinispan.remoting.transport.ValidSingleResponseCollector.addResponse(ValidSingleResponseCollector.java:21)
        at org.infinispan.remoting.transport.impl.SingleTargetRequest.receiveResponse(SingleTargetRequest.java:52)
            ...
            Suppressed: org.infinispan.util.logging.TraceException
            at org.infinispan.remoting.rpc.RpcManagerImpl.blocking(RpcManagerImpl.java:268)
            at org.infinispan.remoting.rpc.RpcManagerImpl.invokeRemotely(RpcManagerImpl.java:323)
                    ...
    Caused by: org.infinispan.commons.CacheException: Error instantiating class: class org.keycloak.models.sessions.infinispan.changes.SessionEntityWrapper$ExternalizerImpl
        at org.infinispan.marshall.core.GlobalMarshaller.readAnnotated(GlobalMarshaller.java:718)
        at org.infinispan.marshall.core.GlobalMarshaller.readNonNullableObject(GlobalMarshaller.java:695)
        at org.infinispan.marshall.core.GlobalMarshaller.readNullableObject(GlobalMarshaller.java:361)
        at org.infinispan.marshall.core.BytesObjectInput.readObject(BytesObjectInput.java:40)
            ...
    Caused by: org.infinispan.commons.CacheException: Error instantiating class: class org.keycloak.models.sessions.infinispan.entities.UserSessionEntity$ExternalizerImpl
        at org.infinispan.marshall.core.GlobalMarshaller.readAnnotated(GlobalMarshaller.java:718)
        at org.infinispan.marshall.core.GlobalMarshaller.readNonNullableObject(GlobalMarshaller.java:695)
        at org.infinispan.marshall.core.GlobalMarshaller.readNullableObject(GlobalMarshaller.java:361)
        at org.infinispan.marshall.core.BytesObjectInput.readObject(BytesObjectInput.java:40)
        at org.keycloak.models.sessions.infinispan.changes.SessionEntityWrapper$ExternalizerImpl.readObject(SessionEntityWrapper.java:195)
        at org.keycloak.models.sessions.infinispan.changes.SessionEntityWrapper$ExternalizerImpl.readObject(SessionEntityWrapper.java:155)
        at org.infinispan.marshall.core.GlobalMarshaller.readAnnotated(GlobalMarshaller.java:716)
        ... 64 more
    Caused by: java.lang.NullPointerException
        at java.util.concurrent.ConcurrentHashMap.putVal(ConcurrentHashMap.java:1011)
        at java.util.concurrent.ConcurrentHashMap.put(ConcurrentHashMap.java:1006)
        at org.keycloak.models.sessions.infinispan.util.KeycloakMarshallUtil.readMap(KeycloakMarshallUtil.java:96)
        at org.keycloak.models.sessions.infinispan.entities.UserSessionEntity$ExternalizerImpl.readObjectVersion1(UserSessionEntity.java:300)
        at org.keycloak.models.sessions.infinispan.entities.UserSessionEntity$ExternalizerImpl.readObject(UserSessionEntity.java:275)
        at org.keycloak.models.sessions.infinispan.entities.UserSessionEntity$ExternalizerImpl.readObject(UserSessionEntity.java:228)
        at org.infinispan.marshall.core.GlobalMarshaller.readAnnotated(GlobalMarshaller.java:716)
        ... 70 more
    

    In the new node that is starting and joining to the cluster a timeout happens:

    ERROR [org.jboss.msc.service.fail] (ServerService Thread Pool -- 50) MSC000001: Failed to start service org.wildfly.clustering.infinispan.cache.keycloak.offlineSessions: org.jboss.msc.service.StartException in service org.wildfly.clustering.infinispan.cache.keycloak.offlineSessions: org.infinispan.commons.CacheException: Unable to invoke method public void org.infinispan.statetransfer.StateTransferManagerImpl.waitForInitialStateTransferToComplete() throws java.lang.Exception on object of type StateTransferManagerImpl
        at org.wildfly.clustering.service.FunctionalService.start(FunctionalService.java:70)
        at org.wildfly.clustering.service.AsyncServiceConfigurator$AsyncService.lambda$start$0(AsyncServiceConfigurator.java:117)
        at org.jboss.threads.ContextClassLoaderSavingRunnable.run(ContextClassLoaderSavingRunnable.java:35)
        at org.jboss.threads.EnhancedQueueExecutor.safeRun(EnhancedQueueExecutor.java:1985)
        at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.doRunTask(EnhancedQueueExecutor.java:1487)
        at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.run(EnhancedQueueExecutor.java:1378)
        at java.lang.Thread.run(Thread.java:748)
        at org.jboss.threads.JBossThread.run(JBossThread.java:485)
    Caused by: org.infinispan.commons.CacheException: Unable to invoke method public void org.infinispan.statetransfer.StateTransferManagerImpl.waitForInitialStateTransferToComplete() throws java.lang.Exception on object of type StateTransferManagerImpl
        at org.infinispan.commons.util.SecurityActions.lambda$invokeAccessibly$0(SecurityActions.java:83)
        at org.infinispan.commons.util.SecurityActions.doPrivileged(SecurityActions.java:71)
        at org.infinispan.commons.util.SecurityActions.invokeAccessibly(SecurityActions.java:76)
        at org.infinispan.commons.util.ReflectionUtil.invokeAccessibly(ReflectionUtil.java:185)
        at org.infinispan.factories.AbstractComponentRegistry$PrioritizedMethod.invoke(AbstractComponentRegistry.java:968)
        at org.infinispan.factories.AbstractComponentRegistry.lambda$invokePrioritizedMethods$6(AbstractComponentRegistry.java:703)
        at org.infinispan.factories.SecurityActions.lambda$run$1(SecurityActions.java:72)
        at org.infinispan.security.Security.doPrivileged(Security.java:44)
        at org.infinispan.factories.SecurityActions.run(SecurityActions.java:71)
        at org.infinispan.factories.AbstractComponentRegistry.invokePrioritizedMethods(AbstractComponentRegistry.java:696)
        at org.infinispan.factories.AbstractComponentRegistry.internalStart(AbstractComponentRegistry.java:689)
        at org.infinispan.factories.AbstractComponentRegistry.start(AbstractComponentRegistry.java:607)
        at org.infinispan.factories.ComponentRegistry.start(ComponentRegistry.java:244)
        ...
    Caused by: org.infinispan.commons.CacheException: Initial state transfer timed out for cache offlineSessions on member02
        at org.infinispan.statetransfer.StateTransferManagerImpl.waitForInitialStateTransferToComplete(StateTransferManagerImpl.java:233)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at org.infinispan.commons.util.SecurityActions.lambda$invokeAccessibly$0(SecurityActions.java:79)
        ... 30 more
    

Environment

  • Red Hat Single Sign-On (RH-SSO)
    • 7
  • Upgrading from RH-SSO 7.0 or 7.1 to a newer version (7.2+)

Subscriber exclusive content

A Red Hat subscription provides unlimited access to our knowledgebase of over 48,000 articles and solutions.

Current Customers and Partners

Log in for full access

Log In