Unable to fetch state on startup Exception in JBoss EAP 5

Solution Verified - Updated -

Environment

Red Hat JBoss Enterprise Application Platform (EAP)

  • 5.x

Issue

  • Some times after restart an EAP 5 instance that is part of a cluster partition the node throws some cache manager errors.
  • Data transferred from the other node is corrupted (or one of the Serializable objects is not reading the same data it wrote, throwing the data stream off).

  • There was an EOFException, which indicates a corrupted data stream.

    • The stack trace below details the EOFException:

          2011-01-26 11:56:29,658 ERROR org.jboss.web.tomcat.service.session.JBossCacheManager.internal Unable to start manager.
    org.jboss.cache.CacheException: java.lang.reflect.InvocationTargetException
    ...
    Caused by: org.jboss.cache.CacheException: Unable to fetch state on startup
    ...
    Caused by: java.io.EOFException
        at java.io.DataInputStream.readInt(DataInputStream.java:375)
        at java.io.ObjectInputStream$BlockDataInputStream.readInt(ObjectInputStream.java:2775)
        at java.io.ObjectInputStream.readInt(ObjectInputStream.java:949)

Resolution

  • Fix the class that has the serialization/deserialization problem. If you cannot identify what is the bad object, you'll have to restart the entire cluster to clear the bad session data. Note that the serialization issue can keep nodes from joining the cluster (existing cluster state can't be transferred).

  • As a workaround (until the serialization issue is resolved), one could disable the clustering functionality. Disabling the clustering functionality will keep sessions from being replicated.

Root Cause

  • There is an object within the session that does not implement the Serializable interface. As described here, all classes put inside the session must implement the Serializable interface.

Diagnostic Steps

Try to identify if there is some object that implements its own serialization with a bug in the serialization or deserialization.

This solution is part of Red Hat’s fast-track publication program, providing a huge library of solutions that Red Hat engineers have created while supporting our customers. To give you the knowledge you need the instant it becomes available, these articles may be presented in a raw and unedited form.

Comments