High Availability cluster shows non-existent estranged nodes in clustat output, or has "Received message has invalid digest... ignoring" messages in logs

Solution Verified - Updated -

Issue

  • A cluster node has many messages like the following in /var/log/messages:

    Jan 24 04:08:25 node1 openais[743]: [MAIN ] Received message has invalid digest... ignoring.  
    Jan 24 04:08:25 node1 openais[743]: [MAIN ] Invalid packet data
    - OR -
    Jan 24 04:08:25 node1 corosync[743]: [MAIN ] Received message has invalid digest... ignoring.  
    Jan 24 04:08:25 node1 corosync[743]: [MAIN ] Invalid packet data
    
  • clustat output shows estranged nodes in the node list that aren't a part of the cluster:

    Cluster Status for test-cluster @ Wed Oct  7 10:18:38 2009
    Member Status: Quorate
    
     Member Name                             ID   Status
     ------ ----                             ---- ------
     node1.test                                  2 Online, Local
     node2.test                                  1 Online
     node7.prod                                  7 Offline, Estranged
    ~~~
    
  • Cluster shows odd behavior such as pulling the wrong configuration file or config_version number when using cman_tool version -r or other synchronization mechanism.

Environment

  • Red Hat Enterprise Linux 5, 6, 7, or 8 (with the High Availability Add-on)

Subscriber exclusive content

A Red Hat subscription provides unlimited access to our knowledgebase, tools, and much more.

Current Customers and Partners

Log in for full access

Log In

New to Red Hat?

Learn more about Red Hat subscriptions

Using a Red Hat product through a public cloud?

How to access this content