cman service start failed with "Starting cman... Timed-out waiting for cluster" on a RHEL 6 High Availability cluster node in a cluster with more than two nodes

Solution Verified - Updated -

Issue

  • We reboot all three nodes regularly, and sometimes one of the nodes comes up without rgmanager started after seeing "Waiting for quorum"... Timed-out waiting for cluster"

    Starting cluster: 
   Checking if cluster has been disabled at boot... [  OK  ]
   Checking Network Manager... [  OK  ]
   Global setup... [  OK  ]
   Loading kernel modules... [  OK  ]
   Mounting configfs... [  OK  ]
   Starting cman... [  OK  ]
   Waiting for quorum... Timed-out waiting for cluster
[FAILED]

  • After a reboot, we found the 3rd node had failed to start cman and join the cluster

    # service cman status
fenced is stopped
# service rgmanager status
rgmanager dead but pid file exists

Environment

  • Red Hat Enterprise Linux (RHEL) 6 with the High Availability Add On
  • Cluster of more than two nodes
  • cman is chkconfig'd on at boot

Subscriber exclusive content

A Red Hat subscription provides unlimited access to our knowledgebase, tools, and much more.

Current Customers and Partners

Log in for full access

Log In

New to Red Hat?

Learn more about Red Hat subscriptions

Using a Red Hat product through a public cloud?

How to access this content