cman fails to start with "Timed-out waiting for cluster" and openais repeatedly logs "The consensus timeout expired." immediately after starting cman in RHEL 5
Issue
- When
cman
starts either automatically on boot or by manually runningservice cman start
, it reports "Timed-out waiting for cluster"
# service cman start
Starting cluster:
Loading modules... done
Mounting configfs... done
Starting ccsd... done
Starting cman... failed
Timed-out waiting for cluster
[FAILED]
- In
/var/log/messages
I see repeated reports ofThe consensus timeout expired
andentering GATHER state from 3
:
Apr 12 17:06:45 openais[14610]: [MAIN ] Service initialized 'openais CMAN membership service 2.01'
Apr 12 17:06:45 openais[14610]: [SERV ] Service initialized 'openais extended virtual synchrony service'
Apr 12 17:06:45 openais[14610]: [SERV ] Service initialized 'openais cluster membership service B.01.01'
Apr 12 17:06:46 openais[14610]: [SERV ] Service initialized 'openais availability management framework B.01.01'
Apr 12 17:06:46 openais[14610]: [SERV ] Service initialized 'openais checkpoint service B.01.01'
Apr 12 17:06:46 openais[14610]: [SERV ] Service initialized 'openais event service B.01.01'
Apr 12 17:06:46 openais[14610]: [SERV ] Service initialized 'openais distributed locking service B.01.01'
Apr 12 17:06:46 openais[14610]: [SERV ] Service initialized 'openais message service B.01.01'
Apr 12 17:06:46 openais[14610]: [SERV ] Service initialized 'openais configuration service'
Apr 12 17:06:46 openais[14610]: [SERV ] Service initialized 'openais cluster closed process group service v1.01'
Apr 12 17:06:46 openais[14610]: [SERV ] Service initialized 'openais cluster config database access v1.01'
Apr 12 17:06:46 openais[14610]: [SYNC ] Not using a virtual synchrony filter.
Apr 12 17:06:46 openais[14610]: [MAIN ] Publishing socket for client connections.
Apr 12 17:06:50 ccsd[14604]: Cluster is not quorate. Refusing connection.
Apr 12 17:06:50 ccsd[14604]: Error while processing connect: Connection refused
Apr 12 17:06:55 ccsd[14604]: Cluster is not quorate. Refusing connection.
Apr 12 17:06:55 ccsd[14604]: Error while processing connect: Connection refused
Apr 12 17:06:57 openais[14610]: [TOTEM] The consensus timeout expired.
Apr 12 17:06:57 openais[14610]: [TOTEM] entering GATHER state from 3.
- When I restart the cman service then it is stopping at fencing
Environment
- Red Hat Enterprise Linux (RHEL) 5 with the High Availability Add On
- A
multicast
element in/etc/cluster/cluster.conf
that specifies an invalid multicast address (one outside the 224.0.0.1 - 239.255.255.255 range), such as:
<cman expected_votes="1" two_node="1".
<multicast addr="192.168.2.10"/>
</cman>
Subscriber exclusive content
A Red Hat subscription provides unlimited access to our knowledgebase, tools, and much more.