Temporary loss of quorum in RHEL 6 cluster when using master_wins and gracefully shutting down the master node
Issue
- In a two-node cluster setup with a quorum device, the quorum temporarily dissolves when stopping the qdisk master node.
Apr 18 17:23:50 hostname1 qdiskd[13247]: Node 2 shutdown
Apr 18 17:23:50 hostname1 kernel: dlm: closing connection to node 2
Apr 18 17:23:50 hostname1 corosync[13201]: [CMAN ] quorum lost, blocking activity
Apr 18 17:23:50 hostname1 corosync[13201]: [QUORUM] This node is within the non-primary component and will NOT provide any services.
Apr 18 17:23:50 hostname1 corosync[13201]: [QUORUM] Members[1]: 1
Apr 18 17:23:50 hostname1 corosync[13201]: [TOTEM ] A processor joined or left the membership and a new membership was formed.
Apr 18 17:23:50 hostname1 corosync[13201]: [CPG ] chosen downlist: sender r(0) ip(X.X.X.X) ; members(old:2 left:1)
Apr 18 17:23:50 hostname1 corosync[13201]: [MAIN ] Completed service synchronization, ready to provide service.
Apr 18 17:23:51 hostname1 corosync[13201]: [CMAN ] quorum device re-registered
Apr 18 17:23:51 hostname1 corosync[13201]: [CMAN ] quorum regained, resuming activity
Apr 18 17:23:51 hostname1 corosync[13201]: [QUORUM] This node is within the primary component and will provide service.
Apr 18 17:23:51 hostname1 corosync[13201]: [QUORUM] Members[1]: 1
Apr 18 17:23:51 hostname1 qdiskd[13247]: Assuming master role
- When the
qdisk
master leaves in amaster_wins
configuration, quorum is lost on the remaining node and services are disabled rather than relocating to it. - Cannot fail over service when shutting down a node
Environment
- Red Hat Enterprise Linux (RHEL) 6 with the High Availability Add On
cluster
(cman
,clusterlib
) release prior to3.0.12.1-49.el6
- Use of a quorum disk (qdisk)
- One of the following:
master_wins="1"
set in thequorumd
tag in/etc/cluster/cluster.conf
- A 2-node cluster with no heuristics defined
Subscriber exclusive content
A Red Hat subscription provides unlimited access to our knowledgebase, tools, and much more.