rgmanager reports "Quorum Dissolved" and stops all resources when using master_wins mode with qdisk on RHEL 5 and RHEL6 and one node leaves the cluster uncleanly
Issue
- Quorum Dissolved when using
master_wins
mode with qdisk on RHEL 5 and RHEL6
Apr 1 15:25:55 rnode2 corosync[14188]: [TOTEM ] A processor failed, forming new configuration.
Apr 1 15:25:57 rnode2 kernel: dlm: closing connection to node 1
Apr 1 15:25:57 node2 corosync[14188]: [CMAN ] quorum lost, blocking activity
Apr 1 15:25:57 node2 corosync[14188]: [QUORUM] This node is within the non-primary component and will NOT provide any services.
Apr 1 15:25:57 node2 corosync[14188]: [QUORUM] Members[1]: 2
Apr 1 15:25:57 node2 corosync[14188]: [TOTEM ] A processor joined or left the membership and a new membership was formed.
Apr 1 15:25:57 node2 corosync[14188]: [CPG ] chosen downlist: sender r(0) ip(192.168.122.162) ; members(old:2 left:1)
Apr 1 15:25:57 node2 corosync[14188]: [MAIN ] Completed service synchronization, ready to provide service.
Apr 1 15:25:57 node2 rgmanager[14577]: #1: Quorum Dissolved
Apr 1 15:25:57 node2 rgmanager[27697]: [ip] Removing IPv4 address 192.168.122.222/24 from eth1
Apr 1 15:25:57 node2 avahi-daemon[2018]: Withdrawing address record for 192.168.122.222 on eth1.
Apr 1 15:25:57 node2 rgmanager[27733]: [script] Executing /etc/cluster/scripts/test_script-1.sh stop
Apr 1 15:25:57 node2 rgmanager[27755]: Executing stop on the script: /etc/cluster/scripts/test_script-1.sh
Apr 1 15:25:57 node2 rgmanager[27773]: The script is not running: /etc/cluster/scripts/test_script-1.sh.
Apr 1 15:26:03 node2 corosync[14188]: [CMAN ] quorum device re-registered
Apr 1 15:26:03 node2 corosync[14188]: [CMAN ] quorum regained, resuming activity
Apr 1 15:26:03 node2 corosync[14188]: [QUORUM] This node is within the primary component and will provide service.
Apr 1 15:26:03 node2 corosync[14188]: [QUORUM] Members[1]: 2
Apr 1 15:26:03 node2 qdiskd[14239]: Assuming master role
Apr 1 15:26:03 node2 fenced[14409]: fencing node rh6node1.examplerh.com
Apr 1 15:26:03 node2 qdiskd[14239]: Writing eviction notice for node 1
Apr 1 15:26:04 node2 qdiskd[14239]: Node 1 evicted
Apr 1 15:26:05 node2 fenced[14409]: fence rh6node1.examplerh.com success
Apr 1 15:26:07 node2 rgmanager[27915]: [ip] Removing IPv4 address 192.168.122.221/24 from eth1
Apr 1 15:26:17 node2 rgmanager[27975]: [ip] Removing IPv4 address 192.168.122.220/24 from eth1
Apr 1 15:26:27 node2 rgmanager[14577]: Quorum Regained
- All my clustered services and resources were stopped, when one of my cluster nodes failed that is using
master_wins
mode forqdisk
.
Environment
- Red Hat Enterprise Linux Server 5 (with the High Availability Add Ons)
- Red Hat Enterprise Linux Server 6 (with the High Availability Add Ons)
- A 2 node cluster using
master_wins
mode forqdisk
.
Subscriber exclusive content
A Red Hat subscription provides unlimited access to our knowledgebase, tools, and much more.