rgmanager reports "Quorum Dissolved" and stops all resources when using master_wins mode with qdisk on RHEL 5 and RHEL6 and one node leaves the cluster uncleanly

Solution Unverified - Updated -

Issue

  • Quorum Dissolved when using master_wins mode with qdisk on RHEL 5 and RHEL6
Apr  1 15:25:55 rnode2 corosync[14188]:   [TOTEM ] A processor failed, forming new configuration.
Apr  1 15:25:57 rnode2 kernel: dlm: closing connection to node 1
Apr  1 15:25:57 node2 corosync[14188]:   [CMAN  ] quorum lost, blocking activity
Apr  1 15:25:57 node2 corosync[14188]:   [QUORUM] This node is within the non-primary component and will NOT provide any services.
Apr  1 15:25:57 node2 corosync[14188]:   [QUORUM] Members[1]: 2
Apr  1 15:25:57 node2 corosync[14188]:   [TOTEM ] A processor joined or left the membership and a new membership was formed.
Apr  1 15:25:57 node2 corosync[14188]:   [CPG   ] chosen downlist: sender r(0) ip(192.168.122.162) ; members(old:2 left:1)
Apr  1 15:25:57 node2 corosync[14188]:   [MAIN  ] Completed service synchronization, ready to provide service.
Apr  1 15:25:57 node2 rgmanager[14577]: #1: Quorum Dissolved
Apr  1 15:25:57 node2 rgmanager[27697]: [ip] Removing IPv4 address 192.168.122.222/24 from eth1
Apr  1 15:25:57 node2 avahi-daemon[2018]: Withdrawing address record for 192.168.122.222 on eth1.
Apr  1 15:25:57 node2 rgmanager[27733]: [script] Executing /etc/cluster/scripts/test_script-1.sh stop
Apr  1 15:25:57 node2 rgmanager[27755]: Executing stop on the script: /etc/cluster/scripts/test_script-1.sh
Apr  1 15:25:57 node2 rgmanager[27773]: The script is not running: /etc/cluster/scripts/test_script-1.sh.
Apr  1 15:26:03 node2 corosync[14188]:   [CMAN  ] quorum device re-registered
Apr  1 15:26:03 node2 corosync[14188]:   [CMAN  ] quorum regained, resuming activity
Apr  1 15:26:03 node2 corosync[14188]:   [QUORUM] This node is within the primary component and will provide service.
Apr  1 15:26:03 node2 corosync[14188]:   [QUORUM] Members[1]: 2
Apr  1 15:26:03 node2 qdiskd[14239]: Assuming master role
Apr  1 15:26:03 node2 fenced[14409]: fencing node rh6node1.examplerh.com
Apr  1 15:26:03 node2 qdiskd[14239]: Writing eviction notice for node 1
Apr  1 15:26:04 node2 qdiskd[14239]: Node 1 evicted
Apr  1 15:26:05 node2 fenced[14409]: fence rh6node1.examplerh.com success
Apr  1 15:26:07 node2 rgmanager[27915]: [ip] Removing IPv4 address 192.168.122.221/24 from eth1
Apr  1 15:26:17 node2 rgmanager[27975]: [ip] Removing IPv4 address 192.168.122.220/24 from eth1
Apr  1 15:26:27 node2 rgmanager[14577]: Quorum Regained
  • All my clustered services and resources were stopped, when one of my cluster nodes failed that is using master_wins mode for qdisk.

Environment

  • Red Hat Enterprise Linux Server 5 (with the High Availability Add Ons)
  • Red Hat Enterprise Linux Server 6 (with the High Availability Add Ons)
  • A 2 node cluster using master_wins mode for qdisk.

Subscriber exclusive content

A Red Hat subscription provides unlimited access to our knowledgebase, tools, and much more.

Current Customers and Partners

Log in for full access

Log In

New to Red Hat?

Learn more about Red Hat subscriptions

Using a Red Hat product through a public cloud?

How to access this content