cman fails when "Waiting for quorum... Timed-out waiting for cluster" and qdiskd logs "Shutdown request received during init" in RHEL 6

Issue

• On a three node cluster, when only one node is booted, the quorum device is registered and then unregisters quickly, during the initiation sequence. The result is that cluster services do not succeed in starting up on the single node.

• Cluster with a quorum device fails to start:

Starting cluster: 
   Checking if cluster has been disabled at boot...        [  OK  ]
   Checking Network Manager...                             [  OK  ]
   Global setup...                                         [  OK  ]
   Loading kernel modules...                               [  OK  ]
   Mounting configfs...                                    [  OK  ]
   Starting cman...                                        [  OK  ]
   Starting qdiskd...                                      [  OK  ]
   Waiting for quorum... Timed-out waiting for cluster
                                                           [FAILED]

• qdiskd log is getting:

Sep 06 14:23:30 qdiskd Node 3 is UP
Sep 06 14:23:35 qdiskd Node 3 is the master
Sep 06 14:23:50 qdiskd Shutdown request received during initialization

• In /var/log/messages, quorum can be seen being registered and then unregistered, followed by corosync service engines being unloaded:

  corosync[28029]:   [CMAN  ] quorum device registered
  qdiskd[28079]: Initial score 5/5
  qdiskd[28079]: Initialization complete
  qdiskd[28079]: Score sufficient for master operation (5/5; required=3); upgrading
  qdiskd[28079]: Unregistering quorum device.
  corosync[28029]:   [CMAN  ] quorum device unregistered
  corosync[28029]:   [SERV  ] Unloading all Corosync service engines.
  

• node 3 is up but after taking nodes 1 and 2 down, we are unable to get either node 1 nor node 2 back into the cluster.