Pacemaker/CMAN HA for OpenStack: What Happens to the cluster if a Node is Destroyed?

Issue

• When developing plans for using Pacemaker and CMAN to provide HA for the database and message queues in a RH OpenStack deployment.

  • One of the senarios, the cluster needs to handle the physical destruction of some of the nodes. How does the surviving sub-cluster would handle STONITH in such conditions, where the nodes in one location are suddenly and physically destroyed? When the remaining nodes lose contact with the destroyed ones they’ll attempt to STONITH them and wait for this to succeed. However it’s obviously impossible to power off a node that physically doesn’t exist anymore.

• Assuming that the data is available and OK, would Pacemaker be able to restart the resources on the surviving nodes even if it never gets a reply to a STONITH attempt on the destroyed nodes please?