RHEL 7 High Availability cluster becomes stuck in a loop of crmd errors "crm_xml_add: Triggered assert at xml.c:2703 : node != NULL" and "State transition S_NOT_DC -> S_PENDING"
Issue
As a node was joining the cluster, one node seems to have gotten stuck in an error loop from crmd with Triggered assert at xml.c:2703
Aug 26 22:53:20 node2 crmd[4511]: warning: Input I_FAIL received in state S_FINALIZE_JOIN from do_cl_join_finalize_respond
Aug 26 22:53:20 node2 crmd[4511]: error: crm_xml_add: Triggered assert at xml.c:2703 : node != NULL
Aug 26 22:53:20 node2 crmd[4511]: error: crm_xml_add: Triggered assert at xml.c:2703 : node != NULL
Aug 26 22:53:20 node2 crmd[4511]: error: Could not send our LRM state to the DC
Aug 26 22:53:20 node2 crmd[4511]: warning: Input I_FAIL received in state S_FINALIZE_JOIN from do_cl_join_finalize_respond
Aug 26 22:53:20 node2 crmd[4511]: error: crm_xml_add: Triggered assert at xml.c:2703 : node != NULL
And the other node is stuck in a state where it doesn't have a DC and the DC keeps rejecting it:
Aug 26 22:53:20 node1 crmd[10927]: notice: State transition S_NOT_DC -> S_PENDING
Aug 26 22:53:20 node1 crmd[10927]: notice: State transition S_PENDING -> S_NOT_DC
Aug 26 22:53:20 node1 crmd[10927]: notice: State transition S_NOT_DC -> S_PENDING
Aug 26 22:53:20 node1 crmd[10927]: notice: State transition S_PENDING -> S_NOT_DC
Aug 26 22:53:20 node1 crmd[10927]: notice: State transition S_NOT_DC -> S_PENDING
Aug 26 22:53:20 node1 crmd[10927]: notice: State transition S_PENDING -> S_NOT_DC
Aug 26 22:53:20 node1 crmd[10927]: notice: State transition S_NOT_DC -> S_PENDING
Aug 26 22:53:20 node1 crmd[10927]: notice: State transition S_PENDING -> S_NOT_DC
Environment
- Red Hat Enterprise Linux (RHEL) 7 with the High Availability Add-On
Subscriber exclusive content
A Red Hat subscription provides unlimited access to our knowledgebase, tools, and much more.