A cluster node leaves the cluster after pacemaker is stop "cman killed by node 2 because we were killed by cman_tool or other application" on RHEL 6
Issue
- A cluster node leaves the cluster after
pacemakeris stop and the message is loggedcman killed by node 2 because we were killed by cman_tool or other applicationon RHEL 6.
Mar 31 15:26:29 rh6node1 pacemaker: Waiting for shutdown of managed resources
Mar 31 15:26:29 rh6node1 pacemaker: DLM lockspace still in use
Mar 31 15:26:30 rh6node1 pacemaker: Attempting to shutdown clvmd
Mar 31 15:26:35 rh6node1 rsyslogd-2177: imuxsock lost 226 messages from pid 8612 due to rate-limiting
Mar 31 15:26:35 rh6node1 pacemaker: Leaving fence domain
Mar 31 15:26:35 rh6node1 pacemaker: Stopping fenced 3606
Mar 31 15:26:35 rh6node1 corosync[3539]: cman killed by node 2 because we were killed by cman_tool or other application
Mar 31 15:26:35 rh6node1 rgmanager[3883]: #67: Shutting down uncleanly
Mar 31 15:26:35 rh6node1 dlm_controld[3633]: cluster is down, exiting
Mar 31 15:26:35 rh6node1 dlm_controld[3633]: daemon cpg_dispatch error 2
Mar 31 15:26:35 rh6node1 dlm_controld[3633]: cpg_dispatch error 2
Mar 31 15:26:35 rh6node1 gfs_controld[3681]: cluster is down, exiting
Mar 31 15:26:35 rh6node1 gfs_controld[3681]: daemon cpg_dispatch error 2
Mar 31 15:26:36 rh6node1 rgmanager[8679]: [ip] Removing IPv4 address 192.168.222.33/24 from eth2
Mar 31 15:26:39 rh6node1 kernel: dlm: closing connection to node 2
Mar 31 15:26:39 rh6node1 kernel: dlm: closing connection to node 1
Mar 31 15:26:39 rh6node1 kernel: dlm: rgmanager: no userland control daemon, stopping lockspace
Mar 31 15:28:48 rh6node1 kernel: INFO: task rgmanager:3883 blocked for more than 120 seconds.
Environment
- Red Hat Enterprise Linux Server 6 (with the High Availability Add Ons)
rgmanageris managing the clustered resources andpacemakeris installed.
Subscriber exclusive content
A Red Hat subscription provides unlimited access to our knowledgebase, tools, and much more.
