pacemaker fails to start on a node and reports "Child process <process> (<pid>) exited: Network is down (100)" for several daemons in a RHEL 6 High Availability cluster
Issue
- Attempts to start
pacemaker
withpcs cluster start
orservice pacemaker start
fail, and the logs show several daemons exiting with error "Network is down". - A node is not joining the cluster with
pcs cluster start
.
Jan 12 23:10:33 node1 pacemakerd[13951]: notice: crm_add_logfile: Additional logging available in /var/log/pacemaker.log
Jan 12 23:10:33 node1 pacemakerd[13951]: notice: crm_add_logfile: Additional logging available in /var/log/cluster/corosync.log
Jan 12 23:10:33 node1 pacemakerd[13951]: notice: main: Starting Pacemaker 1.1.11 (Build: 97629de): generated-manpages agent-manpages ascii-docs ncurses libqb-logging libqb-ipc nagios corosync-plugin cman acls
Jan 12 23:10:33 node1 pacemakerd[13951]: error: pcmk_child_exit: Child process cib (13957) exited: Network is down (100)
Jan 12 23:10:33 node1 pacemakerd[13951]: warning: pcmk_child_exit: Pacemaker child process cib no longer wishes to be respawned. Shutting ourselves down.
Jan 12 23:10:33 node1 pacemakerd[13951]: notice: pcmk_shutdown_worker: Shuting down Pacemaker
Jan 12 23:10:33 node1 pacemakerd[13951]: notice: stop_child: Stopping crmd: Sent -15 to process 13962
Jan 12 23:10:33 node1 pacemakerd[13951]: notice: pcmk_child_exit: Child process crmd terminated with signal 15 (pid=13962, core=0)
Jan 12 23:10:33 node1 pacemakerd[13951]: notice: stop_child: Stopping pengine: Sent -15 to process 13961
Jan 12 23:10:33 node1 pacemakerd[13951]: error: pcmk_child_exit: Child process attrd (13960) exited: Network is down (100)
Jan 12 23:10:33 node1 pacemakerd[13951]: warning: pcmk_child_exit: Pacemaker child process attrd no longer wishes to be respawned. Shutting ourselves down.
Jan 12 23:10:33 node1 pacemakerd[13951]: error: pcmk_child_exit: Child process pengine (13961) exited: Network is down (100)
Jan 12 23:10:33 node1 pacemakerd[13951]: warning: pcmk_child_exit: Pacemaker child process pengine no longer wishes to be respawned. Shutting ourselves down.
Jan 12 23:10:33 node1 pacemakerd[13951]: notice: stop_child: Stopping lrmd: Sent -15 to process 13959
Jan 12 23:10:33 node1 pacemakerd[13951]: notice: pcmk_child_exit: Child process lrmd terminated with signal 15 (pid=13959, core=0)
Jan 12 23:10:33 node1 pacemakerd[13951]: notice: stop_child: Stopping stonith-ng: Sent -15 to process 13958
Jan 12 23:10:33 node1 pacemakerd[13951]: notice: pcmk_child_exit: Child process stonith-ng terminated with signal 15 (pid=13958, core=0)
Jan 12 23:10:33 node1 pacemakerd[13951]: notice: pcmk_shutdown_worker: Shutdown complete
Jan 12 23:10:33 node1 pacemakerd[13951]: notice: pcmk_shutdown_worker: Attempting to inhibit respawning after fatal error
Environment
- Red Hat Enterprise Linux (RHEL) 6 with the High Availability Add On
pacemaker
Subscriber exclusive content
A Red Hat subscription provides unlimited access to our knowledgebase, tools, and much more.