pacemaker fails to start on a node and reports "Child process <process> (<pid>) exited: Network is down (100)" for several daemons in a RHEL 6 High Availability cluster

Solution Unverified - Updated -

Issue

  • Attempts to start pacemaker with pcs cluster start or service pacemaker start fail, and the logs show several daemons exiting with error "Network is down".
  • A node is not joining the cluster with pcs cluster start.
Jan 12 23:10:33 node1 pacemakerd[13951]:   notice: crm_add_logfile: Additional logging available in /var/log/pacemaker.log
Jan 12 23:10:33 node1 pacemakerd[13951]:   notice: crm_add_logfile: Additional logging available in /var/log/cluster/corosync.log
Jan 12 23:10:33 node1 pacemakerd[13951]:   notice: main: Starting Pacemaker 1.1.11 (Build: 97629de):  generated-manpages agent-manpages ascii-docs ncurses libqb-logging libqb-ipc nagios  corosync-plugin cman acls
Jan 12 23:10:33 node1 pacemakerd[13951]:    error: pcmk_child_exit: Child process cib (13957) exited: Network is down (100)
Jan 12 23:10:33 node1 pacemakerd[13951]:  warning: pcmk_child_exit: Pacemaker child process cib no longer wishes to be respawned. Shutting ourselves down.
Jan 12 23:10:33 node1 pacemakerd[13951]:   notice: pcmk_shutdown_worker: Shuting down Pacemaker
Jan 12 23:10:33 node1 pacemakerd[13951]:   notice: stop_child: Stopping crmd: Sent -15 to process 13962
Jan 12 23:10:33 node1 pacemakerd[13951]:   notice: pcmk_child_exit: Child process crmd terminated with signal 15 (pid=13962, core=0)
Jan 12 23:10:33 node1 pacemakerd[13951]:   notice: stop_child: Stopping pengine: Sent -15 to process 13961
Jan 12 23:10:33 node1 pacemakerd[13951]:    error: pcmk_child_exit: Child process attrd (13960) exited: Network is down (100)
Jan 12 23:10:33 node1 pacemakerd[13951]:  warning: pcmk_child_exit: Pacemaker child process attrd no longer wishes to be respawned. Shutting ourselves down.
Jan 12 23:10:33 node1 pacemakerd[13951]:    error: pcmk_child_exit: Child process pengine (13961) exited: Network is down (100)
Jan 12 23:10:33 node1 pacemakerd[13951]:  warning: pcmk_child_exit: Pacemaker child process pengine no longer wishes to be respawned. Shutting ourselves down.
Jan 12 23:10:33 node1 pacemakerd[13951]:   notice: stop_child: Stopping lrmd: Sent -15 to process 13959
Jan 12 23:10:33 node1 pacemakerd[13951]:   notice: pcmk_child_exit: Child process lrmd terminated with signal 15 (pid=13959, core=0)
Jan 12 23:10:33 node1 pacemakerd[13951]:   notice: stop_child: Stopping stonith-ng: Sent -15 to process 13958
Jan 12 23:10:33 node1 pacemakerd[13951]:   notice: pcmk_child_exit: Child process stonith-ng terminated with signal 15 (pid=13958, core=0)
Jan 12 23:10:33 node1 pacemakerd[13951]:   notice: pcmk_shutdown_worker: Shutdown complete
Jan 12 23:10:33 node1 pacemakerd[13951]:   notice: pcmk_shutdown_worker: Attempting to inhibit respawning after fatal error

Environment

  • Red Hat Enterprise Linux (RHEL) 6 with the High Availability Add On
  • pacemaker

Subscriber exclusive content

A Red Hat subscription provides unlimited access to our knowledgebase, tools, and much more.

Current Customers and Partners

Log in for full access

Log In

New to Red Hat?

Learn more about Red Hat subscriptions

Using a Red Hat product through a public cloud?

How to access this content