CMAN: too many transition restarts - will die

Solution Unverified - Updated -

Issue

  • Adding 5th node to running cluster caused full cluster down. Rebooted all nodes fixed the issue.
  • Rebooting or fencing a single node in the cluster caused the entire cluster to go down

  • Messages such as the following are seen in /var/log/messages

    Dec 15 16:17:47 hostname kernel: CMAN: too many transition restarts - will die
Dec 15 16:17:47 hostname kernel: CMAN: we are leaving the cluster. Inconsistent cluster view
Dec 15 16:17:47 hostname kernel: WARNING: dlm_emergency_shutdown
Dec 15 16:17:47 hostname kernel: WARNING: dlm_emergency_shutdown
Dec 15 16:17:47 hostname kernel: SM: 00000002 sm_stop: SG still joined
Dec 15 16:17:47 hostname kernel: SM: 01000003 sm_stop: SG still joined
Dec 15 16:17:47 hostname clurgmgrd[10000]: <warning> #67: Shutting down uncleanly 
Dec 15 16:17:47 hostname kernel: SM: 02000009 sm_stop: SG still joined
Dec 15 16:17:47 hostname kernel: SM: 0300001e sm_stop: SG still joined

Environment

  • Red Hat Enterprise Linux (RHEL) 4 Update 7 or earlier
  • Red Hat Cluster Suite

Subscriber exclusive content

A Red Hat subscription provides unlimited access to our knowledgebase of over 48,000 articles and solutions.

Current Customers and Partners

Log in for full access

Log In

New to Red Hat?

Learn more about Red Hat subscriptions

Close

Welcome! Check out the Getting Started with Red Hat page for quick tours and guides for common tasks.