CMAN: too many transition restarts - will die

Solution Unverified - Updated -

Issue

  • Adding 5th node to running cluster caused full cluster down. Rebooted all nodes fixed the issue.
  • Rebooting or fencing a single node in the cluster caused the entire cluster to go down
  • Messages such as the following are seen in /var/log/messages

    Dec 15 16:17:47 hostname kernel: CMAN: too many transition restarts - will die
    Dec 15 16:17:47 hostname kernel: CMAN: we are leaving the cluster. Inconsistent cluster view
    Dec 15 16:17:47 hostname kernel: WARNING: dlm_emergency_shutdown
    Dec 15 16:17:47 hostname kernel: WARNING: dlm_emergency_shutdown
    Dec 15 16:17:47 hostname kernel: SM: 00000002 sm_stop: SG still joined
    Dec 15 16:17:47 hostname kernel: SM: 01000003 sm_stop: SG still joined
    Dec 15 16:17:47 hostname clurgmgrd[10000]: <warning> #67: Shutting down uncleanly 
    Dec 15 16:17:47 hostname kernel: SM: 02000009 sm_stop: SG still joined
    Dec 15 16:17:47 hostname kernel: SM: 0300001e sm_stop: SG still joined
    

Environment

  • Red Hat Enterprise Linux (RHEL) 4 Update 7 or earlier
  • Red Hat Cluster Suite

Subscriber exclusive content

A Red Hat subscription provides unlimited access to our knowledgebase of over 48,000 articles and solutions.

Current Customers and Partners

Log in for full access

Log In