Failed to start clvmd service on one node while clustat showing both the nodes are online.

Solution Verified - Updated -

Issue

  • Failed to start clvmd service on one node while clustat showing both the nodes are online.

    • Found the following messages logs indicating clvmd in "D" state.

      dlm_controld[2410]: dlm_controld 3.0.12 started
      gfs_controld[2466]: gfs_controld 3.0.12 started
      fence_node[2475]: unfence node1 success
      kernel: dlm: Using TCP for communications
      kernel: dlm: connecting to 2
      kernel: INFO: task clvmd:2586 blocked for more than 120 seconds.
      kernel: "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
      kernel: clvmd         D ee85fe3c     0  2586      1 0x00000080
      kernel: ee3f5030 00000082 00000002 ee85fe3c d16c3b54 00000000 00000000 ee3f5030
      kernel: f70bc570 ee1f4480 000000c7 5a0cd0db 000000c7 c0ae2120 c0ae2120 ee3f52d8
      kernel: c0ae2120 c0addb54 c0ae2120 ee3f52d8 d16c3b54 00000002 00088099 ee3f5030
      kernel: Call Trace:
      kernel: [<c08231e5>] ? schedule_timeout+0x195/0x250
      kernel: [<c0474081>] ? finish_wait+0x31/0x80
      kernel: [<c0822f49>] ? wait_for_common+0xe9/0x150
      kernel: [<c044bf70>] ? default_wake_function+0x0/0x10
      kernel: [<fccf9057>] ? dlm_new_lockspace+0x807/0x880 [dlm]
      kernel: [<fcd0048d>] ? device_write+0x21d/0x610 [dlm]
      kernel: [<c059d6cc>] ? security_file_permission+0xc/0x10
      kernel: [<c0527fb6>] ? rw_verify_area+0x66/0xe0
      kernel: [<fcd00270>] ? device_write+0x0/0x610 [dlm]
      kernel: [<c05280d0>] ? vfs_write+0xa0/0x190
      kernel: [<c04adecc>] ? audit_syscall_entry+0x21c/0x240
      kernel: [<c0528b51>] ? sys_write+0x41/0x70
      kernel: [<c0409adf>] ? sysenter_do_call+0x12/0x28
      kernel: INFO: task clvmd:2586 blocked for more than 120 seconds.
      

Environment

  • Red Hat Enterprise Linux 6
  • Red Hat Cluster Suite

Subscriber exclusive content

A Red Hat subscription provides unlimited access to our knowledgebase of over 48,000 articles and solutions.

Current Customers and Partners

Log in for full access

Log In
Close

Welcome! Check out the Getting Started with Red Hat page for quick tours and guides for common tasks.