GFS2 unmount blocks in dlm_release_lockspace when issuing a reboot of RHEL 6 cluster node
Issue
- Occasionally, when one node is shutdown,
rgmanagerdoes not exit and we see hung tasks in/var/log/messagesfor an unmount command waiting indlm_release_lockspace gfs2filesystems do not unmount.- A hard power shutdown of one node will leave the other node unable to access gfs2 filesystems.
kernel: INFO: task umount:59023 blocked for more than 120 seconds.
kernel: "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
kernel: umount D 0000000000000006 0 59023 57463 0x00000000
kernel: ffff882026f6dce8 0000000000000082 0000000000000000 0000000000000000
kernel: 0000000000000000 0000000000000000 0000000000000000 0000000000000000
kernel: ffff881291ead058 ffff882026f6dfd8 000000000000fb88 ffff881291ead058
kernel: Call Trace:
kernel: [<ffffffff8150ed3e>] __mutex_lock_slowpath+0x13e/0x180
kernel: [<ffffffff8150ebdb>] mutex_lock+0x2b/0x50
kernel: [<ffffffffa047bea8>] dlm_release_lockspace+0x48/0x480 [dlm]
kernel: [<ffffffffa062b757>] gdlm_unmount+0x27/0x40 [gfs2]
kernel: [<ffffffffa0619821>] gfs2_lm_unmount+0x21/0x30 [gfs2]
kernel: [<ffffffffa0628284>] gfs2_put_super+0x184/0x220 [gfs2]
kernel: [<ffffffff8118326b>] generic_shutdown_super+0x5b/0xe0
kernel: [<ffffffff81183321>] kill_block_super+0x31/0x50
kernel: [<ffffffffa06198a3>] gfs2_kill_sb+0x73/0x80 [gfs2]
kernel: [<ffffffff81183af7>] deactivate_super+0x57/0x80
kernel: [<ffffffff811a1b6f>] mntput_no_expire+0xbf/0x110
kernel: [<ffffffff811a25db>] sys_umount+0x7b/0x3a0
kernel: [<ffffffff81086391>] ? sigprocmask+0x71/0x110
kernel: [<ffffffff8100b072>] system_call_fastpath+0x16/0x1b
Environment
- Red Hat Enterprise Linux (RHEL) 6 Update 4 with the Reslient Storage Add On
kernel-2.6.32-358.el6
- GFS2
Subscriber exclusive content
A Red Hat subscription provides unlimited access to our knowledgebase of over 48,000 articles and solutions.
Welcome! Check out the Getting Started with Red Hat page for quick tours and guides for common tasks.
