Why do I see "task libvirtd: blocked for more than 120 seconds" messages in my RHEV hosts?
Issue
- Why do I observe entries like the following at
/var/log/messagesin my RHEV hosts?
May 9 21:57:15 rhevh1 kernel: INFO: task libvirtd:9701 blocked for more than 120 seconds.
May 9 21:57:15 rhevh1 kernel: Not tainted 2.6.32-431.11.2.el6.x86_64 #1
May 9 21:57:15 rhevh1 kernel: "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
May 9 21:57:15 rhevh1 kernel: libvirtd D 0000000000000008 0 9701 1 0x00000080
May 9 21:57:15 rhevh1 kernel: ffff880be7e97d78 0000000000000086 ffffffff810cf1b0 ffff8817f67a0340
May 9 21:57:15 rhevh1 kernel: ffff880be7e97d18 ffffffff811ae33f ffff880be7e97d38 ffff881552f1db00
May 9 21:57:15 rhevh1 kernel: ffff880be7e9b098 ffff880be7e97fd8 000000000000fbc8 ffff880be7e9b098
May 9 21:57:15 rhevh1 kernel: Call Trace:
May 9 21:57:15 rhevh1 kernel: [<ffffffff810cf1b0>] ? proc_cgroup_show+0x0/0x210
May 9 21:57:15 rhevh1 kernel: [<ffffffff811ae33f>] ? single_open+0x6f/0xa0
May 9 21:57:15 rhevh1 kernel: [<ffffffff8118617f>] ? __dentry_open+0x23f/0x360
May 9 21:57:15 rhevh1 kernel: [<ffffffff8152969e>] __mutex_lock_slowpath+0x13e/0x180
May 9 21:57:15 rhevh1 kernel: [<ffffffff8152953b>] mutex_lock+0x2b/0x50
May 9 21:57:15 rhevh1 kernel: [<ffffffff810cf21c>] proc_cgroup_show+0x6c/0x210
May 9 21:57:15 rhevh1 kernel: [<ffffffff811ade82>] seq_read+0xf2/0x400
May 9 21:57:15 rhevh1 kernel: [<ffffffff81189775>] vfs_read+0xb5/0x1a0
May 9 21:57:15 rhevh1 kernel: [<ffffffff811898b1>] sys_read+0x51/0x90
May 9 21:57:15 rhevh1 kernel: [<ffffffff810e1e4e>] ? __audit_syscall_exit+0x25e/0x290
May 9 21:57:15 rhevh1 kernel: [<ffffffff8100b072>] system_call_fastpath+0x16/0x1b
or;
Aug 25 19:02:20 hostname kernel: INFO: task libvirtd:27645 blocked for more than 120 seconds.
Aug 25 19:02:20 hostname kernel: Not tainted 2.6.32-431.20.3.el6.x86_64 #1
Aug 25 19:02:20 hostname kernel: "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
Aug 25 19:02:20 hostname kernel: libvirtd D 0000000000000002 0 27645 1 0x00000080
Aug 25 19:02:20 hostname kernel: ffff8830240f7d78 0000000000000082 ffff8830240f7d68 ffffffff8114ecaf
Aug 25 19:02:20 hostname kernel: 0000000000000000 0000000000000000 ffffffff8100bb8e ffff8830240f7d78
Aug 25 19:02:20 hostname kernel: ffff883033637058 ffff8830240f7fd8 000000000000fbc8 ffff883033637058
Aug 25 19:02:20 hostname kernel: Call Trace:
Aug 25 19:02:20 hostname kernel: [<ffffffff8114ecaf>] ? vma_adjust+0x11f/0x5e0
Aug 25 19:02:20 hostname kernel: [<ffffffff8100bb8e>] ? apic_timer_interrupt+0xe/0x20
Aug 25 19:02:20 hostname kernel: [<ffffffff8105546b>] ? mutex_spin_on_owner+0x9b/0xc0
Aug 25 19:02:20 hostname kernel: [<ffffffff81529f9e>] __mutex_lock_slowpath+0x13e/0x180
Aug 25 19:02:20 hostname kernel: [<ffffffff81529e3b>] mutex_lock+0x2b/0x50
Aug 25 19:02:20 hostname kernel: [<ffffffff810cf04c>] proc_cgroup_show+0x6c/0x210
Aug 25 19:02:20 hostname kernel: [<ffffffff811ae602>] seq_read+0xf2/0x400
Aug 25 19:02:20 hostname kernel: [<ffffffff81189525>] vfs_read+0xb5/0x1a0
Aug 25 19:02:20 hostname kernel: [<ffffffff81189661>] sys_read+0x51/0x90
Aug 25 19:02:20 hostname kernel: [<ffffffff810e1c7e>] ? __audit_syscall_exit+0x25e/0x290
Aug 25 19:02:20 hostname kernel: [<ffffffff8100b072>] system_call_fastpath+0x16/0x1b
- "VM XXXX not responding" message in the RHEV-M GUI.
Environment
- Red Hat Enterprise Virtualization (RHEV) 3.3 / 3.4.2
- numad-0.5-9.20130814git.el6
- numad-0.5-10.20140620git.el6_5
Subscriber exclusive content
A Red Hat subscription provides unlimited access to our knowledgebase, tools, and much more.