qemu-kvm process in uninterruptible sleep in RHEV 3.3 or RHEL 6.5
Issue
- A
qemu-kvm
process hosting a VM enters uninterruptible sleep ('D' state) for unclear reasons, and cannot be killed. - After forcing an unresponsive VM to poweroff, the rhevh server still has a
qemu-kvm
process for that VM in uniterruptible sleep. This makes it impossible to restart the VM without first rebooting the host. This is clearly not a good thing. - Kernel log shows errors similar to the following:
May 5 03:58:20 hostname kernel: INFO: task qemu-kvm:11375 blocked for more than 120 seconds.
May 5 03:58:20 hostname kernel: Not tainted 2.6.32-431.3.1.el6.x86_64 #1
May 5 03:58:20 hostname kernel: "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
May 5 03:58:20 hostname kernel: qemu-kvm D 0000000000000011 0 11375 1 0x00000084
May 5 03:58:20 hostname kernel: ffff88023ec7dcf8 0000000000000086 ffff88023ec7dc88 0000000000000282
May 5 03:58:20 hostname kernel: ffff88023ec7dc88 00000000000007d4 0000000000000086 000000000000000a
May 5 03:58:20 hostname kernel: ffff8808708905f8 ffff88023ec7dfd8 000000000000fbc8 ffff8808708905f8
May 5 03:58:20 hostname kernel: Call Trace:
May 5 03:58:20 hostname kernel: [<ffffffff8109b5de>] ? prepare_to_wait+0x4e/0x80
May 5 03:58:20 hostname kernel: [<ffffffffa035d3cd>] vhost_work_flush+0x9d/0x120 [vhost_net]
May 5 03:58:20 hostname kernel: [<ffffffff8109b2b0>] ? autoremove_wake_function+0x0/0x40
May 5 03:58:20 hostname kernel: [<ffffffffa035d469>] vhost_poll_flush+0x19/0x20 [vhost_net]
May 5 03:58:20 hostname kernel: [<ffffffffa035f148>] vhost_net_flush_vq+0x38/0x60 [vhost_net]
May 5 03:58:20 hostname kernel: [<ffffffffa03603bf>] vhost_net_ioctl+0x31f/0x5d0 [vhost_net]
May 5 03:58:20 hostname kernel: [<ffffffff8122dad1>] ? avc_has_perm+0x71/0x90
May 5 03:58:20 hostname kernel: [<ffffffff8119db52>] vfs_ioctl+0x22/0xa0
May 5 03:58:20 hostname kernel: [<ffffffff8119dcf4>] do_vfs_ioctl+0x84/0x580
May 5 03:58:20 hostname kernel: [<ffffffff8119e271>] sys_ioctl+0x81/0xa0
May 5 03:58:20 hostname kernel: [<ffffffff810e1e6e>] ? __audit_syscall_exit+0x25e/0x290
May 5 03:58:20 hostname kernel: [<ffffffff8100b072>] system_call_fastpath+0x16/0x1b
Environment
- Red Hat Enterprise Virtualization 3.3 / Red Hat Enterprise Virtualization Hypervisor release 6.5 (20140121.0.el6ev)
- also seen in Red Hat Enterprise Linux 6.5
kernel-2.6.32-431.3.1.el6.x86_64
qemu-kvm-rhev-0.12.1.2-2.415.el6_5.3.x86_64
Subscriber exclusive content
A Red Hat subscription provides unlimited access to our knowledgebase, tools, and much more.