qemu-kvm process in uninterruptible sleep in RHEV 3.3 or RHEL 6.5

Solution Unverified - Updated -

Issue

  • A qemu-kvm process hosting a VM enters uninterruptible sleep ('D' state) for unclear reasons, and cannot be killed.
  • After forcing an unresponsive VM to poweroff, the rhevh server still has a qemu-kvm process for that VM in uniterruptible sleep. This makes it impossible to restart the VM without first rebooting the host. This is clearly not a good thing.
  • Kernel log shows errors similar to the following:
May  5 03:58:20 hostname kernel: INFO: task qemu-kvm:11375 blocked for more than 120 seconds.
May  5 03:58:20 hostname kernel:      Not tainted 2.6.32-431.3.1.el6.x86_64 #1
May  5 03:58:20 hostname kernel: "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
May  5 03:58:20 hostname kernel: qemu-kvm      D 0000000000000011     0 11375      1 0x00000084
May  5 03:58:20 hostname kernel: ffff88023ec7dcf8 0000000000000086 ffff88023ec7dc88 0000000000000282
May  5 03:58:20 hostname kernel: ffff88023ec7dc88 00000000000007d4 0000000000000086 000000000000000a
May  5 03:58:20 hostname kernel: ffff8808708905f8 ffff88023ec7dfd8 000000000000fbc8 ffff8808708905f8
May  5 03:58:20 hostname kernel: Call Trace:
May  5 03:58:20 hostname kernel: [<ffffffff8109b5de>] ? prepare_to_wait+0x4e/0x80
May  5 03:58:20 hostname kernel: [<ffffffffa035d3cd>] vhost_work_flush+0x9d/0x120 [vhost_net]
May  5 03:58:20 hostname kernel: [<ffffffff8109b2b0>] ? autoremove_wake_function+0x0/0x40
May  5 03:58:20 hostname kernel: [<ffffffffa035d469>] vhost_poll_flush+0x19/0x20 [vhost_net]
May  5 03:58:20 hostname kernel: [<ffffffffa035f148>] vhost_net_flush_vq+0x38/0x60 [vhost_net]
May  5 03:58:20 hostname kernel: [<ffffffffa03603bf>] vhost_net_ioctl+0x31f/0x5d0 [vhost_net]
May  5 03:58:20 hostname kernel: [<ffffffff8122dad1>] ? avc_has_perm+0x71/0x90
May  5 03:58:20 hostname kernel: [<ffffffff8119db52>] vfs_ioctl+0x22/0xa0
May  5 03:58:20 hostname kernel: [<ffffffff8119dcf4>] do_vfs_ioctl+0x84/0x580
May  5 03:58:20 hostname kernel: [<ffffffff8119e271>] sys_ioctl+0x81/0xa0
May  5 03:58:20 hostname kernel: [<ffffffff810e1e6e>] ? __audit_syscall_exit+0x25e/0x290
May  5 03:58:20 hostname kernel: [<ffffffff8100b072>] system_call_fastpath+0x16/0x1b

Environment

  • Red Hat Enterprise Virtualization 3.3 / Red Hat Enterprise Virtualization Hypervisor release 6.5 (20140121.0.el6ev)
  • also seen in Red Hat Enterprise Linux 6.5
  • kernel-2.6.32-431.3.1.el6.x86_64
  • qemu-kvm-rhev-0.12.1.2-2.415.el6_5.3.x86_64

Subscriber exclusive content

A Red Hat subscription provides unlimited access to our knowledgebase, tools, and much more.

Current Customers and Partners

Log in for full access

Log In

New to Red Hat?

Learn more about Red Hat subscriptions

Using a Red Hat product through a public cloud?

How to access this content