KVM virtual components disappear unexpected or are inaccessible when there is high load

Solution Verified - Updated -

Issue

  • We move our environment from physical into KVM virtual machines. All components have been virtualized and they are working, network configured on KVM. All so far so good, but under load strange things are happening.

  • Virtual components disappear, or are inaccessible.

  • Recently we had the following issues during the tests:

  1. qemu-kvm process was not running.
  2. qemu-kvm process for another virtual machine is running, but it is not accessible.
  3. some work load caused greater CPU usage. At that time main KVM process (libvirtd) disappeared. Xvnc disappeared too and is not directly associated with KVM. We have observed some overall problem with host1, so it was rebooted.
  • In /var/log/messages on guest operating systems we can see:
Jan 23 02:48:05 hostname1 kernel: Clocksource tsc unstable (delta = -51539647886 ns)

Jan 22 03:52:45 hostname1 kernel: BUG: soft lockup - CPU#0 stuck for 65s! [puppy:4340]
Jan 22 04:12:48 hostname1 kernel: BUG: soft lockup - CPU#5 stuck for 67s! [ldapd:17312]
Jan 22 04:13:24 hostname1 kernel: BUG: soft lockup - CPU#2 stuck for 67s! [puppy:14990]
Jan 22 04:13:24 hostname1 kernel: BUG: soft lockup - CPU#6 stuck for 67s! [puppy:14947]
Jan 22 04:18:39 hostname1 kernel: BUG: soft lockup - CPU#5 stuck for 81s! [Sfc:18652]
Jan 23 09:08:56 hostname1 kernel: BUG: soft lockup - CPU#0 stuck for 76s! [scanproc:13165]
  • We tried to add clocksource_failover to GRUB but it did not change anything.

Environment

  • KVM host

    • Red Hat Enterprise Linux 6
  • KVM guests

    • Red Hat Enterprise Linux 5
    • Red Hat Enterprise Linux 6

Subscriber exclusive content

A Red Hat subscription provides unlimited access to our knowledgebase, tools, and much more.

Current Customers and Partners

Log in for full access

Log In

New to Red Hat?

Learn more about Red Hat subscriptions

Using a Red Hat product through a public cloud?

How to access this content