libvirtError: Timed out during operation: cannot acquire state change lock

Solution Verified - Updated -

Issue

  • After rebooting KVM host,virtual machine will not start and remains in paused state. When attempting to resume, shutdown or force off the VM, the following error is seen:

    Traceback (most recent call last):
    File "/usr/share/virt-manager/virtManager/engine.py", line 515, in shutdown_domain
    vm.shutdown()
    File "/usr/share/virt-manager/virtManager/domain.py", line 567, in shutdown
    self.vm.shutdown()
    File "/usr/lib64/python2.4/site-packages/libvirt.py", line 662, in shutdown
    if ret == -1: raise libvirtError ('virDomainShutdown() failed', dom=self)
    libvirtError: Timed out during operation: cannot acquire state change lock
    
  • This could also happen when trying to live migrate a VM and the following error would be seen in /var/log/nova/nova-compute.log:

2020-06-22 13:58:06.722 88272 ERROR oslo_messaging.rpc.server     if ret == -1: raise libvirtError ('virDomainShutdown() failed', dom=self)
2020-06-22 13:58:06.722 88272 ERROR oslo_messaging.rpc.server libvirtError: Timed out during operation: cannot acquire state change lock (held by monitor=qemuProcessReconnect)
2020-06-22 13:58:06.722 88272 ERROR oslo_messaging.rpc.server
  • The following error message could be seen in /var/log/messages:
Jun 22 14:30:52 openstack-nova-compute libvirtd: 2020-06-22 21:30:52.697+0000: 89350: error : virProcessKillPainfully:401 : Failed to terminate process 79972 with SIGKILL: Device or resource busy
Jun 22 14:31:15 openstack-nova-compute libvirtd: 2020-06-22 21:31:15.078+0000: 89352: error : virProcessKillPainfully:401 : Failed to terminate process 64013 with SIGKILL: Device or resource busy
  • Some qemu-kvm process could be stuck in Zombie state:
# ps aufxg | grep qemu-kvm
qemu       64013  0.0  0.0      0     0 ?        Zl   Feb02  25:56 [qemu-kvm] <defunct>

Environment

  • Red Hat Enterprise Linux 5 (RHEL)
  • Red Hat Enterprise Linux 6 (RHEL)
  • Red Hat Enterprise Linux 7 (RHEL)
  • libvirt
  • Red Hat OpenStack Platform 10.0 (RHOSP)

Subscriber exclusive content

A Red Hat subscription provides unlimited access to our knowledgebase of over 48,000 articles and solutions.

Current Customers and Partners

Log in for full access

Log In