libvirtError: Timed out during operation: cannot acquire state change lock
Issue
-
After rebooting KVM host,virtual machine will not start and remains in paused state. When attempting to resume, shutdown or force off the VM, the following error is seen:
Traceback (most recent call last): File "/usr/share/virt-manager/virtManager/engine.py", line 515, in shutdown_domain vm.shutdown() File "/usr/share/virt-manager/virtManager/domain.py", line 567, in shutdown self.vm.shutdown() File "/usr/lib64/python2.4/site-packages/libvirt.py", line 662, in shutdown if ret == -1: raise libvirtError ('virDomainShutdown() failed', dom=self) libvirtError: Timed out during operation: cannot acquire state change lock
-
This could also happen when trying to live migrate a VM and the following error would be seen in
/var/log/nova/nova-compute.log
:
2020-06-22 13:58:06.722 88272 ERROR oslo_messaging.rpc.server if ret == -1: raise libvirtError ('virDomainShutdown() failed', dom=self)
2020-06-22 13:58:06.722 88272 ERROR oslo_messaging.rpc.server libvirtError: Timed out during operation: cannot acquire state change lock (held by monitor=qemuProcessReconnect)
2020-06-22 13:58:06.722 88272 ERROR oslo_messaging.rpc.server
- The following error message could be seen in
/var/log/messages
:
Jun 22 14:30:52 openstack-nova-compute libvirtd: 2020-06-22 21:30:52.697+0000: 89350: error : virProcessKillPainfully:401 : Failed to terminate process 79972 with SIGKILL: Device or resource busy
Jun 22 14:31:15 openstack-nova-compute libvirtd: 2020-06-22 21:31:15.078+0000: 89352: error : virProcessKillPainfully:401 : Failed to terminate process 64013 with SIGKILL: Device or resource busy
- Some qemu-kvm process could be stuck in Zombie state:
# ps aufxg | grep qemu-kvm
qemu 64013 0.0 0.0 0 0 ? Zl Feb02 25:56 [qemu-kvm] <defunct>
Environment
- Red Hat Enterprise Linux 5 (RHEL)
- Red Hat Enterprise Linux 6 (RHEL)
- Red Hat Enterprise Linux 7 (RHEL)
- libvirt
- Red Hat OpenStack Platform 10.0 (RHOSP)
Subscriber exclusive content
A Red Hat subscription provides unlimited access to our knowledgebase, tools, and much more.