RHEV Manager events tab reports "Domain not found: no domain with matching uuid '<uuid>'"

Solution Verified - Updated -

Environment

  • Red Hat Enterprise Virtualization 3.1
  • Issue observed on both Windows and Linux VMs.

Issue

  • Virtual machines created from templates (and cloned machines as well) randomly stop running with this error message: "Domain not found : no domain with matching uuid"
  • Templates/VMs were created according to the administration guide.

Resolution

  • Applying latest updates to the RHEV environment was seen to resolve this issue.
Workaround:
  • Shutdown the VMs by right-clicking them, instead of, powering them off twice, once from inside the VM and once from the RHEV-Manager.

Root Cause

  • What this all means is just that, the VM is shutdown by passing a command from inside it similar to (init 0 or poweroff).
  • When the VM is shutdown there will always be a small period of time in which the GUI still thinks that the VM is up and running and allow the user to send a shutdown command.
  • When that command reaches vdsm/libvirt the "domain not found" error is triggered.
  • The final result is that the VM is down. The message is just an extra information.
  • So there is nothing to be concerned about. This is expected.

Diagnostic Steps

  • In RHEV-Manager Events Tab:

  VM XXXX is down. Exit message: Domain not found: no domain with matching uuid 'bdcaf285-f897-4e6a-8d28-3e0bf198ab47'.
  • Corresponding events in the /var/log/ovirt-engine/engine.log:

2013-03-11 10:57:27,027 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.DestroyVDSCommand] (QuartzScheduler_Worker-74) START, DestroyVDSCommand(HostName = xxxxxxxxx, HostId = fd5a6158-7d14-11e2-a4f4-00241d3027b3, vmId=bdcaf285-f897-4e6a-8d28-3e0bf198ab47, force=false, secondsToWait=0, gracefully=false), log id: 2c91ee66
2013-03-11 10:57:27,136 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.DestroyVDSCommand] (QuartzScheduler_Worker-74) FINISH, DestroyVDSCommand, log id: 2c91ee66
  • In /var/log/vdsm/vdsm.log of the host on which this VM is running:

Thread-1177397::DEBUG::2013-03-11 09:56:58,568::BindingXMLRPC::894::vds::(wrapper) client [192.168.1.226]::call vmDestroy with ('bdcaf285-f897-4e6a-8d28-3e0bf198ab47',) {}
Thread-1177397::INFO::2013-03-11 09:56:58,568::API::301::vds::(destroy) vmContainerLock acquired by vm bdcaf285-f897-4e6a-8d28-3e0bf198ab47
Thread-1177397::DEBUG::2013-03-11 09:56:58,568::libvirtvm::2382::vm.Vm::(destroy) vmId=`bdcaf285-f897-4e6a-8d28-3e0bf198ab47`::destroy Called
Thread-1177397::INFO::2013-03-11 09:56:58,572::libvirtvm::2335::vm.Vm::(releaseVm) vmId=`bdcaf285-f897-4e6a-8d28-3e0bf198ab47`::Release VM resources
Thread-1177397::WARNING::2013-03-11 09:56:58,572::vm::352::vm.Vm::(_set_lastStatus) vmId=`bdcaf285-f897-4e6a-8d28-3e0bf198ab47`::trying to set state to Powering down when already Down
Thread-1177397::DEBUG::2013-03-11 09:56:58,573::utils::335::vm.Vm::(stop) vmId=`bdcaf285-f897-4e6a-8d28-3e0bf198ab47`::Stop statistics collection
Thread-1177397::INFO::2013-03-11 09:56:58,652::logUtils::37::dispatcher::(wrapper) Run and protect: inappropriateDevices(thiefId='bdcaf285-f897-4e6a-8d28-3e0bf198ab47')
Thread-1177397::DEBUG::2013-03-11 09:56:58,657::libvirtvm::2377::vm.Vm::(deleteVm) vmId=`bdcaf285-f897-4e6a-8d28-3e0bf198ab47`::Total desktops after destroy of bdcaf285-f897-4e6a-8d28-3e0bf198ab47 is 8
Thread-825168::DEBUG::2013-03-11 09:56:57,112::libvirtvm::333::vm.Vm::(handleStatsException) vmId=`bdcaf285-f897-4e6a-8d28-3e0bf198ab47`::VM not found, moving to Down
libvirtError: Domain not found: no domain with matching uuid 'bdcaf285-f897-4e6a-8d28-3e0bf198ab47'
Thread-825168::DEBUG::2013-03-11 09:56:57,134::vm::965::vm.Vm::(setDownStatus) vmId=`bdcaf285-f897-4e6a-8d28-3e0bf198ab47`::Changed state to Down: Domain not found: no domain with matching uuid 'bdcaf285-f897-4e6a-8d28-3e0bf198ab47'

This solution is part of Red Hat’s fast-track publication program, providing a huge library of solutions that Red Hat engineers have created while supporting our customers. To give you the knowledge you need the instant it becomes available, these articles may be presented in a raw and unedited form.

Comments