Live migration of a RHEV VM fails with "internal error ifname vnetX not in key map" on one host and "Domain not found: no domain with matching uuid" on the other.

Issue

• We installed two new hypervisors and when we migrate VMs from other existing hypervisors, we found the following error in syslog;

Apr  7 15:21:26 acrhext01 vdsm vm.Vm ERROR vmId=`f9ea54f4-b55e-4b91-abf3-ede2c00d49f2`::The vm start process failed#012Traceback (most recent call last):#012  File "/usr/share/vdsm/vm.py", line 2147, in _startUnderlyingVm#012    self._waitForIncomingMigrationFinish()#012  File "/usr/share/vdsm/vm.py", line 3571, in _waitForIncomingMigrationFinish#012    self._connection.lookupByUUIDString(self.id),#012  File "/usr/lib64/python2.6/site-packages/vdsm/libvirtconnection.py", line 76, in wrapper#012    ret = f(*args, **kwargs)#012  File "/usr/lib64/python2.6/site-packages/libvirt.py", line 2858, in lookupByUUIDString#012    if ret is None:raise libvirtError('virDomainLookupByUUIDString() failed', conn=self)#012libvirtError: Domain not found: no domain with matching uuid 'f9ea54f4-b55e-4b91-abf3-ede2c00d49f2'

• The VDSM logs on the source host contain;

Thread-957613::ERROR::2014-04-07 15:20:23,333::vm::338::vm.Vm::(run) vmId=`f9ea54f4-b55e-4b91-abf3-ede2c00d49f2`::Failed to migrate
Traceback (most recent call last):
  File "/usr/share/vdsm/vm.py", line 324, in run
    self._startUnderlyingMigration()
  File "/usr/share/vdsm/vm.py", line 406, in _startUnderlyingMigration
    None, maxBandwidth)
  File "/usr/share/vdsm/vm.py", line 845, in f
    ret = attr(*args, **kwargs)
  File "/usr/lib64/python2.6/site-packages/vdsm/libvirtconnection.py", line 76, in wrapper
    ret = f(*args, **kwargs)
  File "/usr/lib64/python2.6/site-packages/libvirt.py", line 1178, in migrateToURI2
    if ret == -1: raise libvirtError ('virDomainMigrateToURI2() failed', dom=self)
libvirtError: internal error ifname "vnet0" not in key map

• The VDSM logs on the destination host contain;

Thread-315::ERROR::2014-04-07 15:21:26,254::vm::2169::vm.Vm::(_startUnderlyingVm) vmId=`f9ea54f4-b55e-4b91-abf3-ede2c00d49f2`::The vm start process failed
Traceback (most recent call last):
  File "/usr/share/vdsm/vm.py", line 2147, in _startUnderlyingVm
    self._waitForIncomingMigrationFinish()
  File "/usr/share/vdsm/vm.py", line 3571, in _waitForIncomingMigrationFinish
    self._connection.lookupByUUIDString(self.id),
  File "/usr/lib64/python2.6/site-packages/vdsm/libvirtconnection.py", line 76, in wrapper
    ret = f(*args, **kwargs)
  File "/usr/lib64/python2.6/site-packages/libvirt.py", line 2858, in lookupByUUIDString
    if ret is None:raise libvirtError('virDomainLookupByUUIDString() failed', conn=self)
libvirtError: Domain not found: no domain with matching uuid 'f9ea54f4-b55e-4b91-abf3-ede2c00d49f2

• Only migrations for VMs that used the vmfex hook were failing. VMs that did not use this hook could be migrated successfully.

• VMs that fail to migrate to the destination host can be manually started on that host.