Nova Live Migration Failed to Update Database
Environment
- Red Hat Enterprise Linux OpenStack Platform (RHEL-OSP) 6.0
- Red Hat OpenStack Platform 7.0
- Red Hat OpenStack Platform 8.0
Issue
We migrated a group of guests but one guest failed to migrate properly. The instance is running on the new compute node but the nova database information still reflects the old compute node location for the instance.
Resolution
First of all, verify that the instance is correctly running on the new host. Access it e.g. via SSH and perform a few sanity checks to verify that the migration went through o.k.
To correct the Nova database inconsistency the following database commands need to be run.
Access the Nova database:
mysql nova
As a sanity check, run the following command to output the current information for the instance substituting in the UUID:
select node,host,uuid from instances where uuid='UUID';
Get the full hostname of the destination hypervisor (the hypervisor where the instance is currently running)
[stack@undercloud-7 ~]$ nova service-list | grep compute
| 56 | nova-compute | overcloud-compute-0.localdomain | nova | enabled | up | 2016-09-12T17:49:29.000000 | - |
Update the instance to reflect the new compute node location substituting the NEW COMPUTE HOSTNAME as it appears in the output of nova service list and the UUID of the instance:
update instances set host='<NEW COMPUTE HOSTNAME>',node='<NEW COMPUTE HOSTNAME>' where uuid='<UUID>' and deleted = 0;
It may also be needed to update the vm_state and task_state as well as the power_state, depending on the scenario
update instances set node='$hv',host='$hv',vm_state='active',task_state=NULL,power_state=1 where uuid='<UUID>' and deleted = 0;
E.g.
update instances set node='overcloud-compute-0.localdomain',host='overcloud-compute-0.localdomain',vm_state='active',task_state=NULL,power_state=1 where uuid='<UUID>' and deleted = 0;
Validate with the following database statement substituting in UUID for the instance in question to confirm change:
select node,host,uuid from instances where uuid='UUID';
Validate with the nova show command on the instance to confirm everything looks correct:
nova show <UUID>
This solution is part of Red Hat’s fast-track publication program, providing a huge library of solutions that Red Hat engineers have created while supporting our customers. To give you the knowledge you need the instant it becomes available, these articles may be presented in a raw and unedited form.
Welcome! Check out the Getting Started with Red Hat page for quick tours and guides for common tasks.
