RHOSP 12. Impossible to migrate instance to compute node after major upgrade
Issue
After a major upgrade from RHOSP 10 to RHOSP 12 customer is unable to migrate instances to one compute node. The reason is simple: resource_providers table (from nova_api DB) contains a record for this compute with incorrect UUID.
As a result, we get the following errors in controller's nova-api.log:
nova-placement-api.log.4:2018-09-29 03:00:33.567 13 INFO nova.api.openstack.placement.requestlog [req-6b4052bd-320b-423f-9e4c-70ff982b1cc2 05dece580b6045f69b78773126d70e9f d25886a9aad140bc9bd68e4847d07a83 - default default] 172.31.0.21 "GET /placement/resource_providers/COMPUTE_UUID_THAT_DOESN'T_EXIST_IN_nova_api_DB" status: 404 len: 227 microversion: 1.0
At the same time the following error occurs in compute's nova-compute.log:
2018-09-30 04:00:46.130 1 INFO nova.scheduler.client.report [req-bc2109d9-fff0-42a0-b537-1750bfdc13c0 - - - - -] [req-a4bb47ce-552a-4b0e-ab23-8d8c4e4dc26f] Another thread already created a resource provider with the UUID COMPUTE_UUID_THAT_DOESN'T_EXIST_IN_nova_api_DB. Grabbing that record from the placement API.
2018-09-30 04:00:46.142 1 WARNING nova.scheduler.client.report [req-bc2109d9-fff0-42a0-b537-1750bfdc13c0 - - - - -] Unable to refresh my resource provider record
2018-09-30 04:00:46.213 1 WARNING nova.scheduler.client.report [req-bc2109d9-fff0-42a0-b537-1750bfdc13c0 - - - - -] Unable to submit allocation for instance d38c4fd9-f66c-4656-95a9-717c51d067cb (400 <html>
<head>
<title>400 Bad Request</title>
</head>
<body>
<h1>400 Bad Request</h1>
The server could not comply with the request since it is either malformed or otherwise incorrect.<br /><br />
Allocation for resource provider 'COMPUTE_UUID_THAT_DOESN'T_EXIST_IN_nova_api_DB' that does not exist.
</body>
</html>)
Environment
Red Hat OpenStack Platform 12
Subscriber exclusive content
A Red Hat subscription provides unlimited access to our knowledgebase, tools, and much more.