nova instances remain in BUILD state if qemu-img convert does not execute to completion in Red Hat OpenStack Platform

Issue

nova instances remain in BUILD state if qemu-img convert does not execute to completion in Red Hat OpenStack Platform. Attempts to reset and delete the instance fail.

When a nova instance is spawned and a new base image file is created, nova does not kill qemu-img convert if nova instance creation timeouts are hit.

Example output:

[stack@director ~]$ nova list --all-tenants --fields name,status,task_state,power_state,networks,image,host | grep -i build
(...)
| <UUID> | Test                                    | BUILD   | deleting   | NOSTATE     | net=10.10.10.7                            | {u'id': u'<UUID>', u'links': [{u'href': u'https://<URL>:13774/images/<UUID>', u'rel': u'bookmark'}]} | compute-1.localdomain |

On the compute node, one can observe a conversion process running, even hours or days after the instance was initially started.

[heat-admin@compute-1 ~]$ ps  aux | grep 'qemu-img convert'
heat-ad+  298430  0.0  0.0 112660   976 pts/9    S+   13:12   0:00 grep --color=auto qemu-img convert
nova      571213  0.0  0.0 589004 29556 ?        S     2017   0:01 qemu-img convert -O raw /var/lib/nova/instances/_base/<UUID1>.part /var/lib/nova/instances/_base/<UUID1>.converted -f vmdk
nova      748856  0.0  0.0 589004 29576 ?        S     2017   0:16 qemu-img convert -O raw /var/lib/nova/instances/_base/<UUID2>.part /var/lib/nova/instances/_base/<UUID2>.converted -f vmdk

One can also see incomplete _base files:

[heat-admin@compute-1 ~]$ sudo ls -alZ /var/lib/nova/instances/_base
drwxr-xr-x. nova nova system_u:object_r:nova_var_lib_t:s0 .
drwxr-xr-x. nova nova system_u:object_r:nova_var_lib_t:s0 ..
-rw-r--r--. nova nova system_u:object_r:nova_var_lib_t:s0 <UUID1>.converted
-rw-r--r--. nova nova system_u:object_r:nova_var_lib_t:s0 <UUID1>.part
-rw-r--r--. nova nova system_u:object_r:nova_var_lib_t:s0 <UUID2>.converted
-rw-r--r--. nova nova system_u:object_r:nova_var_lib_t:s0 <UUID2>.part