Some instances aren't able to start after upgrade from OSP 6 to OSP 8

Issue

• After doing an upgrade from RHOSP 6 to 8, some instances are not able to start on OSP 8 version.
• Following call trace reported in message file while trying to start an existing instance :

2016-10-16 11:45:23.316 26103 ERROR oslo_messaging.rpc.dispatcher [req-76f17cee-7601-418f-a464-5f1ccf50d9ce admin 048f23a34f524212be242df3b8032a69 - - -] Exception during message handling: Remote error: InvalidTargetVersion Invalid target version 1.2
[u'Traceback (most recent call last):\n', u'  File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/dispatcher.py", line 142, in _dispatch_and_reply\n    executor_callback))\n', u'  File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/dispatcher.py", line 186, in _dispatch\n    executor_callback)\n', u'  File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/dispatcher.py", line 129, in _do_dispatch\n    result = func(ctxt, **new_args)\n', u'  File "/usr/lib/python2.7/site-packages/nova/conductor/manager.py", line 937, in object_class_action_versions\n    context, objname, objmethod, object_versions, args, kwargs)\n', u'  File "/usr/lib/python2.7/site-packages/nova/conductor/manager.py", line 477, in object_class_action_versions\n    if isinstance(result, nova_object.NovaObject) else result)\n', u'  File "/usr/lib/python2.7/site-packages/oslo_versionedobjects/base.py", line 514, in obj_to_primitive\n    raise exception.InvalidTargetVersion(version=target_version)\n', u'InvalidTargetVersion: Invalid target version 1.2\n'].
2016-10-16 11:45:23.316 26103 ERROR oslo_messaging.rpc.dispatcher Traceback (most recent call last):
2016-10-16 11:45:23.316 26103 ERROR oslo_messaging.rpc.dispatcher   File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/dispatcher.py", line 142, in _dispatch_and_reply
2016-10-16 11:45:23.316 26103 ERROR oslo_messaging.rpc.dispatcher     executor_callback))
2016-10-16 11:45:23.316 26103 ERROR oslo_messaging.rpc.dispatcher   File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/dispatcher.py", line 186, in _dispatch
2016-10-16 11:45:23.316 26103 ERROR oslo_messaging.rpc.dispatcher     executor_callback)
2016-10-16 11:45:23.316 26103 ERROR oslo_messaging.rpc.dispatcher   File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/dispatcher.py", line 129, in _do_dispatch
2016-10-16 11:45:23.316 26103 ERROR oslo_messaging.rpc.dispatcher     result = func(ctxt, **new_args)
2016-10-16 11:45:23.316 26103 ERROR oslo_messaging.rpc.dispatcher   File "/usr/lib/python2.7/site-packages/nova/exception.py", line 89, in wrapped
2016-10-16 11:45:23.316 26103 ERROR oslo_messaging.rpc.dispatcher     payload)
2016-10-16 11:45:23.316 26103 ERROR oslo_messaging.rpc.dispatcher   File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 204, in __exit__
2016-10-16 11:45:23.316 26103 ERROR oslo_messaging.rpc.dispatcher     six.reraise(self.type_, self.value, self.tb)
2016-10-16 11:45:23.316 26103 ERROR oslo_messaging.rpc.dispatcher   File "/usr/lib/python2.7/site-packages/nova/exception.py", line 72, in wrapped
2016-10-16 11:45:23.316 26103 ERROR oslo_messaging.rpc.dispatcher     return f(self, context, *args, **kw)
2016-10-16 11:45:23.316 26103 ERROR oslo_messaging.rpc.dispatcher   File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 350, in decorated_function
2016-10-16 11:45:23.316 26103 ERROR oslo_messaging.rpc.dispatcher     LOG.warning(msg, e, instance=instance)
2016-10-16 11:45:23.316 26103 ERROR oslo_messaging.rpc.dispatcher   File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 204, in __exit__
2016-10-16 11:45:23.316 26103 ERROR oslo_messaging.rpc.dispatcher     six.reraise(self.type_, self.value, self.tb)
2016-10-16 11:45:23.316 26103 ERROR oslo_messaging.rpc.dispatcher   File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 323, in decorated_function
2016-10-16 11:45:23.316 26103 ERROR oslo_messaging.rpc.dispatcher     return function(self, context, *args, **kwargs)
2016-10-16 11:45:23.316 26103 ERROR oslo_messaging.rpc.dispatcher   File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 400, in decorated_function
2016-10-16 11:45:23.316 26103 ERROR oslo_messaging.rpc.dispatcher     return function(self, context, *args, **kwargs)
2016-10-16 11:45:23.316 26103 ERROR oslo_messaging.rpc.dispatcher   File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 378, in decorated_function
2016-10-16 11:45:23.316 26103 ERROR oslo_messaging.rpc.dispatcher     kwargs['instance'], e, sys.exc_info())
2016-10-16 11:45:23.316 26103 ERROR oslo_messaging.rpc.dispatcher   File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 204, in __exit__
2016-10-16 11:45:23.316 26103 ERROR oslo_messaging.rpc.dispatcher     six.reraise(self.type_, self.value, self.tb)
2016-10-16 11:45:23.316 26103 ERROR oslo_messaging.rpc.dispatcher   File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 366, in decorated_function
2016-10-16 11:45:23.316 26103 ERROR oslo_messaging.rpc.dispatcher     return function(self, context, *args, **kwargs)
2016-10-16 11:45:23.316 26103 ERROR oslo_messaging.rpc.dispatcher   File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 2532, in start_instance
2016-10-16 11:45:23.316 26103 ERROR oslo_messaging.rpc.dispatcher     self._power_on(context, instance)
2016-10-16 11:45:23.316 26103 ERROR oslo_messaging.rpc.dispatcher   File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 2505, in _power_on
2016-10-16 11:45:23.316 26103 ERROR oslo_messaging.rpc.dispatcher     block_device_info)
2016-10-16 11:45:23.316 26103 ERROR oslo_messaging.rpc.dispatcher   File "/usr/lib/python2.7/site-packages/nova/virt/libvirt/driver.py", line 2428, in power_on
2016-10-16 11:45:23.316 26103 ERROR oslo_messaging.rpc.dispatcher     self._hard_reboot(context, instance, network_info, block_device_info)
2016-10-16 11:45:23.316 26103 ERROR oslo_messaging.rpc.dispatcher   File "/usr/lib/python2.7/site-packages/nova/virt/libvirt/driver.py", line 2286, in _hard_reboot
2016-10-16 11:45:23.316 26103 ERROR oslo_messaging.rpc.dispatcher     write_to_disk=True)
2016-10-16 11:45:23.316 26103 ERROR oslo_messaging.rpc.dispatcher   File "/usr/lib/python2.7/site-packages/nova/virt/libvirt/driver.py", line 4505, in _get_guest_xml
2016-10-16 11:45:23.316 26103 ERROR oslo_messaging.rpc.dispatcher     context)
2016-10-16 11:45:23.316 26103 ERROR oslo_messaging.rpc.dispatcher   File "/usr/lib/python2.7/site-packages/nova/virt/libvirt/driver.py", line 4311, in _get_guest_config
2016-10-16 11:45:23.316 26103 ERROR oslo_messaging.rpc.dispatcher     instance.numa_topology, flavor, pci_devs, allowed_cpus, image_meta)
2016-10-16 11:45:23.316 26103 ERROR oslo_messaging.rpc.dispatcher   File "/usr/lib/python2.7/site-packages/oslo_versionedobjects/base.py", line 66, in getter
2016-10-16 11:45:23.316 26103 ERROR oslo_messaging.rpc.dispatcher     self.obj_load_attr(name)
2016-10-16 11:45:23.316 26103 ERROR oslo_messaging.rpc.dispatcher   File "/usr/lib/python2.7/site-packages/nova/objects/instance.py", line 866, in obj_load_attr
2016-10-16 11:45:23.316 26103 ERROR oslo_messaging.rpc.dispatcher     self._load_numa_topology()
2016-10-16 11:45:23.316 26103 ERROR oslo_messaging.rpc.dispatcher   File "/usr/lib/python2.7/site-packages/nova/objects/instance.py", line 738, in _load_numa_topology
2016-10-16 11:45:23.316 26103 ERROR oslo_messaging.rpc.dispatcher     self._context, self.uuid)
2016-10-16 11:45:23.316 26103 ERROR oslo_messaging.rpc.dispatcher   File "/usr/lib/python2.7/site-packages/oslo_versionedobjects/base.py", line 169, in wrapper
2016-10-16 11:45:23.316 26103 ERROR oslo_messaging.rpc.dispatcher     args, kwargs)
2016-10-16 11:45:23.316 26103 ERROR oslo_messaging.rpc.dispatcher   File "/usr/lib/python2.7/site-packages/nova/conductor/rpcapi.py", line 229, in object_class_action
2016-10-16 11:45:23.316 26103 ERROR oslo_messaging.rpc.dispatcher     args, kwargs)
2016-10-16 11:45:23.316 26103 ERROR oslo_messaging.rpc.dispatcher   File "/usr/lib/python2.7/site-packages/nova/conductor/rpcapi.py", line 237, in object_class_action_versions
2016-10-16 11:45:23.316 26103 ERROR oslo_messaging.rpc.dispatcher     args=args, kwargs=kwargs)
2016-10-16 11:45:23.316 26103 ERROR oslo_messaging.rpc.dispatcher   File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/client.py", line 158, in call
2016-10-16 11:45:23.316 26103 ERROR oslo_messaging.rpc.dispatcher     retry=self.retry)
2016-10-16 11:45:23.316 26103 ERROR oslo_messaging.rpc.dispatcher   File "/usr/lib/python2.7/site-packages/oslo_messaging/transport.py", line 90, in _send
2016-10-16 11:45:23.316 26103 ERROR oslo_messaging.rpc.dispatcher     timeout=timeout, retry=retry)
2016-10-16 11:45:23.316 26103 ERROR oslo_messaging.rpc.dispatcher   File "/usr/lib/python2.7/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 431, in send
2016-10-16 11:45:23.316 26103 ERROR oslo_messaging.rpc.dispatcher     retry=retry)
2016-10-16 11:45:23.316 26103 ERROR oslo_messaging.rpc.dispatcher   File "/usr/lib/python2.7/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 422, in _send
2016-10-16 11:45:23.316 26103 ERROR oslo_messaging.rpc.dispatcher     raise result
2016-10-16 11:45:23.316 26103 ERROR oslo_messaging.rpc.dispatcher RemoteError: Remote error: InvalidTargetVersion Invalid target version 1.2
2016-10-16 11:45:23.316 26103 ERROR oslo_messaging.rpc.dispatcher [u'Traceback (most recent call last):\n', u'  File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/dispatcher.py", line 142, in _dispatch_and_reply\n    executor_callback))\n', u'  File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/dispatcher.py", line 186, in _dispatch\n    executor_callback)\n', u'  File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/dispatcher.py", line 129, in _do_dispatch\n    result = func(ctxt, **new_args)\n', u'  File "/usr/lib/python2.7/site-packages/nova/conductor/manager.py", line 937, in object_class_action_versions\n    context, objname, objmethod, object_versions, args, kwargs)\n', u'  File "/usr/lib/python2.7/site-packages/nova/conductor/manager.py", line 477, in object_class_action_versions\n    if isinstance(result, nova_object.NovaObject) else result)\n', u'  File "/usr/lib/python2.7/site-packages/oslo_versionedobjects/base.py", line 514, in obj_to_primitive\n    raise exception.InvalidTargetVersion(version=target_version)\n', u'InvalidTargetVersion: Invalid target version 1.2\n'].