Cannot live migrate cinder volume attached to instance from one backend to another backend

Solution Verified - Updated -

Issue

  • Cannot live migrate cinder volume attached to instance from one backend to another backend.
  • Migration is getting failed with following call trace in /var/log/cinder/volume.log file.
2017-03-21 10:22:32.937 910658 DEBUG cinder.volume.drivers.rbd [req-b982cc01-c2d4-4f76-bd5f-750fdd3699a2 3571b83dfc3f499ca9772ca6bd54edcc 077cf6fb5a314cefb0a36bc4c98744ed - 
default default] opening connection to ceph cluster (timeout=-1). _connect_to_rados /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:328
2017-03-21 10:22:32.931 910683 ERROR oslo_messaging.rpc.server [req-b982cc01-c2d4-4f76-bd5f-750fdd3699a2 3571b83dfc3f499ca9772ca6bd54edcc 077cf6fb5a314cefb0a36bc4c98744ed - 
default default] Exception during message handling
2017-03-21 10:22:32.931 910683 ERROR oslo_messaging.rpc.server Traceback (most recent call last):
2017-03-21 10:22:32.931 910683 ERROR oslo_messaging.rpc.server   File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/server.py", line 133, in _process_incoming
2017-03-21 10:22:32.931 910683 ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
2017-03-21 10:22:32.931 910683 ERROR oslo_messaging.rpc.server   File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/dispatcher.py", line 150, in dispatch
2017-03-21 10:22:32.931 910683 ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
2017-03-21 10:22:32.931 910683 ERROR oslo_messaging.rpc.server   File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/dispatcher.py", line 121, in _do_dispatch
2017-03-21 10:22:32.931 910683 ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
2017-03-21 10:22:32.931 910683 ERROR oslo_messaging.rpc.server   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 4427, in migrate_volume
2017-03-21 10:22:32.931 910683 ERROR oslo_messaging.rpc.server     new_type_id=new_type_id, volume=volume)
2017-03-21 10:22:32.931 910683 ERROR oslo_messaging.rpc.server   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 1990, in migrate_volume
2017-03-21 10:22:32.931 910683 ERROR oslo_messaging.rpc.server     volume.save()
2017-03-21 10:22:32.931 910683 ERROR oslo_messaging.rpc.server   File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__
2017-03-21 10:22:32.931 910683 ERROR oslo_messaging.rpc.server     self.force_reraise()
2017-03-21 10:22:32.931 910683 ERROR oslo_messaging.rpc.server   File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise
2017-03-21 10:22:32.931 910683 ERROR oslo_messaging.rpc.server     six.reraise(self.type_, self.value, self.tb)
2017-03-21 10:22:32.931 910683 ERROR oslo_messaging.rpc.server   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 1983, in migrate_volume
2017-03-21 10:22:32.931 910683 ERROR oslo_messaging.rpc.server     new_type_id)
2017-03-21 10:22:32.931 910683 ERROR oslo_messaging.rpc.server   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 1795, in _migrate_volume_generic
2017-03-21 10:22:32.931 910683 ERROR oslo_messaging.rpc.server     new_volume)
2017-03-21 10:22:32.931 910683 ERROR oslo_messaging.rpc.server   File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__
2017-03-21 10:22:32.931 910683 ERROR oslo_messaging.rpc.server     self.force_reraise()
2017-03-21 10:22:32.931 910683 ERROR oslo_messaging.rpc.server   File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise
2017-03-21 10:22:32.931 910683 ERROR oslo_messaging.rpc.server     six.reraise(self.type_, self.value, self.tb)
2017-03-21 10:22:32.931 910683 ERROR oslo_messaging.rpc.server   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 1788, in _migrate_volume_generic
2017-03-21 10:22:32.931 910683 ERROR oslo_messaging.rpc.server     new_volume.id)
2017-03-21 10:22:32.931 910683 ERROR oslo_messaging.rpc.server   File "/usr/lib/python2.7/site-packages/cinder/compute/nova.py", line 178, in update_server_volume
2017-03-21 10:22:32.931 910683 ERROR oslo_messaging.rpc.server     nova = novaclient(context, admin_endpoint=True, privileged_user=True)
2017-03-21 10:22:32.931 910683 ERROR oslo_messaging.rpc.server   File "/usr/lib/python2.7/site-packages/cinder/compute/nova.py", line 136, in novaclient
2017-03-21 10:22:32.931 910683 ERROR oslo_messaging.rpc.server     **region_filter)
2017-03-21 10:22:32.931 910683 ERROR oslo_messaging.rpc.server   File "/usr/lib/python2.7/site-packages/novaclient/service_catalog.py", line 84, in url_for
2017-03-21 10:22:32.931 910683 ERROR oslo_messaging.rpc.server     raise novaclient.exceptions.EndpointNotFound()
2017-03-21 10:22:32.931 910683 ERROR oslo_messaging.rpc.server EndpointNotFound
2017-03-21 10:22:32.931 910683 ERROR oslo_messaging.rpc.server 

Environment

  • Red Hat OpenStack Platform 10

Subscriber exclusive content

A Red Hat subscription provides unlimited access to our knowledgebase of over 48,000 articles and solutions.

Current Customers and Partners

Log in for full access

Log In
Close

Welcome! Check out the Getting Started with Red Hat page for quick tours and guides for common tasks.