Cannot live migrate cinder volume attached to instance from one backend to another backend
Issue
- Cannot live migrate cinder volume attached to instance from one backend to another backend.
- Migration is getting failed with following call trace in
/var/log/cinder/volume.logfile.
2017-03-21 10:22:32.937 910658 DEBUG cinder.volume.drivers.rbd [req-b982cc01-c2d4-4f76-bd5f-750fdd3699a2 3571b83dfc3f499ca9772ca6bd54edcc 077cf6fb5a314cefb0a36bc4c98744ed -
default default] opening connection to ceph cluster (timeout=-1). _connect_to_rados /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:328
2017-03-21 10:22:32.931 910683 ERROR oslo_messaging.rpc.server [req-b982cc01-c2d4-4f76-bd5f-750fdd3699a2 3571b83dfc3f499ca9772ca6bd54edcc 077cf6fb5a314cefb0a36bc4c98744ed -
default default] Exception during message handling
2017-03-21 10:22:32.931 910683 ERROR oslo_messaging.rpc.server Traceback (most recent call last):
2017-03-21 10:22:32.931 910683 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/server.py", line 133, in _process_incoming
2017-03-21 10:22:32.931 910683 ERROR oslo_messaging.rpc.server res = self.dispatcher.dispatch(message)
2017-03-21 10:22:32.931 910683 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/dispatcher.py", line 150, in dispatch
2017-03-21 10:22:32.931 910683 ERROR oslo_messaging.rpc.server return self._do_dispatch(endpoint, method, ctxt, args)
2017-03-21 10:22:32.931 910683 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/dispatcher.py", line 121, in _do_dispatch
2017-03-21 10:22:32.931 910683 ERROR oslo_messaging.rpc.server result = func(ctxt, **new_args)
2017-03-21 10:22:32.931 910683 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 4427, in migrate_volume
2017-03-21 10:22:32.931 910683 ERROR oslo_messaging.rpc.server new_type_id=new_type_id, volume=volume)
2017-03-21 10:22:32.931 910683 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 1990, in migrate_volume
2017-03-21 10:22:32.931 910683 ERROR oslo_messaging.rpc.server volume.save()
2017-03-21 10:22:32.931 910683 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__
2017-03-21 10:22:32.931 910683 ERROR oslo_messaging.rpc.server self.force_reraise()
2017-03-21 10:22:32.931 910683 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise
2017-03-21 10:22:32.931 910683 ERROR oslo_messaging.rpc.server six.reraise(self.type_, self.value, self.tb)
2017-03-21 10:22:32.931 910683 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 1983, in migrate_volume
2017-03-21 10:22:32.931 910683 ERROR oslo_messaging.rpc.server new_type_id)
2017-03-21 10:22:32.931 910683 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 1795, in _migrate_volume_generic
2017-03-21 10:22:32.931 910683 ERROR oslo_messaging.rpc.server new_volume)
2017-03-21 10:22:32.931 910683 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__
2017-03-21 10:22:32.931 910683 ERROR oslo_messaging.rpc.server self.force_reraise()
2017-03-21 10:22:32.931 910683 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise
2017-03-21 10:22:32.931 910683 ERROR oslo_messaging.rpc.server six.reraise(self.type_, self.value, self.tb)
2017-03-21 10:22:32.931 910683 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 1788, in _migrate_volume_generic
2017-03-21 10:22:32.931 910683 ERROR oslo_messaging.rpc.server new_volume.id)
2017-03-21 10:22:32.931 910683 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/cinder/compute/nova.py", line 178, in update_server_volume
2017-03-21 10:22:32.931 910683 ERROR oslo_messaging.rpc.server nova = novaclient(context, admin_endpoint=True, privileged_user=True)
2017-03-21 10:22:32.931 910683 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/cinder/compute/nova.py", line 136, in novaclient
2017-03-21 10:22:32.931 910683 ERROR oslo_messaging.rpc.server **region_filter)
2017-03-21 10:22:32.931 910683 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/novaclient/service_catalog.py", line 84, in url_for
2017-03-21 10:22:32.931 910683 ERROR oslo_messaging.rpc.server raise novaclient.exceptions.EndpointNotFound()
2017-03-21 10:22:32.931 910683 ERROR oslo_messaging.rpc.server EndpointNotFound
2017-03-21 10:22:32.931 910683 ERROR oslo_messaging.rpc.server
Environment
- Red Hat OpenStack Platform 10
Subscriber exclusive content
A Red Hat subscription provides unlimited access to our knowledgebase of over 48,000 articles and solutions.
Welcome! Check out the Getting Started with Red Hat page for quick tours and guides for common tasks.
