Unable to Migrate cinder volumes to different storage instances

Solution In Progress - Updated -

Issue

  • We have three LVM backends in three different storage nodes. We are unable to migrate volumes successfully from one storage node to another.
  • From source storage node:
2015-07-02 15:08:09.954 2992 ERROR cinder.volume.manager [req-5315513d-d267-4b11-a015-d89f819c2288 38cedeb84107464d803f72640da29f98 724e67dd9d7647f3bd32ebb2dcedaa1d - - -] Failed to copy volume 7b6f5b4a-bbf0-4f88-ac8a-6a3174739620 to 5c551980-4688-42c0-a934-df80e42aca92
2015-07-02 15:08:10.192 2992 ERROR oslo.messaging.rpc.dispatcher [req-5315513d-d267-4b11-a015-d89f819c2288 38cedeb84107464d803f72640da29f98 724e67dd9d7647f3bd32ebb2dcedaa1d - - -] Exception during message handling: 
2015-07-02 15:08:10.192 2992 TRACE oslo.messaging.rpc.dispatcher Traceback (most recent call last):
2015-07-02 15:08:10.192 2992 TRACE oslo.messaging.rpc.dispatcher   File "/usr/lib/python2.7/site-packages/oslo/messaging/rpc/dispatcher.py", line 133, in _dispatch_and_reply
2015-07-02 15:08:10.192 2992 TRACE oslo.messaging.rpc.dispatcher     incoming.message))
2015-07-02 15:08:10.192 2992 TRACE oslo.messaging.rpc.dispatcher   File "/usr/lib/python2.7/site-packages/oslo/messaging/rpc/dispatcher.py", line 176, in _dispatch
2015-07-02 15:08:10.192 2992 TRACE oslo.messaging.rpc.dispatcher     return self._do_dispatch(endpoint, method, ctxt, args)
2015-07-02 15:08:10.192 2992 TRACE oslo.messaging.rpc.dispatcher   File "/usr/lib/python2.7/site-packages/oslo/messaging/rpc/dispatcher.py", line 122, in _do_dispatch
2015-07-02 15:08:10.192 2992 TRACE oslo.messaging.rpc.dispatcher     result = getattr(endpoint, method)(ctxt, **new_args)
2015-07-02 15:08:10.192 2992 TRACE oslo.messaging.rpc.dispatcher   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 1099, in migrate_volume
2015-07-02 15:08:10.192 2992 TRACE oslo.messaging.rpc.dispatcher     self.db.volume_update(ctxt, volume_ref['id'], updates)
2015-07-02 15:08:10.192 2992 TRACE oslo.messaging.rpc.dispatcher   File "/usr/lib/python2.7/site-packages/cinder/openstack/common/excutils.py", line 68, in __exit__
2015-07-02 15:08:10.192 2992 TRACE oslo.messaging.rpc.dispatcher     six.reraise(self.type_, self.value, self.tb)
2015-07-02 15:08:10.192 2992 TRACE oslo.messaging.rpc.dispatcher   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 1090, in migrate_volume
2015-07-02 15:08:10.192 2992 TRACE oslo.messaging.rpc.dispatcher     new_type_id)
2015-07-02 15:08:10.192 2992 TRACE oslo.messaging.rpc.dispatcher   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 975, in _migrate_volume_generic
2015-07-02 15:08:10.192 2992 TRACE oslo.messaging.rpc.dispatcher     new_volume['migration_status'] = None
2015-07-02 15:08:10.192 2992 TRACE oslo.messaging.rpc.dispatcher   File "/usr/lib/python2.7/site-packages/cinder/openstack/common/excutils.py", line 68, in __exit__
2015-07-02 15:08:10.192 2992 TRACE oslo.messaging.rpc.dispatcher     six.reraise(self.type_, self.value, self.tb)
2015-07-02 15:08:10.192 2992 TRACE oslo.messaging.rpc.dispatcher   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 964, in _migrate_volume_generic
2015-07-02 15:08:10.192 2992 TRACE oslo.messaging.rpc.dispatcher     volume['id'], new_volume['id'])
2015-07-02 15:08:10.192 2992 TRACE oslo.messaging.rpc.dispatcher   File "/usr/lib/python2.7/site-packages/cinder/compute/nova.py", line 120, in update_server_volume
2015-07-02 15:08:10.192 2992 TRACE oslo.messaging.rpc.dispatcher     novaclient(context).volumes.update_server_volume(server_id,
2015-07-02 15:08:10.192 2992 TRACE oslo.messaging.rpc.dispatcher   File "/usr/lib/python2.7/site-packages/cinder/compute/nova.py", line 95, in novaclient
2015-07-02 15:08:10.192 2992 TRACE oslo.messaging.rpc.dispatcher     endpoint_type=endpoint_type)
2015-07-02 15:08:10.192 2992 TRACE oslo.messaging.rpc.dispatcher   File "/usr/lib/python2.7/site-packages/novaclient/service_catalog.py", line 84, in url_for
2015-07-02 15:08:10.192 2992 TRACE oslo.messaging.rpc.dispatcher     raise novaclient.exceptions.EndpointNotFound()

Environment

  • Red Hat Open Stack

Subscriber exclusive content

A Red Hat subscription provides unlimited access to our knowledgebase, tools, and much more.

Current Customers and Partners

Log in for full access

Log In

New to Red Hat?

Learn more about Red Hat subscriptions

Using a Red Hat product through a public cloud?

How to access this content