Cinder does not retype volume with Hitachi backend
Issue
-
Why cinder does not
retype
volume with Hitachi backend? -
Configured multiple Hitachi backends differentiated by port assigment. Cinder
retype
is not possible from backend to another.
enabled_backends = Tier1, Tier2, Tier3
default_volume_type = Tier3
[Tier1]
volume_driver=cinder.volume.drivers.hitachi.hbsd.hbsd_fc.HBSDFCDriver
volume_backend_name=Tier1
hitachi_storage_id=xxxxx
hitachi_pool=005
hitachi_thin_pool=008
hitachi_horcm_user=OS
hitachi_horcm_password_path=/opt/hitachi/hbsd/horcm_password
hitachi_target_ports=CL1-F,CL2-F
hitachi_compute_target_ports=CL1-F,CL2-F
hitachi_zoning_request=False
hitachi_default_copy_method=FULL
use_multipath_for_image_xfer=True
hitachi_copy_version=1.0
hitachi_horcm_add_conf=False
hitachi_horcm_numbers=200,201
[Tier2]
volume_driver=cinder.volume.drivers.hitachi.hbsd.hbsd_fc.HBSDFCDriver
volume_backend_name=Tier2
hitachi_storage_id=xxxxx
hitachi_pool=006
hitachi_thin_pool=008
hitachi_horcm_user=OS
hitachi_horcm_password_path=/opt/hitachi/hbsd/horcm_password
hitachi_target_ports=CL1-F,CL2-F
hitachi_compute_target_ports=CL1-F,CL2-F
hitachi_zoning_request=False
hitachi_default_copy_method=FULL
use_multipath_for_image_xfer=True
hitachi_copy_version=1.0
hitachi_horcm_add_conf=False
hitachi_horcm_numbers=202,203
[Tier3]
volume_driver=cinder.volume.drivers.hitachi.hbsd.hbsd_fc.HBSDFCDriver
volume_backend_name=Tier3
hitachi_storage_id=xxxxx
hitachi_pool=007
hitachi_thin_pool=008
hitachi_horcm_user=OS
hitachi_horcm_password_path=/opt/hitachi/hbsd/horcm_password
hitachi_target_ports=CL1-F,CL2-F
hitachi_compute_target_ports=CL1-F,CL2-F
hitachi_zoning_request=False
hitachi_default_copy_method=FULL
use_multipath_for_image_xfer=True
hitachi_copy_version=1.0
hitachi_horcm_add_conf=False
hitachi_horcm_numbers=204,205
- cinder
retype
operation is getting failed with following call trace and it's deleting the temporary cinder volumes which was created during the retype operation.
2017-03-20 13:15:07.884 4435 ERROR cinder.volume.manager [req-ce0fc17b-0c9d-46e9-82b3-722de16ca19a 8c1aa3f10dfb47f2b061c3674be85515 eb67a97587234f7790070d0a58032ba7 - - -] Failed to copy volume b949b243-123c-4c0f-80a1-01672966cc10 to 4f2b48c5-8330-450b-b668-6b67103ca238
2017-03-20 13:15:07.884 4435 ERROR cinder.volume.manager Traceback (most recent call last):
2017-03-20 13:15:07.884 4435 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 1768, in _migrate_volume_generic
2017-03-20 13:15:07.884 4435 ERROR cinder.volume.manager self._copy_volume_data(ctxt, volume, new_volume, remote='dest')
2017-03-20 13:15:07.884 4435 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 1708, in _copy_volume_data
2017-03-20 13:15:07.884 4435 ERROR cinder.volume.manager remote=dest_remote)
2017-03-20 13:15:07.884 4435 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 1649, in _detach_volume
2017-03-20 13:15:07.884 4435 ERROR cinder.volume.manager rpcapi.terminate_connection(ctxt, volume, properties, force=force)
2017-03-20 13:15:07.884 4435 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/cinder/volume/rpcapi.py", line 248, in terminate_connection
2017-03-20 13:15:07.884 4435 ERROR cinder.volume.manager connector=connector, force=force)
2017-03-20 13:15:07.884 4435 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/client.py", line 158, in call
2017-03-20 13:15:07.884 4435 ERROR cinder.volume.manager retry=self.retry)
2017-03-20 13:15:07.884 4435 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/oslo_messaging/transport.py", line 90, in _send
2017-03-20 13:15:07.884 4435 ERROR cinder.volume.manager timeout=timeout, retry=retry)
2017-03-20 13:15:07.884 4435 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 470, in send
2017-03-20 13:15:07.884 4435 ERROR cinder.volume.manager retry=retry)
2017-03-20 13:15:07.884 4435 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 459, in _send
2017-03-20 13:15:07.884 4435 ERROR cinder.volume.manager result = self._waiter.wait(msg_id, timeout)
2017-03-20 13:15:07.884 4435 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 342, in wait
2017-03-20 13:15:07.884 4435 ERROR cinder.volume.manager message = self.waiters.get(msg_id, timeout=timeout)
2017-03-20 13:15:07.884 4435 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 244, in get
2017-03-20 13:15:07.884 4435 ERROR cinder.volume.manager 'to message ID %s' % msg_id)
2017-03-20 13:15:07.884 4435 ERROR cinder.volume.manager MessagingTimeout: Timed out waiting for a reply to message ID 69113807c44e41e6914d01acb99d40b6
2017-03-20 13:15:07.884 4435 ERROR cinder.volume.manager
2017-03-20 13:15:08.070 4436 INFO cinder.volume.drivers.hitachi.hbsd.hbsd_traceutils [req-ce0fc17b-0c9d-46e9-82b3-722de16ca19a 8c1aa3f10dfb47f2b061c3674be85515 eb67a97587234f7790070d0a58032ba7 - - -] MSGID0000-I: The method delete_volume was called. (config_group: Tier2)
2017-03-20 13:15:08.103 4435 ERROR oslo_messaging.rpc.dispatcher [req-ce0fc17b-0c9d-46e9-82b3-722de16ca19a 8c1aa3f10dfb47f2b061c3674be85515 eb67a97587234f7790070d0a58032ba7 - - -] Exception during message handling: Timed out waiting for a reply to message ID 69113807c44e41e6914d01acb99d40b6
2017-03-20 13:15:08.103 4435 ERROR oslo_messaging.rpc.dispatcher Traceback (most recent call last):
2017-03-20 13:15:08.103 4435 ERROR oslo_messaging.rpc.dispatcher File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/dispatcher.py", line 138, in _dispatch_and_reply
2017-03-20 13:15:08.103 4435 ERROR oslo_messaging.rpc.dispatcher incoming.message))
2017-03-20 13:15:08.103 4435 ERROR oslo_messaging.rpc.dispatcher File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/dispatcher.py", line 185, in _dispatch
2017-03-20 13:15:08.103 4435 ERROR oslo_messaging.rpc.dispatcher return self._do_dispatch(endpoint, method, ctxt, args)
2017-03-20 13:15:08.103 4435 ERROR oslo_messaging.rpc.dispatcher File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/dispatcher.py", line 127, in _do_dispatch
2017-03-20 13:15:08.103 4435 ERROR oslo_messaging.rpc.dispatcher result = func(ctxt, **new_args)
2017-03-20 13:15:08.103 4435 ERROR oslo_messaging.rpc.dispatcher File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2310, in retype
2017-03-20 13:15:08.103 4435 ERROR oslo_messaging.rpc.dispatcher new_reservations, status_update)
2017-03-20 13:15:08.103 4435 ERROR oslo_messaging.rpc.dispatcher File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__
2017-03-20 13:15:08.103 4435 ERROR oslo_messaging.rpc.dispatcher self.force_reraise()
2017-03-20 13:15:08.103 4435 ERROR oslo_messaging.rpc.dispatcher File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise
2017-03-20 13:15:08.103 4435 ERROR oslo_messaging.rpc.dispatcher six.reraise(self.type_, self.value, self.tb)
2017-03-20 13:15:08.103 4435 ERROR oslo_messaging.rpc.dispatcher File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2306, in retype
2017-03-20 13:15:08.103 4435 ERROR oslo_messaging.rpc.dispatcher new_type_id=new_type_id)
2017-03-20 13:15:08.103 4435 ERROR oslo_messaging.rpc.dispatcher File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 1983, in migrate_volume
2017-03-20 13:15:08.103 4435 ERROR oslo_messaging.rpc.dispatcher volume.save()
2017-03-20 13:15:08.103 4435 ERROR oslo_messaging.rpc.dispatcher File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__
2017-03-20 13:15:08.103 4435 ERROR oslo_messaging.rpc.dispatcher self.force_reraise()
2017-03-20 13:15:08.103 4435 ERROR oslo_messaging.rpc.dispatcher File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise
2017-03-20 13:15:08.103 4435 ERROR oslo_messaging.rpc.dispatcher six.reraise(self.type_, self.value, self.tb)
2017-03-20 13:15:08.103 4435 ERROR oslo_messaging.rpc.dispatcher File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 1976, in migrate_volume
2017-03-20 13:15:08.103 4435 ERROR oslo_messaging.rpc.dispatcher new_type_id)
2017-03-20 13:15:08.103 4435 ERROR oslo_messaging.rpc.dispatcher File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 1791, in _migrate_volume_generic
2017-03-20 13:15:08.103 4435 ERROR oslo_messaging.rpc.dispatcher new_volume)
2017-03-20 13:15:08.103 4435 ERROR oslo_messaging.rpc.dispatcher File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__
2017-03-20 13:15:08.103 4435 ERROR oslo_messaging.rpc.dispatcher self.force_reraise()
2017-03-20 13:15:08.103 4435 ERROR oslo_messaging.rpc.dispatcher File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise
2017-03-20 13:15:08.103 4435 ERROR oslo_messaging.rpc.dispatcher six.reraise(self.type_, self.value, self.tb)
2017-03-20 13:15:08.103 4435 ERROR oslo_messaging.rpc.dispatcher File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 1768, in _migrate_volume_generic
2017-03-20 13:15:08.103 4435 ERROR oslo_messaging.rpc.dispatcher self._copy_volume_data(ctxt, volume, new_volume, remote='dest')
2017-03-20 13:15:08.103 4435 ERROR oslo_messaging.rpc.dispatcher File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 1708, in _copy_volume_data
2017-03-20 13:15:08.103 4435 ERROR oslo_messaging.rpc.dispatcher remote=dest_remote)
2017-03-20 13:15:08.103 4435 ERROR oslo_messaging.rpc.dispatcher File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 1649, in _detach_volume
2017-03-20 13:15:08.103 4435 ERROR oslo_messaging.rpc.dispatcher rpcapi.terminate_connection(ctxt, volume, properties, force=force)
2017-03-20 13:15:08.103 4435 ERROR oslo_messaging.rpc.dispatcher File "/usr/lib/python2.7/site-packages/cinder/volume/rpcapi.py", line 248, in terminate_connection
2017-03-20 13:15:08.103 4435 ERROR oslo_messaging.rpc.dispatcher connector=connector, force=force)
2017-03-20 13:15:08.103 4435 ERROR oslo_messaging.rpc.dispatcher File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/client.py", line 158, in call
2017-03-20 13:15:08.103 4435 ERROR oslo_messaging.rpc.dispatcher retry=self.retry)
2017-03-20 13:15:08.103 4435 ERROR oslo_messaging.rpc.dispatcher File "/usr/lib/python2.7/site-packages/oslo_messaging/transport.py", line 90, in _send
2017-03-20 13:15:08.103 4435 ERROR oslo_messaging.rpc.dispatcher timeout=timeout, retry=retry)
2017-03-20 13:15:08.103 4435 ERROR oslo_messaging.rpc.dispatcher File "/usr/lib/python2.7/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 470, in send
2017-03-20 13:15:08.103 4435 ERROR oslo_messaging.rpc.dispatcher retry=retry)
2017-03-20 13:15:08.103 4435 ERROR oslo_messaging.rpc.dispatcher File "/usr/lib/python2.7/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 459, in _send
2017-03-20 13:15:08.103 4435 ERROR oslo_messaging.rpc.dispatcher result = self._waiter.wait(msg_id, timeout)
2017-03-20 13:15:08.103 4435 ERROR oslo_messaging.rpc.dispatcher File "/usr/lib/python2.7/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 342, in wait
2017-03-20 13:15:08.103 4435 ERROR oslo_messaging.rpc.dispatcher message = self.waiters.get(msg_id, timeout=timeout)
2017-03-20 13:15:08.103 4435 ERROR oslo_messaging.rpc.dispatcher File "/usr/lib/python2.7/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 244, in get
2017-03-20 13:15:08.103 4435 ERROR oslo_messaging.rpc.dispatcher 'to message ID %s' % msg_id)
2017-03-20 13:15:08.103 4435 ERROR oslo_messaging.rpc.dispatcher MessagingTimeout: Timed out waiting for a reply to message ID 69113807c44e41e6914d01acb99d40b6
2017-03-20 13:15:08.103 4435 ERROR oslo_messaging.rpc.dispatcher
2017-03-20 13:15:12.464 4436 INFO cinder.volume.drivers.hitachi.hbsd.hbsd_traceutils [req-ce0fc17b-0c9d-46e9-82b3-722de16ca19a 8c1aa3f10dfb47f2b061c3674be85515 eb67a97587234f7790070d0a58032ba7 - - -] MSGID0002-I: The method delete_volume completed successfully. (config_group: Tier2)
Environment
- Red Hat OpenStack Platform 9.0
- Hitachi configured as cinder backend through FC
Subscriber exclusive content
A Red Hat subscription provides unlimited access to our knowledgebase, tools, and much more.