live migration and instance creation fails with "NoFibreChannelVolumeDeviceFound: Unable to find a Fibre Channel volume device." in HP3Par cinder backend

Solution Verified - Updated -

Issue

  • live migration and instance creation fails with "NoFibreChannelVolumeDeviceFound: Unable to find a Fibre Channel volume device." in HP3Par cinder backend
2019-01-18 07:51:03.627 285661 WARNING os_brick.initiator.connectors.fibre_channel [-] Fibre Channel volume device not yet found. Will rescan & retry.  Try number: 0.
2019-01-18 07:51:05.625 285661 WARNING os_brick.initiator.connectors.fibre_channel [-] Fibre Channel volume device not yet found. Will rescan & retry.  Try number: 1.
2019-01-18 07:51:07.624 285661 WARNING os_brick.initiator.connectors.fibre_channel [-] Fibre Channel volume device not yet found. Will rescan & retry.  Try number: 2.
2019-01-18 07:51:09.622 285661 WARNING os_brick.initiator.connectors.fibre_channel [-] Fibre Channel volume device not yet found. Will rescan & retry.  Try number: 3.
2019-01-18 07:51:11.619 285661 WARNING os_brick.initiator.connectors.fibre_channel [-] Fibre Channel volume device not yet found. Will rescan & retry.  Try number: 4.
2019-01-18 07:51:13.619 285661 ERROR os_brick.initiator.connectors.fibre_channel [-] Fibre Channel volume device not found.
2019-01-18 07:51:13.619 285661 ERROR oslo.service.loopingcall [-] Fixed interval looping call 'os_brick.initiator.connectors.fibre_channel._wait_for_device_discovery' failed
2019-01-18 07:51:13.619 285661 ERROR oslo.service.loopingcall Traceback (most recent call last):
2019-01-18 07:51:13.619 285661 ERROR oslo.service.loopingcall   File "/usr/lib/python2.7/site-packages/oslo_service/loopingcall.py", line 136, in _run_loop
2019-01-18 07:51:13.619 285661 ERROR oslo.service.loopingcall     result = func(*self.args, **self.kw)
2019-01-18 07:51:13.619 285661 ERROR oslo.service.loopingcall   File "/usr/lib/python2.7/site-packages/os_brick/initiator/connectors/fibre_channel.py", line 157, in _wait_for_device_discovery
2019-01-18 07:51:13.619 285661 ERROR oslo.service.loopingcall     raise exception.NoFibreChannelVolumeDeviceFound()
2019-01-18 07:51:13.619 285661 ERROR oslo.service.loopingcall NoFibreChannelVolumeDeviceFound: Unable to find a Fibre Channel volume device.
2019-01-18 07:51:13.619 285661 ERROR oslo.service.loopingcall 
2019-01-18 07:51:13.655 285661 ERROR oslo_messaging.rpc.server [req-d0656815-46d3-4494-ae14-867cb826acd6 07330a460401434bb02fed5acb0e3d3c 1b6c62dd0a6a4f7c8917304292298cb0 - - -] Exception during message handling
2019-01-18 07:51:13.655 285661 ERROR oslo_messaging.rpc.server Traceback (most recent call last):
2019-01-18 07:51:13.655 285661 ERROR oslo_messaging.rpc.server   File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/server.py", line 133, in _process_incoming
2019-01-18 07:51:13.655 285661 ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
2019-01-18 07:51:13.655 285661 ERROR oslo_messaging.rpc.server   File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/dispatcher.py", line 150, in dispatch
2019-01-18 07:51:13.655 285661 ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
2019-01-18 07:51:13.655 285661 ERROR oslo_messaging.rpc.server   File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/dispatcher.py", line 121, in _do_dispatch
2019-01-18 07:51:13.655 285661 ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
2019-01-18 07:51:13.655 285661 ERROR oslo_messaging.rpc.server   File "/usr/lib/python2.7/site-packages/nova/exception_wrapper.py", line 75, in wrapped
2019-01-18 07:51:13.655 285661 ERROR oslo_messaging.rpc.server     function_name, call_dict, binary)
2019-01-18 07:51:13.655 285661 ERROR oslo_messaging.rpc.server   File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__
2019-01-18 07:51:13.655 285661 ERROR oslo_messaging.rpc.server     self.force_reraise()
2019-01-18 07:51:13.655 285661 ERROR oslo_messaging.rpc.server   File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise
2019-01-18 07:51:13.655 285661 ERROR oslo_messaging.rpc.server     six.reraise(self.type_, self.value, self.tb)
2019-01-18 07:51:13.655 285661 ERROR oslo_messaging.rpc.server   File "/usr/lib/python2.7/site-packages/nova/exception_wrapper.py", line 66, in wrapped
2019-01-18 07:51:13.655 285661 ERROR oslo_messaging.rpc.server     return f(self, context, *args, **kw)
2019-01-18 07:51:13.655 285661 ERROR oslo_messaging.rpc.server   File "/usr/lib/python2.7/site-packages/nova/compute/utils.py", line 613, in decorated_function
2019-01-18 07:51:13.655 285661 ERROR oslo_messaging.rpc.server     return function(self, context, *args, **kwargs)
2019-01-18 07:51:13.655 285661 ERROR oslo_messaging.rpc.server   File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 216, in decorated_function
2019-01-18 07:51:13.655 285661 ERROR oslo_messaging.rpc.server     kwargs['instance'], e, sys.exc_info())
2019-01-18 07:51:13.655 285661 ERROR oslo_messaging.rpc.server   File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__
2019-01-18 07:51:13.655 285661 ERROR oslo_messaging.rpc.server     self.force_reraise()
2019-01-18 07:51:13.655 285661 ERROR oslo_messaging.rpc.server   File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise
2019-01-18 07:51:13.655 285661 ERROR oslo_messaging.rpc.server     six.reraise(self.type_, self.value, self.tb)
2019-01-18 07:51:13.655 285661 ERROR oslo_messaging.rpc.server   File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 204, in decorated_function
2019-01-18 07:51:13.655 285661 ERROR oslo_messaging.rpc.server     return function(self, context, *args, **kwargs)
2019-01-18 07:51:13.655 285661 ERROR oslo_messaging.rpc.server   File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 5230, in pre_live_migration
2019-01-18 07:51:13.655 285661 ERROR oslo_messaging.rpc.server     migrate_data)
2019-01-18 07:51:13.655 285661 ERROR oslo_messaging.rpc.server   File "/usr/lib/python2.7/site-packages/nova/virt/libvirt/driver.py", line 6807, in pre_live_migration
2019-01-18 07:51:13.655 285661 ERROR oslo_messaging.rpc.server     self._connect_volume(connection_info, disk_info, instance)
2019-01-18 07:51:13.655 285661 ERROR oslo_messaging.rpc.server   File "/usr/lib/python2.7/site-packages/nova/virt/libvirt/driver.py", line 1122, in _connect_volume
2019-01-18 07:51:13.655 285661 ERROR oslo_messaging.rpc.server     vol_driver.connect_volume(connection_info, disk_info, instance)
2019-01-18 07:51:13.655 285661 ERROR oslo_messaging.rpc.server   File "/usr/lib/python2.7/site-packages/nova/virt/libvirt/volume/fibrechannel.py", line 53, in connect_volume
2019-01-18 07:51:13.655 285661 ERROR oslo_messaging.rpc.server     device_info = self.connector.connect_volume(connection_info['data'])
2019-01-18 07:51:13.655 285661 ERROR oslo_messaging.rpc.server   File "/usr/lib/python2.7/site-packages/os_brick/utils.py", line 137, in trace_logging_wrapper
2019-01-18 07:51:13.655 285661 ERROR oslo_messaging.rpc.server     return f(*args, **kwargs)
2019-01-18 07:51:13.655 285661 ERROR oslo_messaging.rpc.server   File "/usr/lib/python2.7/site-packages/oslo_concurrency/lockutils.py", line 271, in inner
2019-01-18 07:51:13.655 285661 ERROR oslo_messaging.rpc.server     return f(*args, **kwargs)
2019-01-18 07:51:13.655 285661 ERROR oslo_messaging.rpc.server   File "/usr/lib/python2.7/site-packages/os_brick/initiator/connectors/fibre_channel.py", line 171, in connect_volume
2019-01-18 07:51:13.655 285661 ERROR oslo_messaging.rpc.server     timer.start(interval=2).wait()
2019-01-18 07:51:13.655 285661 ERROR oslo_messaging.rpc.server   File "/usr/lib/python2.7/site-packages/eventlet/event.py", line 121, in wait
2019-01-18 07:51:13.655 285661 ERROR oslo_messaging.rpc.server     return hubs.get_hub().switch()
2019-01-18 07:51:13.655 285661 ERROR oslo_messaging.rpc.server   File "/usr/lib/python2.7/site-packages/eventlet/hubs/hub.py", line 294, in switch
2019-01-18 07:51:13.655 285661 ERROR oslo_messaging.rpc.server     return self.greenlet.switch()
2019-01-18 07:51:13.655 285661 ERROR oslo_messaging.rpc.server   File "/usr/lib/python2.7/site-packages/oslo_service/loopingcall.py", line 136, in _run_loop
2019-01-18 07:51:13.655 285661 ERROR oslo_messaging.rpc.server     result = func(*self.args, **self.kw)
2019-01-18 07:51:13.655 285661 ERROR oslo_messaging.rpc.server   File "/usr/lib/python2.7/site-packages/os_brick/initiator/connectors/fibre_channel.py", line 157, in _wait_for_device_discovery
2019-01-18 07:51:13.655 285661 ERROR oslo_messaging.rpc.server     raise exception.NoFibreChannelVolumeDeviceFound()
2019-01-18 07:51:13.655 285661 ERROR oslo_messaging.rpc.server NoFibreChannelVolumeDeviceFound: Unable to find a Fibre Channel volume device.
2019-01-18 07:51:13.655 285661 ERROR oslo_messaging.rpc.server 

Environment

  • Red Hat OpenStack Platform 10
  • cinder backend HP3Par

Subscriber exclusive content

A Red Hat subscription provides unlimited access to our knowledgebase, tools, and much more.

Current Customers and Partners

Log in for full access

Log In

New to Red Hat?

Learn more about Red Hat subscriptions

Using a Red Hat product through a public cloud?

How to access this content