Unable to attach Cinder Volume to the instance.

Solution Verified - Updated -

Issue

  • After the creation of the Cinder volume from the OpenStack dashboard, the nova-volume attachment operation fails, while it work with the classic cinder volume. The error seems to be related to the communication of the nova process with the libvirt subsystem, but we can't figure out what is the specific problem.
  • From nova-compute.log:
2016-12-21 16:10:47.391 21637 INFO nova.compute.manager [req-0d4b0a6e-6bdf-4aa7-89b0-5734066e245e b0f1f05bf426480ea6592a5bc7dd35e5 b318bdbc48444f9281778316ebc83c84 - - -] [instance: dad77ab3-d879-4e32-9e8e-12458a821cf3] Attaching volume 16178f0e-c8ac-4274-b9b6-e69542b2873f to /dev/vdc
2016-12-21 16:10:48.123 21637 ERROR nova.virt.libvirt.driver [req-0d4b0a6e-6bdf-4aa7-89b0-5734066e245e b0f1f05bf426480ea6592a5bc7dd35e5 b318bdbc48444f9281778316ebc83c84 - - -] [instance: dad77ab3-d879-4e32-9e8e-12458a821cf3] Failed to attach volume at mountpoint: /dev/vdc
2016-12-21 16:10:48.123 21637 TRACE nova.virt.libvirt.driver [instance: dad77ab3-d879-4e32-9e8e-12458a821cf3] Traceback (most recent call last):
2016-12-21 16:10:48.123 21637 TRACE nova.virt.libvirt.driver [instance: dad77ab3-d879-4e32-9e8e-12458a821cf3]   File "/usr/lib/python2.7/site-packages/nova/virt/libvirt/driver.py", line 1133, in attach_volume
2016-12-21 16:10:48.123 21637 TRACE nova.virt.libvirt.driver [instance: dad77ab3-d879-4e32-9e8e-12458a821cf3]     virt_dom.attachDeviceFlags(conf.to_xml(), flags)
2016-12-21 16:10:48.123 21637 TRACE nova.virt.libvirt.driver [instance: dad77ab3-d879-4e32-9e8e-12458a821cf3]   File "/usr/lib/python2.7/site-packages/nova/virt/libvirt/config.py", line 80, in to_xml
2016-12-21 16:10:48.123 21637 TRACE nova.virt.libvirt.driver [instance: dad77ab3-d879-4e32-9e8e-12458a821cf3]     root = self.format_dom()
2016-12-21 16:10:48.123 21637 TRACE nova.virt.libvirt.driver [instance: dad77ab3-d879-4e32-9e8e-12458a821cf3]   File "/usr/lib/python2.7/site-packages/nova/virt/libvirt/config.py", line 779, in format_dom
2016-12-21 16:10:48.123 21637 TRACE nova.virt.libvirt.driver [instance: dad77ab3-d879-4e32-9e8e-12458a821cf3]     uuid=self.auth_secret_uuid))
2016-12-21 16:10:48.123 21637 TRACE nova.virt.libvirt.driver [instance: dad77ab3-d879-4e32-9e8e-12458a821cf3]   File "lxml.etree.pyx", line 2808, in lxml.etree.Element (src/lxml/lxml.etree.c:66342)
2016-12-21 16:10:48.123 21637 TRACE nova.virt.libvirt.driver [instance: dad77ab3-d879-4e32-9e8e-12458a821cf3]   File "apihelpers.pxi", line 129, in lxml.etree._makeElement (src/lxml/lxml.etree.c:14430)
2016-12-21 16:10:48.123 21637 TRACE nova.virt.libvirt.driver [instance: dad77ab3-d879-4e32-9e8e-12458a821cf3]   File "apihelpers.pxi", line 117, in lxml.etree._makeElement (src/lxml/lxml.etree.c:14313)
2016-12-21 16:10:48.123 21637 TRACE nova.virt.libvirt.driver [instance: dad77ab3-d879-4e32-9e8e-12458a821cf3]   File "apihelpers.pxi", line 264, in lxml.etree._initNodeAttributes (src/lxml/lxml.etree.c:15878)
2016-12-21 16:10:48.123 21637 TRACE nova.virt.libvirt.driver [instance: dad77ab3-d879-4e32-9e8e-12458a821cf3]   File "apihelpers.pxi", line 286, in lxml.etree._addAttributeToNode (src/lxml/lxml.etree.c:16347)
2016-12-21 16:10:48.123 21637 TRACE nova.virt.libvirt.driver [instance: dad77ab3-d879-4e32-9e8e-12458a821cf3]   File "apihelpers.pxi", line 1356, in lxml.etree._utf8 (src/lxml/lxml.etree.c:26059)
2016-12-21 16:10:48.123 21637 TRACE nova.virt.libvirt.driver [instance: dad77ab3-d879-4e32-9e8e-12458a821cf3] TypeError: Argument must be bytes or unicode, got 'NoneType'
2016-12-21 16:10:48.123 21637 TRACE nova.virt.libvirt.driver [instance: dad77ab3-d879-4e32-9e8e-12458a821cf3] 
2016-12-21 16:10:48.179 21637 ERROR nova.virt.block_device [req-0d4b0a6e-6bdf-4aa7-89b0-5734066e245e b0f1f05bf426480ea6592a5bc7dd35e5 b318bdbc48444f9281778316ebc83c84 - - -] [instance: dad77ab3-d879-4e32-9e8e-12458a821cf3] Driver failed to attach volume 16178f0e-c8ac-4274-b9b6-e69542b2873f at /dev/vdc
2016-12-21 16:10:48.179 21637 TRACE nova.virt.block_device [instance: dad77ab3-d879-4e32-9e8e-12458a821cf3] Traceback (most recent call last):
2016-12-21 16:10:48.179 21637 TRACE nova.virt.block_device [instance: dad77ab3-d879-4e32-9e8e-12458a821cf3]   File "/usr/lib/python2.7/site-packages/nova/virt/block_device.py", line 287, in attach
2016-12-21 16:10:48.179 21637 TRACE nova.virt.block_device [instance: dad77ab3-d879-4e32-9e8e-12458a821cf3]     device_type=self['device_type'], encryption=encryption)
2016-12-21 16:10:48.179 21637 TRACE nova.virt.block_device [instance: dad77ab3-d879-4e32-9e8e-12458a821cf3]   File "/usr/lib/python2.7/site-packages/nova/virt/libvirt/driver.py", line 1144, in attach_volume
2016-12-21 16:10:48.179 21637 TRACE nova.virt.block_device [instance: dad77ab3-d879-4e32-9e8e-12458a821cf3]     self._disconnect_volume(connection_info, disk_dev)
2016-12-21 16:10:48.179 21637 TRACE nova.virt.block_device [instance: dad77ab3-d879-4e32-9e8e-12458a821cf3]   File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 85, in __exit__
2016-12-21 16:10:48.179 21637 TRACE nova.virt.block_device [instance: dad77ab3-d879-4e32-9e8e-12458a821cf3]     six.reraise(self.type_, self.value, self.tb)
2016-12-21 16:10:48.179 21637 TRACE nova.virt.block_device [instance: dad77ab3-d879-4e32-9e8e-12458a821cf3]   File "/usr/lib/python2.7/site-packages/nova/virt/libvirt/driver.py", line 1133, in attach_volume
2016-12-21 16:10:48.179 21637 TRACE nova.virt.block_device [instance: dad77ab3-d879-4e32-9e8e-12458a821cf3]     virt_dom.attachDeviceFlags(conf.to_xml(), flags)
2016-12-21 16:10:48.179 21637 TRACE nova.virt.block_device [instance: dad77ab3-d879-4e32-9e8e-12458a821cf3]   File "/usr/lib/python2.7/site-packages/nova/virt/libvirt/config.py", line 80, in to_xml
2016-12-21 16:10:48.179 21637 TRACE nova.virt.block_device [instance: dad77ab3-d879-4e32-9e8e-12458a821cf3]     root = self.format_dom()
2016-12-21 16:10:48.179 21637 TRACE nova.virt.block_device [instance: dad77ab3-d879-4e32-9e8e-12458a821cf3]   File "/usr/lib/python2.7/site-packages/nova/virt/libvirt/config.py", line 779, in format_dom
2016-12-21 16:10:48.179 21637 TRACE nova.virt.block_device [instance: dad77ab3-d879-4e32-9e8e-12458a821cf3]     uuid=self.auth_secret_uuid))
2016-12-21 16:10:48.179 21637 TRACE nova.virt.block_device [instance: dad77ab3-d879-4e32-9e8e-12458a821cf3]   File "lxml.etree.pyx", line 2808, in lxml.etree.Element (src/lxml/lxml.etree.c:66342)
2016-12-21 16:10:48.179 21637 TRACE nova.virt.block_device [instance: dad77ab3-d879-4e32-9e8e-12458a821cf3]   File "apihelpers.pxi", line 129, in lxml.etree._makeElement (src/lxml/lxml.etree.c:14430)
2016-12-21 16:10:48.179 21637 TRACE nova.virt.block_device [instance: dad77ab3-d879-4e32-9e8e-12458a821cf3]   File "apihelpers.pxi", line 117, in lxml.etree._makeElement (src/lxml/lxml.etree.c:14313)
2016-12-21 16:10:48.179 21637 TRACE nova.virt.block_device [instance: dad77ab3-d879-4e32-9e8e-12458a821cf3]   File "apihelpers.pxi", line 264, in lxml.etree._initNodeAttributes (src/lxml/lxml.etree.c:15878)
2016-12-21 16:10:48.179 21637 TRACE nova.virt.block_device [instance: dad77ab3-d879-4e32-9e8e-12458a821cf3]   File "apihelpers.pxi", line 286, in lxml.etree._addAttributeToNode (src/lxml/lxml.etree.c:16347)
2016-12-21 16:10:48.179 21637 TRACE nova.virt.block_device [instance: dad77ab3-d879-4e32-9e8e-12458a821cf3]   File "apihelpers.pxi", line 1356, in lxml.etree._utf8 (src/lxml/lxml.etree.c:26059)
2016-12-21 16:10:48.179 21637 TRACE nova.virt.block_device [instance: dad77ab3-d879-4e32-9e8e-12458a821cf3] TypeError: Argument must be bytes or unicode, got 'NoneType'
2016-12-21 16:10:48.179 21637 TRACE nova.virt.block_device [instance: dad77ab3-d879-4e32-9e8e-12458a821cf3] 

Environment

  • Red Hat Open Stack
  • CEPH backend for Cinder

Subscriber exclusive content

A Red Hat subscription provides unlimited access to our knowledgebase, tools, and much more.

Current Customers and Partners

Log in for full access

Log In

New to Red Hat?

Learn more about Red Hat subscriptions

Using a Red Hat product through a public cloud?

How to access this content