When attaching a FC device, nova-compute will assign /dev/sd* even though there are 4 paths and a DM device

Solution Unverified - Updated -

Issue

  • When attaching a FC device, nova-compute will assign only one /dev/sd* device even though there are 4 paths (or more) and a DM-* device . Shouldn't nova-compute leverage using a dm-* device instead of one of the underlying device ? If that path fails, then the VM will lose the volume as if it was using dm-* instead, it would simply failover one of the 3 remaining paths. When the volume is detached, it's only deleting one of the device instead of all device.
Before attaching a cinder volume to an instance running on compute5:
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
[root@overcloud-compute-5 by-path]# pwd
/dev/disk/by-path

[root@overcloud-compute-5 by-path]# ls -al
total 0
drwxr-xr-x. 2 root root 200 Dec  1 11:24 .
drwxr-xr-x. 6 root root 120 Dec  1 07:00 ..
lrwxrwxrwx. 1 root root   9 Dec  1 07:00 pci-0000:01:00.0-scsi-0:0:2:0 -> ../../sda
lrwxrwxrwx. 1 root root  10 Dec  1 07:00 pci-0000:01:00.0-scsi-0:0:2:0-part1 -> ../../sda1
lrwxrwxrwx. 1 root root  10 Dec  1 07:00 pci-0000:01:00.0-scsi-0:0:2:0-part2 -> ../../sda2
lrwxrwxrwx. 1 root root   9 Dec  1 07:00 pci-0000:01:00.0-scsi-0:0:3:0 -> ../../sdb
lrwxrwxrwx. 1 root root   9 Dec  1 11:24 pci-0000:0b:00.0-fc-0x500601643ea062e9-lun-0 -> ../../sde
lrwxrwxrwx. 1 root root   9 Dec  1 11:24 pci-0000:0b:00.0-fc-0x5006016c3ea062e9-lun-0 -> ../../sdf
lrwxrwxrwx. 1 root root   9 Dec  1 11:06 pci-0000:0c:00.0-fc-0x500601673ea062e9-lun-0 -> ../../sdd
lrwxrwxrwx. 1 root root   9 Dec  1 11:06 pci-0000:0c:00.0-fc-0x5006016f3ea062e9-lun-0 -> ../../sdc


++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++


After attaching a cinder volume to an instance running on compute5 we see 4 more disks. (We have four paths to our storage, and hence for every volume created we see 4 devices)
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
[root@overcloud-compute-5 by-path]# pwd
/dev/disk/by-path
[root@overcloud-compute-5 by-path]# ls -al
total 0
drwxr-xr-x. 2 root root 280 Dec  1 12:10 .
drwxr-xr-x. 6 root root 120 Dec  1 07:00 ..
lrwxrwxrwx. 1 root root   9 Dec  1 07:00 pci-0000:01:00.0-scsi-0:0:2:0 -> ../../sda
lrwxrwxrwx. 1 root root  10 Dec  1 07:00 pci-0000:01:00.0-scsi-0:0:2:0-part1 -> ../../sda1
lrwxrwxrwx. 1 root root  10 Dec  1 07:00 pci-0000:01:00.0-scsi-0:0:2:0-part2 -> ../../sda2
lrwxrwxrwx. 1 root root   9 Dec  1 07:00 pci-0000:01:00.0-scsi-0:0:3:0 -> ../../sdb
lrwxrwxrwx. 1 root root   9 Dec  1 11:24 pci-0000:0b:00.0-fc-0x500601643ea062e9-lun-0 -> ../../sde
lrwxrwxrwx. 1 root root   9 Dec  1 12:10 pci-0000:0b:00.0-fc-0x500601643ea062e9-lun-111 -> ../../sdg
lrwxrwxrwx. 1 root root   9 Dec  1 11:24 pci-0000:0b:00.0-fc-0x5006016c3ea062e9-lun-0 -> ../../sdf
lrwxrwxrwx. 1 root root   9 Dec  1 12:10 pci-0000:0b:00.0-fc-0x5006016c3ea062e9-lun-111 -> ../../sdh
lrwxrwxrwx. 1 root root   9 Dec  1 11:06 pci-0000:0c:00.0-fc-0x500601673ea062e9-lun-0 -> ../../sdd
lrwxrwxrwx. 1 root root   9 Dec  1 12:10 pci-0000:0c:00.0-fc-0x500601673ea062e9-lun-111 -> ../../sdj
lrwxrwxrwx. 1 root root   9 Dec  1 11:06 pci-0000:0c:00.0-fc-0x5006016f3ea062e9-lun-0 -> ../../sdc
lrwxrwxrwx. 1 root root   9 Dec  1 12:10 pci-0000:0c:00.0-fc-0x5006016f3ea062e9-lun-111 -> ../../sdi
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++


If you notice, it created 4 new block device got created.
Now if I detach the disk from the instance, it only deletes one block device, in this case "sdg" (pci-0000:0b:00.0-fc-0x500601643ea062e9-lun-111 -> ../../sdg)
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
[root@overcloud-compute-5 by-path]# ls -al
total 0
drwxr-xr-x. 2 root root 260 Dec  1 12:18 .
drwxr-xr-x. 6 root root 120 Dec  1 07:00 ..
lrwxrwxrwx. 1 root root   9 Dec  1 07:00 pci-0000:01:00.0-scsi-0:0:2:0 -> ../../sda
lrwxrwxrwx. 1 root root  10 Dec  1 07:00 pci-0000:01:00.0-scsi-0:0:2:0-part1 -> ../../sda1
lrwxrwxrwx. 1 root root  10 Dec  1 07:00 pci-0000:01:00.0-scsi-0:0:2:0-part2 -> ../../sda2
lrwxrwxrwx. 1 root root   9 Dec  1 07:00 pci-0000:01:00.0-scsi-0:0:3:0 -> ../../sdb
lrwxrwxrwx. 1 root root   9 Dec  1 11:24 pci-0000:0b:00.0-fc-0x500601643ea062e9-lun-0 -> ../../sde
lrwxrwxrwx. 1 root root   9 Dec  1 11:24 pci-0000:0b:00.0-fc-0x5006016c3ea062e9-lun-0 -> ../../sdf
lrwxrwxrwx. 1 root root   9 Dec  1 12:10 pci-0000:0b:00.0-fc-0x5006016c3ea062e9-lun-111 -> ../../sdh
lrwxrwxrwx. 1 root root   9 Dec  1 11:06 pci-0000:0c:00.0-fc-0x500601673ea062e9-lun-0 -> ../../sdd
lrwxrwxrwx. 1 root root   9 Dec  1 12:10 pci-0000:0c:00.0-fc-0x500601673ea062e9-lun-111 -> ../../sdj
lrwxrwxrwx. 1 root root   9 Dec  1 11:06 pci-0000:0c:00.0-fc-0x5006016f3ea062e9-lun-0 -> ../../sdc
lrwxrwxrwx. 1 root root   9 Dec  1 12:10 pci-0000:0c:00.0-fc-0x5006016f3ea062e9-lun-111 -> ../../sdi
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++

Environment

  • Red Hat Open Stack

Subscriber exclusive content

A Red Hat subscription provides unlimited access to our knowledgebase, tools, and much more.

Current Customers and Partners

Log in for full access

Log In

New to Red Hat?

Learn more about Red Hat subscriptions

Using a Red Hat product through a public cloud?

How to access this content