When attaching a FC device, nova-compute will assign /dev/sd* even though there are 4 paths and a DM device

Solution Unverified - Updated -

Issue

  • When attaching a FC device, nova-compute will assign only one /dev/sd* device even though there are 4 paths (or more) and a DM-* device . Shouldn't nova-compute leverage using a dm-* device instead of one of the underlying device ? If that path fails, then the VM will lose the volume as if it was using dm-* instead, it would simply failover one of the 3 remaining paths. When the volume is detached, it's only deleting one of the device instead of all device.
Before attaching a cinder volume to an instance running on compute5:
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
[root@overcloud-compute-5 by-path]# pwd
/dev/disk/by-path

[root@overcloud-compute-5 by-path]# ls -al
total 0
drwxr-xr-x. 2 root root 200 Dec  1 11:24 .
drwxr-xr-x. 6 root root 120 Dec  1 07:00 ..
lrwxrwxrwx. 1 root root   9 Dec  1 07:00 pci-0000:01:00.0-scsi-0:0:2:0 -> ../../sda
lrwxrwxrwx. 1 root root  10 Dec  1 07:00 pci-0000:01:00.0-scsi-0:0:2:0-part1 -> ../../sda1
lrwxrwxrwx. 1 root root  10 Dec  1 07:00 pci-0000:01:00.0-scsi-0:0:2:0-part2 -> ../../sda2
lrwxrwxrwx. 1 root root   9 Dec  1 07:00 pci-0000:01:00.0-scsi-0:0:3:0 -> ../../sdb
lrwxrwxrwx. 1 root root   9 Dec  1 11:24 pci-0000:0b:00.0-fc-0x500601643ea062e9-lun-0 -> ../../sde
lrwxrwxrwx. 1 root root   9 Dec  1 11:24 pci-0000:0b:00.0-fc-0x5006016c3ea062e9-lun-0 -> ../../sdf
lrwxrwxrwx. 1 root root   9 Dec  1 11:06 pci-0000:0c:00.0-fc-0x500601673ea062e9-lun-0 -> ../../sdd
lrwxrwxrwx. 1 root root   9 Dec  1 11:06 pci-0000:0c:00.0-fc-0x5006016f3ea062e9-lun-0 -> ../../sdc


++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++


After attaching a cinder volume to an instance running on compute5 we see 4 more disks. (We have four paths to our storage, and hence for every volume created we see 4 devices)
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
[root@overcloud-compute-5 by-path]# pwd
/dev/disk/by-path
[root@overcloud-compute-5 by-path]# ls -al
total 0
drwxr-xr-x. 2 root root 280 Dec  1 12:10 .
drwxr-xr-x. 6 root root 120 Dec  1 07:00 ..
lrwxrwxrwx. 1 root root   9 Dec  1 07:00 pci-0000:01:00.0-scsi-0:0:2:0 -> ../../sda
lrwxrwxrwx. 1 root root  10 Dec  1 07:00 pci-0000:01:00.0-scsi-0:0:2:0-part1 -> ../../sda1
lrwxrwxrwx. 1 root root  10 Dec  1 07:00 pci-0000:01:00.0-scsi-0:0:2:0-part2 -> ../../sda2
lrwxrwxrwx. 1 root root   9 Dec  1 07:00 pci-0000:01:00.0-scsi-0:0:3:0 -> ../../sdb
lrwxrwxrwx. 1 root root   9 Dec  1 11:24 pci-0000:0b:00.0-fc-0x500601643ea062e9-lun-0 -> ../../sde
lrwxrwxrwx. 1 root root   9 Dec  1 12:10 pci-0000:0b:00.0-fc-0x500601643ea062e9-lun-111 -> ../../sdg
lrwxrwxrwx. 1 root root   9 Dec  1 11:24 pci-0000:0b:00.0-fc-0x5006016c3ea062e9-lun-0 -> ../../sdf
lrwxrwxrwx. 1 root root   9 Dec  1 12:10 pci-0000:0b:00.0-fc-0x5006016c3ea062e9-lun-111 -> ../../sdh
lrwxrwxrwx. 1 root root   9 Dec  1 11:06 pci-0000:0c:00.0-fc-0x500601673ea062e9-lun-0 -> ../../sdd
lrwxrwxrwx. 1 root root   9 Dec  1 12:10 pci-0000:0c:00.0-fc-0x500601673ea062e9-lun-111 -> ../../sdj
lrwxrwxrwx. 1 root root   9 Dec  1 11:06 pci-0000:0c:00.0-fc-0x5006016f3ea062e9-lun-0 -> ../../sdc
lrwxrwxrwx. 1 root root   9 Dec  1 12:10 pci-0000:0c:00.0-fc-0x5006016f3ea062e9-lun-111 -> ../../sdi
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++


If you notice, it created 4 new block device got created.
Now if I detach the disk from the instance, it only deletes one block device, in this case "sdg" (pci-0000:0b:00.0-fc-0x500601643ea062e9-lun-111 -> ../../sdg)
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
[root@overcloud-compute-5 by-path]# ls -al
total 0
drwxr-xr-x. 2 root root 260 Dec  1 12:18 .
drwxr-xr-x. 6 root root 120 Dec  1 07:00 ..
lrwxrwxrwx. 1 root root   9 Dec  1 07:00 pci-0000:01:00.0-scsi-0:0:2:0 -> ../../sda
lrwxrwxrwx. 1 root root  10 Dec  1 07:00 pci-0000:01:00.0-scsi-0:0:2:0-part1 -> ../../sda1
lrwxrwxrwx. 1 root root  10 Dec  1 07:00 pci-0000:01:00.0-scsi-0:0:2:0-part2 -> ../../sda2
lrwxrwxrwx. 1 root root   9 Dec  1 07:00 pci-0000:01:00.0-scsi-0:0:3:0 -> ../../sdb
lrwxrwxrwx. 1 root root   9 Dec  1 11:24 pci-0000:0b:00.0-fc-0x500601643ea062e9-lun-0 -> ../../sde
lrwxrwxrwx. 1 root root   9 Dec  1 11:24 pci-0000:0b:00.0-fc-0x5006016c3ea062e9-lun-0 -> ../../sdf
lrwxrwxrwx. 1 root root   9 Dec  1 12:10 pci-0000:0b:00.0-fc-0x5006016c3ea062e9-lun-111 -> ../../sdh
lrwxrwxrwx. 1 root root   9 Dec  1 11:06 pci-0000:0c:00.0-fc-0x500601673ea062e9-lun-0 -> ../../sdd
lrwxrwxrwx. 1 root root   9 Dec  1 12:10 pci-0000:0c:00.0-fc-0x500601673ea062e9-lun-111 -> ../../sdj
lrwxrwxrwx. 1 root root   9 Dec  1 11:06 pci-0000:0c:00.0-fc-0x5006016f3ea062e9-lun-0 -> ../../sdc
lrwxrwxrwx. 1 root root   9 Dec  1 12:10 pci-0000:0c:00.0-fc-0x5006016f3ea062e9-lun-111 -> ../../sdi
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++

Environment

  • Red Hat Open Stack

Subscriber exclusive content

A Red Hat subscription provides unlimited access to our knowledgebase, tools, and much more.

Current Customers and Partners

Log in for full access

Log In