How to ensure all SCSI disks attached to VM use the virtio-scsi driver?
Issue
- Currently it is possible to boot an instance with a
virtio-scsi
controller. This can be achieved by settinghw_scsi_model=virtio-scsi
via
glance image-update --property hw_scsi_model=virtio-scsi <image>
- This will then add the following XML to the instance being booted from
<image>
:
<controller type='scsi' index='0' model='virtio-scsi'>
<alias name='scsi0'/>
<address type='pci' domain='0x0000' bus='0x00' slot='0x04' function='0x0'/>
</controller>
- Now to get the disks to use SCSI we also set
glance image-update --property hw_disk_bus=scsi <image>
- This leads to the main disk for the vm to be attached to the above SCSI controller:
<disk type='file' device='disk'>
<driver name='qemu' type='qcow2' cache='none'/>
<source file='/var/lib/nova/instances/37b3ef61-093c-4a02-a4ac-c2ee33c7b59b/disk'/>
<backingStore type='file' index='1'>
<format type='raw'/>
<source file='/var/lib/nova/instances/_base/88bb4625907e2d39d2832357ce522c06b55d4c76'/>
<backingStore/>
</backingStore>
<target dev='sda' bus='scsi'/>
<alias name='scsi0-0-0-0'/>
<address type='drive' controller='0' bus='0' target='0' unit='0'/>
</disk>
- Also when attaching a further disk via
nova volume-attach <instance> <volume-id>
this also gets attached to the same controller:
<disk type='block' device='disk'>
<driver name='qemu' type='raw' cache='none'/>
<source dev='/dev/disk/by-id/emc-vol-6de5eed267d5d8ec-b62443c100000000'/>
<backingStore/>
<target dev='sdb' bus='scsi'/>
<serial>003fe9f2-f83e-4d10-8869-6ddb8e9dd6ac</serial>
<alias name='scsi0-0-0-1'/>
<address type='drive' controller='0' bus='0' target='0' unit='1'/>
</disk>
- This works for the first 6 disks. When attaching the 7th disk, a new SCSI controller is attached:
<controller type='scsi' index='1'>
<alias name='scsi1'/>
<address type='pci' domain='0x0000' bus='0x00' slot='0x06' function='0x0'/>
</controller>
- And the disk will be attached to this controller:
<disk type='block' device='disk'>
<driver name='qemu' type='raw' cache='none'/>
<source dev='/dev/disk/by-id/emc-vol-6de5eed267d5d8ec-b62443c800000007'/>
<backingStore/>
<target dev='sdh' bus='scsi'/>
<serial>9f9251fc-6264-4e08-9ab8-4ae40d6ae4ba</serial>
<alias name='scsi1-0-0-0'/>
<address type='drive' controller='1' bus='0' target='0' unit='0'/>
</disk>
-
This repeats again for the 13th disk and so on.
-
This behavior has the following 2 issues:
- Only the first SCSI controller is of type
virtio-scsi
, the subsequent controllers do not use thevirtio-scsi
driver and thus exhibit worse disk performance. - As the PCI slots available inside of the instance are limited to 32 and a few of those are used to attach other devices like USB, Ethernet, Video and the Memory Baloon, only around 26 PCI slots are available, effectively limiting the amount of disks per instance to 6 * 26
- Is it possible to ensure more (all) disks use the first SCSI controller?
- If not, is it possible to ensure all SCSI controllers use the virtio-scsi driver?
- How do we configure this?
Environment
- Red Hat Open Stack Platform 12.
- Red Hat Open Stack Platform 11.
- Red Hat Open Stack Platform 10.
- Red Hat Open Stack Platform 9.
- Red Hat Open Stack Platform 8.
- Red Hat Enterprise Linux OpenStack Platform 7.
- Red Hat Enterprise Linux OpenStack Platform 6.
Subscriber exclusive content
A Red Hat subscription provides unlimited access to our knowledgebase, tools, and much more.