How to ensure all SCSI disks attached to VM use the virtio-scsi driver?

Solution Verified - Updated -

Issue

  • Currently it is possible to boot an instance with a virtio-scsi controller. This can be achieved by setting hw_scsi_model=virtio-scsi via
glance image-update --property hw_scsi_model=virtio-scsi <image>
  • This will then add the following XML to the instance being booted from <image>:
    <controller type='scsi' index='0' model='virtio-scsi'>
      <alias name='scsi0'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x04' function='0x0'/>
    </controller>
  • Now to get the disks to use SCSI we also set
glance image-update --property hw_disk_bus=scsi <image>
  • This leads to the main disk for the vm to be attached to the above SCSI controller:
    <disk type='file' device='disk'>
      <driver name='qemu' type='qcow2' cache='none'/>
      <source file='/var/lib/nova/instances/37b3ef61-093c-4a02-a4ac-c2ee33c7b59b/disk'/>
      <backingStore type='file' index='1'>
        <format type='raw'/>
        <source file='/var/lib/nova/instances/_base/88bb4625907e2d39d2832357ce522c06b55d4c76'/>
        <backingStore/>
      </backingStore>
      <target dev='sda' bus='scsi'/>
      <alias name='scsi0-0-0-0'/>
      <address type='drive' controller='0' bus='0' target='0' unit='0'/>
    </disk>
  • Also when attaching a further disk via nova volume-attach <instance> <volume-id> this also gets attached to the same controller:
    <disk type='block' device='disk'>
      <driver name='qemu' type='raw' cache='none'/>
      <source dev='/dev/disk/by-id/emc-vol-6de5eed267d5d8ec-b62443c100000000'/>
      <backingStore/>
      <target dev='sdb' bus='scsi'/>
      <serial>003fe9f2-f83e-4d10-8869-6ddb8e9dd6ac</serial>
      <alias name='scsi0-0-0-1'/>
      <address type='drive' controller='0' bus='0' target='0' unit='1'/>
    </disk>
  • This works for the first 6 disks. When attaching the 7th disk, a new SCSI controller is attached:
    <controller type='scsi' index='1'>
      <alias name='scsi1'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x06' function='0x0'/>
    </controller>
  • And the disk will be attached to this controller:
    <disk type='block' device='disk'>
      <driver name='qemu' type='raw' cache='none'/>
      <source dev='/dev/disk/by-id/emc-vol-6de5eed267d5d8ec-b62443c800000007'/>
      <backingStore/>
      <target dev='sdh' bus='scsi'/>
      <serial>9f9251fc-6264-4e08-9ab8-4ae40d6ae4ba</serial>
      <alias name='scsi1-0-0-0'/>
      <address type='drive' controller='1' bus='0' target='0' unit='0'/>
    </disk>
  • This repeats again for the 13th disk and so on.

  • This behavior has the following 2 issues:

  1. Only the first SCSI controller is of type virtio-scsi, the subsequent controllers do not use the virtio-scsi driver and thus exhibit worse disk performance.
  2. As the PCI slots available inside of the instance are limited to 32 and a few of those are used to attach other devices like USB, Ethernet, Video and the Memory Baloon, only around 26 PCI slots are available, effectively limiting the amount of disks per instance to 6 * 26
  • Is it possible to ensure more (all) disks use the first SCSI controller?
  • If not, is it possible to ensure all SCSI controllers use the virtio-scsi driver?
  • How do we configure this?

Environment

  • Red Hat Open Stack Platform 12.
  • Red Hat Open Stack Platform 11.
  • Red Hat Open Stack Platform 10.
  • Red Hat Open Stack Platform 9.
  • Red Hat Open Stack Platform 8.
  • Red Hat Enterprise Linux OpenStack Platform 7.
  • Red Hat Enterprise Linux OpenStack Platform 6.

Subscriber exclusive content

A Red Hat subscription provides unlimited access to our knowledgebase, tools, and much more.

Current Customers and Partners

Log in for full access

Log In

New to Red Hat?

Learn more about Red Hat subscriptions

Using a Red Hat product through a public cloud?

How to access this content