What is the suggested I/O scheduler to improve disk performance when using Red Hat Enterprise Linux with virtualization?

Solution Verified - Updated -

Environment

  • Red Hat Enterprise Linux (RHEL) 4, 5, 6 or 7
  • Virtualization, e.g. KVM, Xen, VMware or Microsoft Hyper-V
  • Virtualization guest or virtualization host
  • Virtual disk

Issue

  • What is the recommended I/O scheduler for Red Hat Enterprise Linux as a virtualization host?

Resolution

Red Hat Enterprise Linux as a virtualization host (Xen, KVM or VMware)

  • When using RHEL as a host for virtualized guests, the default cfq scheduler is usually ideal. This scheduler performs well on nearly all workloads.
  • If, however, minimizing I/O latency is more important than maximizing I/O throughput on the guest workloads, it may be beneficial to use the deadline scheduler . The deadline is also the scheduler used by the tuned profile virtual-host.

Red Hat Enterprise Linux as a virtualization guest (any hypervisor technology)

  • RHEL guests often benefit greatly from the noop I/O scheduler, which allows the host/hypervisor to optimize the I/O requests. The noop scheduler can still combine small requests from the guest OS into larger requests before handing the I/O to the hypervisor, however noop follows the idea to spend as few CPU cycles as possible in the guest for I/O scheduling. The host/hypervisor will have an overview of the requests of all guests and have a separate strategy for handling I/O.
  • Depending on the workload I/O and how storage devices are attached, schedulers like deadline can be more advantageous. Performance testing is required to verify which scheduler is the most advantageous.
  • Guests using storage accessed by iSCSI, SR-IOV or physical device pass-through should not use the noop scheduler, since these methods do not allow the host to optimize I/O requests to the underlying physical device.

Root Cause

  • In virtualized environments, it is sometimes not beneficial to schedule I/O at both the host and guest layers. If multiple guests use storage on a filesystem or block device managed by the host operating system, the host may be able to schedule I/O more efficiently because it is aware of requests from all guests and knows the physical layout of storage, which may not map linearly to the guests' virtual storage. On the other hand, depending on the workload, it can also be beneficial to use a scheduler like deadline in the guest.

  • All scheduler tuning should be tested under normal operating conditions, as synthetic benchmarks typically do not accurately compare performance of systems using shared resources in virtual environments.

Configuring the I/O scheduler on Red Hat Enterprise Linux 4, 5 and 6

  • The I/O scheduler can be selected at boot time using the elevator kernel parameter. In the following example grub.conf stanza, the system has been configured to use the noop scheduler:

    title Red Hat Enterprise Linux Server (2.6.18-8.el5)
    root (hd0,0)
    kernel /vmlinuz-2.6.18-8.el5 ro root=/dev/vg0/lv0 elevator=noop
    initrd /initrd-2.6.18-8.el5.img
    

Configuring the I/O scheduler on Red Hat Enterprise Linux 7

  • To make the changes persistent through boot you have to add elevator=noop toGRUB_CMDLINE_LINUX in /etc/default/grub as shown below.

    # cat /etc/default/grub
    [...]
    GRUB_CMDLINE_LINUX="crashkernel=auto rd.lvm.lv=vg00/lvroot rhgb quiet elevator=noop"
    [...]
    
    • After the entry has been created/updated, rebuild the /boot/grub2/grub.cfg file to include the new configuration with the added parameter:
      • On BIOS-based machines: ~]# grub2-mkconfig -o /boot/grub2/grub.cfg
      • On UEFI-based machines: ~]# grub2-mkconfig -o /boot/efi/EFI/redhat/grub.cfg
  • Another way to change the default I/O scheduler is to use tuned.

Online configuring the I/O scheduler on Red Hat Enterprise Linux

  • In Red Hat Enterprise Linux 5, 6 or 7 it is also possible to change the I/O scheduler for a particular disk after the system has been booted. This makes it possible to use different I/O schedulers for different disks.

    # cat /sys/block/hda/queue/scheduler
    noop anticipatory deadline [cfq]
    
    # echo 'noop' > /sys/block/hda/queue/scheduler
    # cat /sys/block/hda/queue/scheduler
    [noop] anticipatory deadline cfq
    

This solution is part of Red Hat’s fast-track publication program, providing a huge library of solutions that Red Hat engineers have created while supporting our customers. To give you the knowledge you need the instant it becomes available, these articles may be presented in a raw and unedited form.

Close

Welcome! Check out the Getting Started with Red Hat page for quick tours and guides for common tasks.