VMs on ESX/VSphere experience poor I/O performance after VM reboot
Issue
Sporadically, after a soft reboot (shutdown -r), some RHEL VMs show poor I/O performance relative to their normal function.
When VMs are in this state, they demonstrate significant I/O performance loss.
For example, as a baseline measure. The VM in a "good" state:
goodVM# /usr/local/fio-3.16/bin/fio --filename=/logdevices/testfile111.out --name fio_test_4k --direct=1 --rw=randwrite --bs=4k --size=64mb --numjobs=1 --iodepth=1 --time_based --ioengine=libaio --runtime=10 --group_reporting --norandommap | grep IOPS
write: IOPS=402, BW=1610KiB/s (1649kB/s)(15.7MiB/10001msec)
This same VM, when exhibiting the degradation post-reboot:
badVM# /usr/local/fio-3.16/bin/fio --filename=/logdevices/testfile111.out --name fio_test_4k --direct=1 --rw=randwrite --bs=4k --size=64mb --numjobs=1 --iodepth=1 --time_based --ioengine=libaio --runtime=10 --group_reporting --norandommap | grep IOPS
write: IOPS=10.0k, BW=39.2MiB/s (41.1MB/s)(392MiB/10001msec)
The IOPs seen on the bad VM are notably low.
Environment
Red Hat Enterprise Linux Virtual Machine (VM)
VMWare ESX Hypervisor
Subscriber exclusive content
A Red Hat subscription provides unlimited access to our knowledgebase, tools, and much more.