rhel6: Why do live migrations of busy VMs cause a much larger downtime than configured in vdsm.conf?

Solution Unverified - Updated -

Environment

  • Red Hat Enterprise Linux 6
  • Memory busy virtual machines.
  • qemu-kvm-rhev version 0.12.1.2-2.355.el6_4.3

Issue

  • When live migrating a (memory) busy VM, downtime during the last phase of the live migration takes much longer than the maximum specified in vdsm.conf.

Resolution

  • RHEL6.5 stream: Update qemu-kvm packages to version 0.12.1.2-2.415.el6_5.16 (released with RHBA-2016-1845) or newer which include fixes for this issue.
  • RHEL6 stream: Update qemu-kvm packages to version 0.12.1.2-2.448.el6_6.2 or newer. See the Security Advisory for more details. RHEL6.7GA and later already include this fix.

Diagnostic Steps

  • Default maximum downtime in vdsm.conf is 500 milliseconds:

    # cat /usr/share/doc/vdsm-4.10.2/vdsm.conf.sample | grep -B3 migration_downtime
# Maxmium allowed downtime for live migration in milliseconds
# (anything below 100ms is ignored) if you do not care about liveness
# of migration, set to a very high value, such as 600000.
# migration_downtime = 500

  • Pinging the VM during the live migration shows an effective downtime of almost 9 seconds:

    64 bytes from 10.33.20.94: icmp_seq=61 ttl=61 time=0.522 ms
64 bytes from 10.33.20.94: icmp_seq=62 ttl=61 time=0.644 ms
64 bytes from 10.33.20.94: icmp_seq=63 ttl=61 time=9609 ms  <--- ***
64 bytes from 10.33.20.94: icmp_seq=64 ttl=61 time=8609 ms
64 bytes from 10.33.20.94: icmp_seq=65 ttl=61 time=7609 ms
64 bytes from 10.33.20.94: icmp_seq=66 ttl=61 time=6609 ms
64 bytes from 10.33.20.94: icmp_seq=67 ttl=61 time=5609 ms
64 bytes from 10.33.20.94: icmp_seq=68 ttl=61 time=4609 ms
64 bytes from 10.33.20.94: icmp_seq=69 ttl=61 time=3609 ms
64 bytes from 10.33.20.94: icmp_seq=70 ttl=61 time=2609 ms
64 bytes from 10.33.20.94: icmp_seq=71 ttl=61 time=1609 ms
64 bytes from 10.33.20.94: icmp_seq=72 ttl=61 time=609 ms
64 bytes from 10.33.20.94: icmp_seq=73 ttl=61 time=18.4 ms
64 bytes from 10.33.20.94: icmp_seq=74 ttl=61 time=0.709 ms
64 bytes from 10.33.20.94: icmp_seq=75 ttl=61 time=0.340 ms
  • Component
  • kvm

This solution is part of Red Hat’s fast-track publication program, providing a huge library of solutions that Red Hat engineers have created while supporting our customers. To give you the knowledge you need the instant it becomes available, these articles may be presented in a raw and unedited form.

Close

Welcome! Check out the Getting Started with Red Hat page for quick tours and guides for common tasks.