Chapter 9. Migrating virtual machines

• Live storage migration cannot be performed on RHEL 8, but you can migrate storage while the VM is powered down. Note that live storage migration is available on Red Hat Virtualization.

• Migrating VMs from or to a session connection of libvirt is unreliable and therefore not recommended.

• VMs that use certain features and configurations will not work correctly if migrated, or the migration will fail. Such features include:

  • Device passthrough
  • SR-IOV device assignment
  • Mediated devices, such as vGPUs
• A migration between hosts that use Non-Uniform Memory Access (NUMA) pinning works only if the hosts have similar topology. However, the performance on running workloads might be negatively affected by the migration.

• The emulated CPUs, both on the source VM and the destination VM, must be identical, otherwise the migration might fail. Any differences between the VMs in the following CPU related areas can cause problems with the migration:

  • CPU model

    Note

    Migrating between an Intel 64 host and an AMD64 host is unsupported, even though they share the x86-64 instruction set.

  • Firmware settings
  • Microcode version
  • BIOS version
  • BIOS settings
  • QEMU version
  • Kernel version

• Live migrating a VM that uses more than 1 TB of memory may in some cases not work reliably. The stability of such a migration depends on the following:

  • The current workload of the VM
  • The network bandwidth that the host can use for migration
  • The downtime that your deployment is able to support

  For live migration scenarios that involve VMs with more than 1 TB of memory, customers should consult Red Hat.

Prerequisites

• The VM intended for migration is shut down.
• Optional: A host system is available for hosting the storage that is not the source or destination host, but both the source and the destination host can reach it through the network. This is the optimal solution for shared storage and is recommended by Red Hat.
• Make sure that NFS file locking is not used as it is not supported in KVM.

• The NFS is installed and enabled on the source and destination hosts. If it is not:

  1. Install the NFS packages:

     # yum install nfs-utils

  2. Make sure that the ports for NFS, such as 2049, are open in the firewall.

     # firewall-cmd --permanent --add-service=nfs
     # firewall-cmd --permanent --add-service=mountd
     # firewall-cmd --permanent --add-service=rpc-bind
     # firewall-cmd --permanent --add-port=2049/tcp
     # firewall-cmd --permanent --add-port=2049/udp
     # firewall-cmd --reload

  3. Start the NFS service.

     # systemctl start nfs-server

Verification

• To verify the process was successful, start the VM on the source host and observe if it boots correctly.

Prerequisites

• The source host and the destination host both use the KVM hypervisor.
• The source host and the destination host are able to reach each other over the network. Use the ping utility to verify this.

• Ensure the following ports are open on the destination host.

  • Port 22 is needed for connecting to the destination host by using SSH.
  • Port 16509 is needed for connecting to the destination host by using TLS.
  • Port 16514 is needed for connecting to the destination host by using TCP.
  • Ports 49152-49215 are needed by QEMU for transfering the memory and disk migration data.
• For the migration to be supportable by Red Hat, the source host and destination host must be using specific operating systems and machine types. To ensure this is the case, see Supported hosts for virtual machine migration.

• The disk images of VMs that will be migrated are located on a separate networked location accessible to both the source host and the destination host. This is optional for offline migration, but required for migrating a running VM.

  For instructions to set up such shared VM storage, see Sharing virtual machine disk images with other hosts.

• When migrating a running VM, your network bandwidth must be higher than the rate in which the VM generates dirty memory pages.

  To obtain the dirty page rate of your VM before you start the live migration, do the following:

  1. Monitor the rate of dirty page generation of the VM for a short period of time.

     # virsh domdirtyrate-calc vm-name 30

  2. After the monitoring finishes, obtain its results:

     # virsh domstats vm-name --dirtyrate
     Domain: 'vm-name'
       dirtyrate.calc_status=2
       dirtyrate.calc_start_time=200942
       dirtyrate.calc_period=30
       dirtyrate.megabytes_per_second=2

     In this example, the VM is generating 2 MB of dirty memory pages per second. Attempting to live-migrate such a VM on a network with a bandwidth of 2 MB/s or less will cause the live migration not to progress if you do not pause the VM or lower its workload.

     To ensure that the live migration finishes successfully, Red Hat recommends that your network bandwidth is significantly greater than the VM’s dirty page generation rate.

• When migrating an existing VM in a public bridge tap network, the source and destination hosts must be located on the same network. Otherwise, the VM network will not operate after migration.

• Ensure that the libvirtd service is enabled and running.

  # systemctl enable --now libvirtd.service

Verification

• On the destination host, list the available VMs to verify if the VM has been migrated:

  # virsh list
  Id Name                 State
  ----------------------------------
  10 wanderer1              running

  If the migration is still running, this command will list the VM state as paused.

Troubleshooting

• In some cases, the target host will not be compatible with certain values of the migrated VM’s XML configuration, such as the network name or CPU type. As a result, the VM will fail to boot on the target host. To fix these problems, you can update the problematic values by using the virsh edit command. After updating the values, you must restart the VM for the changes to be applied.

• If a live migration is taking a long time to complete, this may be because the VM is under heavy load and too many memory pages are changing for live migration to be possible. To fix this problem, change the migration to a non-live one by suspending the VM.

  # virsh suspend wanderer1

Prerequisites

• The web console VM plug-in is installed on your system.
• The source and destination hosts are running.

• Ensure the following ports are open on the destination host.

  • Port 22 is needed for connecting to the destination host by using SSH.
  • Port 16509 is needed for connecting to the destination host by using TLS.
  • Port 16514 is needed for connecting to the destination host by using TCP.
  • Ports 49152-49215 are needed by QEMU for transfering the memory and disk migration data.
• The VM’s disk images are located on a shared storage that is accessible to the source host as well as the destination host.

• When migrating a running VM, your network bandwidth must be higher than the rate in which the VM generates dirty memory pages.

  To obtain the dirty page rate of your VM before you start the live migration, do the following in your command-line interface:

  1. Monitor the rate of dirty page generation of the VM for a short period of time.

     # virsh domdirtyrate-calc vm-name 30

  2. After the monitoring finishes, obtain its results:

     # virsh domstats vm-name --dirtyrate
     Domain: 'vm-name'
       dirtyrate.calc_status=2
       dirtyrate.calc_start_time=200942
       dirtyrate.calc_period=30
       dirtyrate.megabytes_per_second=2

     In this example, the VM is generating 2 MB of dirty memory pages per second. Attempting to live-migrate such a VM on a network with a bandwidth of 2 MB/s or less will cause the live migration not to progress if you do not pause the VM or lower its workload.

     To ensure that the live migration finishes successfully, Red Hat recommends that your network bandwidth is significantly greater than the VM’s dirty page generation rate.

• Confirm whether the VM appears in the list of VMs available on the destination host.
• Start the migrated VM and observe if it boots up.