Chapter 9. Migrating virtual machines
If the current host of a virtual machine (VM) becomes unsuitable or cannot be used anymore, you can migrate the VM to another KVM host. The sections below provide information and instructions for such a migration.
9.1. How migrating virtual machines works
The essential part of virtual machine (VM) migration is copying the XML configuration of a VM to a different host machine. If the migrated VM is not shut down, the migration also transfers the state of the VM’s memory and any virtualized devices to a destination host machine. For the VM to remain functional on the destination host, the VM’s disk images must remain available to it.
You can migrate a running VM using live or non-live migrations. To migrate a shut-off VM, an offline migration must be used.
In a live migration, the VM continues to run on the source host machine while KVM is transferring the VM’s memory pages to the destination host. When the migration is nearly complete, KVM very briefly suspends the VM, and resumes it on the destination host.
This is useful for VMs that require constant uptime. However, VMs that modify memory pages faster than KVM can transfer them, such as VMs under heavy I/O load, cannot be live-migrated, and non-live migration must be used instead.
A non-live migration suspends the VM, copies its configuration and its memory to the destination host, and resumes the VM. This creates downtime for the VM, but is generally more reliable than live migration.Important
For the migration of a running VM (both live and non-live) to work properly, the VM’s disk images must be located on a shared network, accessible both to the source host and the destination host. For instructions on setting up such shared storage, see Section 9.3, “Sharing virtual machine images with other hosts”.
- An offline migration moves the VM’s configuration to the destination host. When an offline migration is used, the VM’s disk images do not have to be available on a shared network, and can be copied or moved manually to the destination host instead.
Migrating VMs can be useful for:
- Load balancing
- VMs can be moved to host machines with lower usage if their host becomes overloaded, or if another host is under-utilized.
- Hardware independence
- When you need to upgrade, add, or remove hardware devices on the host machine, you can safely relocate VMs to other hosts. This means that VMs do not experience any downtime for hardware improvements.
- Energy saving
- VMs can be redistributed to other hosts, and the unloaded host systems can thus be powered off to save energy and cut costs during low usage periods.
- Geographic migration
- VMs can be moved to another physical location for lower latency or when required for other reasons.
9.2. Requirements and limitations for migrating virtual machines
Before using virtual machine (VM) migration in RHEL 8, make sure that your system fulfills the migration’s requirements, and that you are aware of its limitations.
- The source host and the destination host must both be using the KVM hypervisor.
The source host and the destination host must be able to reach each other over the network. Use the
pingutility to verify this.
- For the migration to be supportable by Red Hat, the source host and destination host must be using specific operating systems and machine types. To ensure this is the case, see the VM migration compatibility table.
Red Hat recommends for the disk images of VMs that will be migrated to be located on a separate networked location accessible to both the source host and the destination host. This is optional for offline migration, but required for migrating a running VM.
For instructions to set up such shared VM storage, see Section 9.3, “Sharing virtual machine images with other hosts”.
Make sure that the
libvirtdservice is enabled and running.
# systemctl enable libvirtd.service # systemctl restart libvirtd.service
- When migrating an existing VM in a public bridge tap network, the source and destination hosts must be located on the same network. Otherwise, the VM network will not operate after migration.
VM migration has the following limitations when used on RHEL 8:
- Live storage migration cannot be performed on RHEL 8, but you can migrate storage while the VM is powered down. Note that live storage migration is available on Red Hat Virtualization.
Migrating VMs from or to a user session of
libvirtis unreliable and therefore not recommended.
VMs that use certain features and configurations will not work correctly if migrated, or the migration will fail. Such features include:
- Device passthrough
- SR-IOV device assignment
- Mediated devices, such as vGPUs
- Non-Uniform Memory Access (NUMA) pinning
9.4. Migrating a virtual machine using the command-line interface
This section provides instructions for migrating a virtual machine (VM) from one KVM host to another, as well as examples for various scenarios of such migrations.
virsh migratecommand with options appropriate for your migration requirements.
The following migrates the
kal-elVM from your local host to the system session of the
ter.rahost. The VM will remain running during the migration.
# virsh migrate --persistent --live kal-el qemu+ssh://ter.ra/system
The following enables you to make manual adjustments to the configuration of the
jor-elVM running on your local host, and then migrates the VM to the
ter.rahost. The migrated VM will automatically use the updated configuration.
# virsh dumpxml --migratable jor-el >jor-el.xml # vi jor-el.xml # virsh migrate --live --persistent --xml jor-el.xml jor-el qemu+ssh://ter.ra/system
This procedure can be useful for example when the destination host needs to use a different path to access the shared VM storage or when configuring a feature specific to the destination host.
The following suspends the
zodVM from the
krypt.onhost, migrates it to the
ter.rahost, and instructs it to use the adjusted XML configuration, provided by the
zod-alt.xmlfile. When the migration is completed,
libvirtresumes the VM on the destination host.
# virsh migrate --persistent zod qemu+ssh://krypt.on/system qemu+ssh://ter.ra/system --xml zod-alt.xml
The following deletes the shut-down
faoraVM from the
krypt.onhost, and moves its configuration to the
# virsh migrate --offline --persistent --undefinesource faora qemu+ssh://krypt.on/system qemu+ssh://ter.ra/system
Note that this type of migration does not require moving the VM’s disk image to shared storage. However, for the VM to be usable on the destination host, you need to migrate the VM’s disk image. For example:
# scp email@example.com:/var/lib/libvirt/images/faora.qcow2 firstname.lastname@example.org:/var/lib/libvirt/images/faora.qcow2
Wait for the migration to complete. The process may take some time depending on network bandwidth, system load, and the size of the VM. If the
--verboseoption is not used for
virsh migrate, the CLI does not display any progress indicators except errors.
In addition, you can use the
virsh domjobinfoutility to display the migration statistics.
On the destination host, list the available VMs to verify if the VM has been migrated:
# virsh list Id Name State ---------------------------------- 10 kal-el running
Note that if the migration is still running, this command will list the VM state as
If a live migration is taking a long time to complete, this may be because the VM is under heavy load and too many memory pages are changing for live migration to be possible. To fix this problem, change the migration to a non-live one by suspending the VM.
# virsh suspend kal-el
For further options and examples for virtual machine migration, see the
9.5. Supported hosts for virtual machine migration
For the virtual machine (VM) migration to work properly and be supported by Red Hat, the source and destination hosts must be specific RHEL versions and machine types. The following table shows supported VM migration paths.
Table 9.1. Live migration compatibility
|Migration method||Release type||Example||Support status|
7.6+ → 8.1
On supported RHEL 7 systems: machine types i440fx and q35
8.1 → 7.6+
On supported RHEL 8 systems: machine types i440fx and q35
8.0.1+ → 8.1+
On supported RHEL 7 systems: machine types i440fx and q35 on RHEL 7.6.0 and later.
On supported RHEL 8 systems: machine type q35.
8.1 → 8.0.1
On supported RHEL 7 systems. Fully supported for machine types i440fx and q35.
On supported RHEL 8 systems: machine type q35.
- For information on the currently supported versions of RHEL 7 and RHEL 8, see Red Hat Knowledge Base.
9.6. Additional resources
You can also migrate VMs from a non-KVM hypervisor to a RHEL 7 or RHEL 8 host. This is also referred to as a
V2V conversion, and you can find additional information and instructions in the Red Hat Knowledgebase.