Chapter 9. Migrating virtual machines Red Hat Enterprise Linux 8

9.1. How migrating virtual machines works

The essential part of virtual machine (VM) migration is copying the XML configuration of a VM to a different host machine. If the migrated VM is not shut down, the migration also transfers the state of the VM’s memory and any virtualized devices to a destination host machine. For the VM to remain functional on the destination host, the VM’s disk images must remain available to it.

By default, the migrated VM is transient on the destination host, and remains defined also on the source host.

You can migrate a running VM using live or non-live migrations. To migrate a shut-off VM, you must use an offline migration. For details, see the following table.

Table 9.1. VM migration types

Migration type	Description	Use case	Storage requirements
Live migration	The VM continues to run on the source host machine while KVM is transferring the VM’s memory pages to the destination host. When the migration is nearly complete, KVM very briefly suspends the VM, and resumes it on the destination host.	Useful for VMs that require constant uptime. However, VMs that modify memory pages faster than KVM can transfer them, such as VMs under heavy I/O load, cannot be live-migrated, and non-live migration must be used instead.	The VM’s disk images must be located on a shared network, accessible both to the source host and the destination host.
Non-live migration	Suspends the VM, copies its configuration and its memory to the destination host, and resumes the VM.	Creates downtime for the VM, but is generally more reliable than live migration. Recommended for VMs under heavy I/O load.	The VM’s disk images must be located on a shared network, accessible both to the source host and the destination host.
Offline migration	Moves the VM’s configuration to the destination host	Recommended for shut-off VMs.	The VM’s disk images do not have to be available on a shared network, and can be copied or moved manually to the destination host instead.

Additional resources

For more information on the benefits of VM migration, see Section 9.2, “Benefits of migrating virtual machines”.
For instructions on setting up shared storage for migrating VMs, see Section 9.4, “Sharing virtual machine disk images with other hosts”.

9.2. Benefits of migrating virtual machines

Migrating virtual machines (VMs) can be useful for:

Load balancing: VMs can be moved to host machines with lower usage if their host becomes overloaded, or if another host is under-utilized.
Hardware independence: When you need to upgrade, add, or remove hardware devices on the host machine, you can safely relocate VMs to other hosts. This means that VMs do not experience any downtime for hardware improvements.
Energy saving: VMs can be redistributed to other hosts, and the unloaded host systems can thus be powered off to save energy and cut costs during low usage periods.
Geographic migration: VMs can be moved to another physical location for lower latency or when required for other reasons.

9.3. Limitations for migrating virtual machines

Before migrating virtual machines (VMs) in RHEL 8, ensure you are aware of the migration’s limitations.

Live storage migration cannot be performed on RHEL 8, but you can migrate storage while the VM is powered down. Note that live storage migration is available on Red Hat Virtualization.
Migrating VMs from or to a session connection of libvirt is unreliable and therefore not recommended.
VMs that use certain features and configurations will not work correctly if migrated, or the migration will fail. Such features include:
- Device passthrough
- SR-IOV device assignment
- Mediated devices, such as vGPUs
- Non-Uniform Memory Access (NUMA) pinning

9.4. Sharing virtual machine disk images with other hosts

To perform a live migration of a virtual machine (VM) between supported KVM hosts, shared VM storage is required. This section provides instructions for sharing a locally stored VM image with the source host and the destination host using the NFS protocol.

Prerequisites

The VM intended for migration is shut down.
Optional: A host system is available for hosting the storage that is not the source or destination host, but both the source and the destination host can reach it through the network. This is the optimal solution for shared storage and is recommended by Red Hat.
Make sure that NFS file locking is not used as it is not supported in KVM.

The NFS is installed and enabled on the source and destination hosts. If it is not:

Install the NFS packages:
```
# yum install nfs-utils
```

Make sure that the ports for NFS, such as 2049, are open in the firewall.

# firewall-cmd --permanent --add-service=nfs
# firewall-cmd --permanent --add-service=mountd
# firewall-cmd --permanent --add-service=rpc-bind
# firewall-cmd --permanent --add-port=2049/tcp
# firewall-cmd --permanent --add-port=2049/udp
# firewall-cmd --reload

Start the NFS service.
```
# systemctl start nfs-server
```

Procedure

Connect to the host that will provide shared storage. In this example, it is the cargo-bay host:
```
# ssh root@cargo-bay
root@cargo-bay's password:
Last login: Mon Sep 24 12:05:36 2019
root~#
```
Create a directory that will hold the disk image and will be shared with the migration hosts.
```
# mkdir /var/lib/libvirt/shared-images
```
Copy the disk image of the VM from the source host to the newly created directory. For example, the following copies the disk image of the wanderer1 VM to the /var/lib/libvirt/shared-images/ directory on the`cargo-bay` host:
```
# scp /var/lib/libvirt/images/wanderer1.qcow2 root@cargo-bay:/var/lib/libvirt/shared-images/wanderer1.qcow2
```
On the host that you want to use for sharing the storage, add the sharing directory to the /etc/exports file. The following example shares the /var/lib/libvirt/shared-images directory with the source-example and dest-example hosts:
```
/var/lib/libvirt/shared-images source-example(rw,no_root_squash) dest-example(rw,no_root_squash)
```
On both the source and destination host, mount the shared directory in the /var/lib/libvirt/images directory:
```
# mount cargo-bay:/var/lib/libvirt/shared-images /var/lib/libvirt/images
```

Verification

To verify the process was successful, start the VM on the source host and observe if it boots correctly.

Additional resources

For detailed information on configuring NFS and firewall rules, see Exporting NFS shares.

9.5. Migrating a virtual machine using the command-line interface

If the current host of a virtual machine (VM) becomes unsuitable or cannot be used anymore, or if you want to redistribute the hosting workload, you can migrate the VM to another KVM host. This section provides instructions and examples for various scenarios of such migrations.

Prerequisites

The source host and the destination host both use the KVM hypervisor.
The source host and the destination host are able to reach each other over the network. Use the ping utility to verify this.
For the migration to be supportable by Red Hat, the source host and destination host must be using specific operating systems and machine types. To ensure this is the case, see the VM migration compatibility table.
Red Hat recommends for the disk images of VMs that will be migrated to be located on a separate networked location accessible to both the source host and the destination host. This is optional for offline migration, but required for migrating a running VM.
For instructions to set up such shared VM storage, see Section 9.4, “Sharing virtual machine disk images with other hosts”.
When migrating an existing VM in a public bridge tap network, the source and destination hosts must be located on the same network. Otherwise, the VM network will not operate after migration.

Procedure

Ensure that the libvirtd service is enabled and running.

# systemctl enable libvirtd.service
# systemctl restart libvirtd.service

Use the virsh migrate command with options appropriate for your migration requirements.
- The following migrates the wanderer1 VM from your local host to the system connection of the dest-example host. The VM will remain running during the migration.
```
# virsh migrate --persistent --live wanderer1 qemu+ssh://dest-example/system
```
- The following enables you to make manual adjustments to the configuration of the wanderer2 VM running on your local host, and then migrates the VM to the dest-example host. The migrated VM will automatically use the updated configuration.
```
# virsh dumpxml --migratable wanderer2 >wanderer2.xml
# vi wanderer2.xml
# virsh migrate --live --persistent --xml wanderer2.xml wanderer2 qemu+ssh://dest-example/system
```
  This procedure can be useful for example when the destination host needs to use a different path to access the shared VM storage or when configuring a feature specific to the destination host.
- The following suspends the wanderer3 VM from the source-example host, migrates it to the dest-example host, and instructs it to use the adjusted XML configuration, provided by the wanderer3-alt.xml file. When the migration is completed, libvirt resumes the VM on the destination host.
```
# virsh migrate wanderer3 qemu+ssh://source-example/system qemu+ssh://dest-example/system --xml wanderer3-alt.xml
```
  After the migration, the VM remains in the suspended state on the source host, and the migrated copy is deleted after it is shut down.
- The following deletes the shut-down wanderer4 VM from the source-example host, and moves its configuration to the dest-example host.
```
# virsh migrate --offline --persistent --undefinesource wanderer4 qemu+ssh://source-example/system qemu+ssh://dest-example/system
```
  Note that this type of migration does not require moving the VM’s disk image to shared storage. However, for the VM to be usable on the destination host, you need to migrate the VM’s disk image. For example:
```
# scp root@source-example:/var/lib/libvirt/images/wanderer4.qcow2 root@dest-example:/var/lib/libvirt/images/wanderer4.qcow2
```
Wait for the migration to complete. The process may take some time depending on network bandwidth, system load, and the size of the VM. If the --verbose option is not used for virsh migrate, the CLI does not display any progress indicators except errors.
When the migration is in progress, you can use the virsh domjobinfo utility to display the migration statistics.

Verification

On the destination host, list the available VMs to verify if the VM has been migrated:
```
# virsh list
Id Name                 State
----------------------------------
10 wanderer1              running
```
Note that if the migration is still running, this command will list the VM state as paused.

Troubleshooting

In some cases, the target host will not be compatible with certain values of the migrated VM’s XML configuration, such as the network name or CPU type. As a result, the VM will fail to boot on the target host. To fix these problems, you can update the problematic values by using the virsh edit command.
If a live migration is taking a long time to complete, this may be because the VM is under heavy load and too many memory pages are changing for live migration to be possible. To fix this problem, change the migration to a non-live one by suspending the VM.
```
# virsh suspend wanderer1
```

Additional resources

For further options and examples for virtual machine migration, use virsh migrate --help or see the virsh man page.

9.6. Supported hosts for virtual machine migration

For the virtual machine (VM) migration to work properly and be supported by Red Hat, the source and destination hosts must be specific RHEL versions and machine types. The following table shows supported VM migration paths.

Table 9.2. Live migration compatibility

Migration method	Release type	Example	Support status
Forward	Major release	7.6+ → 8.1	On supported RHEL 7 systems: machine types i440fx and q35
Backward	Major release	8.1 → 7.6+	On supported RHEL 8 systems: machine types i440fx and q35
Forward	Minor release	8.0.1+ → 8.1+	On supported RHEL 7 systems: machine types i440fx and q35 on RHEL 7.6.0 and later. On supported RHEL 8 systems: machine type q35.
Backward	Minor release	8.1 → 8.0.1	On supported RHEL 7 systems. Fully supported for machine types i440fx and q35. On supported RHEL 8 systems: machine type q35.

Additional resources

For information on the currently supported versions of RHEL 7 and RHEL 8, see Red Hat Knowledgebase.

9.7. Additional resources

You can also migrate VMs from a non-KVM hypervisor to a RHEL 7 or RHEL 8 host. This is also referred to as a V2V conversion, and you can find additional information and instructions in the Red Hat Knowledgebase.

Select Your Language

Infrastructure and Management

Cloud Computing

Storage

Runtimes

Integration and Automation

Red Hat Training

Chapter 9. Migrating virtual machines

9.1. How migrating virtual machines works

9.2. Benefits of migrating virtual machines

9.3. Limitations for migrating virtual machines

9.5. Migrating a virtual machine using the command-line interface

9.6. Supported hosts for virtual machine migration

9.7. Additional resources