Infrastructure Migration Solution Guide

Red Hat Infrastructure Migration Solution 1.1

Migrating from VMware to Red Hat Virtualization or Red Hat OpenStack Platform

Red Hat IMS Documentation Team

Abstract

This guide describes how to migrate VMware virtual machines to Red Hat Virtualization 4.2 or Red Hat OpenStack Platform 13 or 14, using Red Hat CloudForms 4.7.3.

Preface

Red Hat Infrastructure Migration Solution (IMS) 1.1 enables you to migrate virtual machines from VMware 5.5 (and later) to Red Hat Virtualization or Red Hat OpenStack Platform, using Red Hat CloudForms.

Red Hat Virtualization 4.2

Red Hat Virtualization provides a virtualization platform built on Red Hat Enterprise Linux. You can manage your virtual infrastructure, including hosts, virtual machines, networks, storage, and users, from a centralized graphical user interface or with a REST API. See Red Hat Virtualization 4.2 documentation for more information.

See Part I, “Migrating from VMware to Red Hat Virtualization”.

Red Hat OpenStack Platform 13 and 14

Red Hat OpenStack Platform provides a scaleable, fault-tolerant, private or public cloud based on Red Hat Enterprise Linux. See Red Hat OpenStack Platform 13 or Red Hat OpenStack Platform 14 documentation for more information.

See Part II, “Migrating from VMware to Red Hat OpenStack Platform”.

Red Hat CloudForms 4.7.3
Red Hat CloudForms is the environment in which you perform the migration. Red Hat CloudForms is a unified set of management tools for use across virtualized, private cloud, and public cloud platforms. See Red Hat CloudForms 4.7 documentation for more information.

Part I. Migrating from VMware to Red Hat Virtualization

The migration process involves the following tasks:

  1. Planning the migration. See Chapter 1, Planning the migration.
  2. Preparing the VMware, Red Hat Virtualization, and CloudForms environments. See Chapter 2, Preparing the environment for migration.
  3. Migrating the virtual machines. See Chapter 3, Migrating the virtual machines.
  4. Troubleshooting, if necessary. See Chapter 4, Troubleshooting.

Chapter 1. Planning the migration

During the planning phase, you will formulate a specific migration goal, for example, "I want to migrate 2000 virtual machines, with 200 TB of data, in less than 6 months".

Review the following information to plan your migration:

The migration workflow describes the migration process in greater detail.

Figure 1.1. VMware to Red Hat Virtualization migration workflow

vmware to rhv migration workflow

25 You create and run a migration plan in CloudForms.

25 CloudForms uses the migration plan to locate the source virtual machines.

25 If you are using VDDK transformation to migrate your virtual machines, CloudForms captures the ESXi host fingerprint for authentication during the virtual machine conversion process.

25 Using the attributes defined for the Red Hat Virtualization environment, CloudForms initiates communication with the conversion hosts (Red Hat Virtualization hosts with virt-v2v and virt-v2v-wrapper installed).

25 virt-v2v-wrapper connects to the source datastore through the ESXi host. virt-v2v streams the source disks to the target data domain and converts the source disks.

25 virt-v2v-wrapper creates a target Red Hat Virtualization virtual machine, using the source virtual machine’s metadata in order to maintain its attributes (tags, power state, MAC address, CPU count, memory, disks, and virtual machine name) after migration.

25 virt-v2v attaches the converted disks to the Red Hat Virtualization virtual machine. (The virtual machine’s power state is the same as the source virtual machine’s power state.)

The migration process is complete and the migration plan’s status is displayed in CloudForms.

1.1. Questions to ask before migration

The following questions can help you to estimate the resources and time required for migration.

What am I migrating?
  • Identify the VMware virtual machines that you will be migrating.
What is the maximum number of disks or virtual machines that I can migrate?
  • There is no maximum number of disks or virtual machines that you can migrate. However, you may not want to migrate all your virtual machines at the same time, in order to minimize the impact on your users.

    Important

    If you exceed the capabilities of your environment, the migrations will fail. This situation could affect existing applications running on virtual machines attached to the network and storage.

What operating systems can I migrate?
What am I missing?
  • Identify resource gaps, such as bandwidth, storage, licenses, or a suitable maintenance window, before you begin the migration.
What impact will the migration have on my users?
  • Assess the effects the migration may have on a production environment.

    It may be possible to migrate your applications in phases, without downtime at the application layer, if the applications are distributed in a high-availability architecture.

  • Check whether users will lose access to critical applications.
How long will the migration take?

There is no formula to estimate how long the actual migration will take. This is determined on a case-by-case basis.

The following example is provided as a guide:

  • Duration of migration: 1:15:53 (hh:mm:ss)
  • 10 virtual machines
  • Total data migrated, using VDDK: 1000 GB
  • Hardware:

    • Strong host (40 cores, 500 GB RAM)
    • Fast SSD XtremIO storage
    • Fibre Channel 8 interface for host-to-storage connection
    • 10 GbE network interface cards for all other connections
How many conversion hosts do I need?

The number of conversion hosts you create depends on the size of your migration. All the virtual machines in a migration plan are migrated at the same time, in parallel. The number of virtual machines that you can migrate simultaneously depends on your infrastructure capabilities. Each migration requires a certain amount of network bandwidth, I/O throughput, and processing power for the conversion process.

Multiple conversion hosts provide load-balancing and better performance, even for small migrations.

Conversion hosts are limited to a maximum of ten concurrent migrations, unless you change the default values.

You should test your environment thoroughly before the migration to determine how many migrations it can support without negative effects, for example, five conversion hosts, each running ten concurrent migrations.

Should I migrate my virtual machines with VDDK or SSH?

You can configure your conversion hosts to migrate the virtual machines by using either the VMware Virtual Disk Development Kit (VDDK) or SSH. The choice is determined by the number of virtual machines you are migrating and the capabilities of your infrastructure.

The following table compares the options.

Table 1.1. VDDK and SSH comparison

 ProsCons

VDDK

  • Recommended because it is approximately twice as fast as SSH
  • Compiled with nbdkit, which provides direct connectivity to the virtual machine disks
  • Requires VDDK package download
  • Requires VDDK package location for conversion host configuration
  • Limited to 20 concurrent migrations per conversion host
  • Limited to 10 concurrent migrations per VMware hypervisor unless you increase the hypervisor’s NFC service memory

SSH

  • Better suited for large-scale migrations than VDDK (not affected by hypervisor’s NFC service memory limitation)
  • Does not require additional downloads, packages, tools, or compilation
  • Slower than VDDK
  • Requires VMware hypervisors to have SSH access enabled
  • Requires VMware hypervisors and conversion hosts to have shared SSH keys

1.2. Recommendations and best practices

The following recommendations will help to minimize the impact of the migration on your environment.

Scheduling the migration

  • Schedule your migration carefully, to minimize the impact on your users.
  • Prepare your users for downtime.

    Important

    Currently, IMS supports only cold migration. Virtual machines are powered off gracefully as part of the migration process.

  • Stagger the migration schedules.
  • Move critical applications during maintenance windows.

Distributing the migration workload

  • Create migration groups, so that you are not migrating all of your virtual machines at the same time, keeping in mind the following considerations:

    • How are the virtual machines grouped now?
    • Which virtual machines should be migrated together?
    • Which workloads or linked applications should be migrated together?
    • What applications must remain available?
  • Consider which parts of the workload to migrate first:

    • Databases
    • Applications
    • Web servers
    • Load balancers

Deploying the conversion hosts

  • Create a sufficient number of conversion hosts for your migration, with sufficient resources.
  • Choose the best transformation option (SSH or VDDK) for your environment.

Controlling the migration process

  • Create multiple migration plans for finer control.
  • Perform test migrations with different maximum numbers of concurrent migrations to assess the capabilities of your environment’s infrastructure.

Chapter 2. Preparing the environment for migration

2.1. Preparing the VMware environment

Preparing the VMware environment for migration involves the following tasks:

  1. Extending the VMware network. See Section 2.1.1, “Preparing the VMware network”.
  2. Preparing the VMware virtual machines. See Section 2.1.2, “Preparing the VMware virtual machines”.
  3. Configuring the VMware hypervisors if you are using SSH transformation. See Section 2.1.3, “Preparing the VMware hypervisors for SSH transformation”.
  4. Configuring a VMware hypervisor if you are using VDDK and performing more than ten concurrent migrations from that hypervisor. See Section 2.1.4, “Configuring a VMware hypervisor for more than ten concurrent VDDK migrations”.

2.1.1. Preparing the VMware network

Extend the VMware network to the Red Hat Virtualization environment.

Important
  • The network configuration must not be changed in any way during the migration.
  • IP addresses, VLANs, and other network configuration must not be changed before or after migration because the conversion process preserves the source virtual machine MAC addresses.

You can prepare the VMware virtual machines. See Section 2.1.2, “Preparing the VMware virtual machines”.

2.1.2. Preparing the VMware virtual machines

Perform the following steps on each VMware virtual machine that you are migrating:

  1. Install VMware Tools to capture IP addresses.

    To download and install VMware Tools, see VMware Workstation 5.0: Installing VMware Tools.

  2. Unmount mounted ISO/CDROM disks.
  3. Ensure that attached disks are not encrypted.
  4. Ensure that each NIC has no more than one IPv4 and/or one IPv6 address.
  5. Ensure that the virtual machine names contain only upper- or lower-case letters, numbers, underscores (_), hyphens (-), or periods (.).

    Note

    International characters and spaces are not permitted.

  6. Ensure that the virtual machine names do not duplicate names of virtual machines in the Red Hat Virtualization environment.

If you are using SSH transformation, you can configure the VMware hypervisors. See Section 2.1.2, “Preparing the VMware virtual machines”.

If you are using VDDK transformation and performing more than ten concurrent migrations from a VMware hypervisor, you must configure the hypervisor to support the additional connections. See Section 2.1.4, “Configuring a VMware hypervisor for more than ten concurrent VDDK migrations”.

2.1.3. Preparing the VMware hypervisors for SSH transformation

If you are using SSH transformation, you can configure the VMware hypervisors for passwordless access by sharing a conversion host’s public SSH key with the hypervisors.

Important

A single SSH key pair is recommended because the key pair is used only for virtual machine conversion and it simplifies conversion host management.

If you wish to use a dedicated SSH key pair for each conversion host, you can copy the public key of each conversion host to all the VMware hypervisors.

Procedure

  1. Enable SSH access on each VMware hypervisor:

    For instructions, navigate to VMware vSphere Documentation. In the navigation pane, click vSphere versionESXi and vCenter ServerVMware ESXi Installation and SetupInstalling and Setting Up ESXiSetting Up ESXiEnable ESXi Shell and SSH Access with the Direct Console User Interface.

    Note

    If your security policies allow you to collect the SSH public keys of the VMware hypervisors at this stage, save them for copying to the conversion hosts.

  2. Generate an SSH key pair without a passphrase:

    # ssh-keygen -N ''
  3. Copy the public SSH key to /etc/ssh/keys-root/authorized_keys on each VMware hypervisor.

    You will use the private SSH key to configure the conversion hosts.

2.1.4. Configuring a VMware hypervisor for more than ten concurrent VDDK migrations

If you are performing more than ten concurrent migrations from a VMware hypervisor using VDDK transformation, the migration will fail because the hypervisor’s NFC service memory buffer is limited to ten parallel connections. See VMware vSphere 6.5 NFC session connection limits and Virt-v2v. VDDK: ESXi NFC service memory limits for details.

You can increase the hypervisor’s NFC service memory to enable additional connections for migrations.

Procedure

  1. Log in to a VMware hypervisor.
  2. Change the value of maxMemory to 1000000000 in /etc/vmware/hostd/config.xml:

          <nfcsvc>
             <path>libnfcsvc.so</path>
             <enabled>true</enabled>
             <maxMemory>1000000000</maxMemory>
             <maxStreamMemory>10485760</maxStreamMemory>
          </nfcsvc>
  3. Restart hostd:

    # /etc/init.d/hostd restart

    You do not need to reboot the VMware hypervisor.

2.2. Preparing the Red Hat Virtualization environment

Preparing the Red Hat Virtualization environment involves the following key steps:

  1. Installing and configuring Red Hat Virtualization 4.2. See Section 2.2.1, “Installing and configuring Red Hat Virtualization 4.2”.
  2. Installing and configuring CloudForms 4.7.3. See Section 2.2.2, “Installing and configuring CloudForms 4.7.3”.
  3. Reinstalling ipa-client, if you are configuring the conversion hosts to use SSSD with single sign-on. See Section 2.2.3, “Reinstalling ipa-client.

Prerequisites

The following prerequisites apply:

  • BIOS settings of physical hosts adjusted for optimal performance (rather than power-saving), according to the vendor’s recommendations
  • C1E halt state disabled, if applicable
  • Compatible software versions:

    Table 2.1. Software compatibility

    SoftwareVersion

    VMware

    5.5 or later

    Red Hat Virtualization

    4.2.8

    CloudForms

    4.7.3, with CFME 5.10.3

    CFME 5.10.4 does not support migration.

2.2.1. Installing and configuring Red Hat Virtualization 4.2

  1. Install Red Hat Virtualization Manager 4.2 on the Manager machine. See Installing the Red Hat Virtualization Manager in the Red Hat Virtualization Installation Guide.
  2. Install Red Hat Virtualization Host 4.2 or Red Hat Enterprise Linux 7.6 on physical machines. See Installing Red Hat Virtualization Host or Installing Red Hat Enterprise Linux Hosts in the Red Hat Virtualization Installation Guide.
  3. Enable the following ports in the conversion host network:

  4. Create and attach data and ISO storage domains to the data center, ensuring that the data domains have sufficient space for the migrated virtual machines. See Storage in the Red Hat Virtualization Administration Guide.

    Note

    IMS only supports shared storage, such as NFS, iSCSI, or FCP. Local storage is not supported.

  5. Upload the VirtIO and Guest Tool image files to the ISO storage domain. See Uploading the VirtIO and Guest Tool Image Files to an ISO Storage Domain in the Red Hat Virtualization Administration Guide.

    The VirtIO file name must include the version number: virtio-win-version.iso. The Guest Tool is required for migrating Windows virtual machines.

  6. Optionally, you can create a MAC address pool that includes the MAC addresses of the VMware virtual machines to be migrated. See Creating MAC Address Pools in the Red Hat Virtualization Administration Guide.

    Important

    If the Red Hat Virtualization MAC address pool range overlaps the VMware MAC address range, you must ensure that the MAC addresses of the migrating virtual machines do not duplicate those of existing virtual machines. Otherwise, the migration will fail.

    If you do not create a MAC address pool, the migrated virtual machines will not have MAC addresses in the same range as virtual machines created in Red Hat Virtualization.

2.2.2. Installing and configuring CloudForms 4.7.3

  1. Install Red Hat CloudForms 4.7.3 (CFME 5.10.3) on the Manager machine. See Installing Red Hat CloudForms on Red Hat Virtualization.

    Important

    CFME 5.10.4 does not support migration.

  2. Add VMware to CloudForms as a provider. See Adding a VMware vCenter Provider in Red Hat CloudForms: Managing Providers.
  3. Add Red Hat Virtualization to CloudForms as a provider. Adding a Red Hat Virtualization Provider in Red Hat CloudForms: Managing Providers.

    Note

    Removing or changing a provider (including refreshing Red Hat Virtualization hosts) before, during, or after the migration will cause errors in the infrastructure mappings and migration plans.

2.2.3. Reinstalling ipa-client

If you are configuring your conversion hosts for SSSD with single sign-on, you must reinstall ipa-client without the OpenSSH client. Otherwise, SSH will fail for the vdsm user.

See BZ#1544379: ipa-client-install changes system-wide SSH configuration. This issue cannot be resolved by modifying the configuration file because the file is restored during upgrades.

  1. Log in to the Manager machine using SSH.
  2. Uninstall ipa-client:

    # ipa-client-install --uninstall
  3. Reinstall ipa-client without OpenSSH:

    # ipa-client-install --no-ssh

2.3. Configuring the conversion hosts for VDDK or SSH transformation

Perform the following steps to configure your conversion hosts for VDDK or SSH transformation:

2.3.1. Downloading VDDK

Download and save the VMware Virtual Disk Development Kit:

  1. Navigate to VMware Documentation.
  2. Click VMware SDK & API Product Documentation to expand.
  3. Click VMware Virtual Disk Development Kit (VDDK).
  4. Click Latest Releases and select the latest VDDK release.
  5. Click Download SDKs to download the VDDK archive file.
  6. Save the VDDK archive file in an HTTP-accessible location and record its path.

You can configure the conversion hosts with Ansible playbooks.

2.3.2. Configuring the conversion hosts with Ansible playbooks

You can configure your conversion hosts for VDDK or SSH transformation with Ansible playbooks:

  1. Create the extra_vars.yml file for the conversion host parameters. See Section 2.3.2.1, “Creating the extra_vars.yml file”.
  2. Create the secure_vars.yml file for secure variables. See Section 2.3.2.2, “Creating the secure_vars.yml file”.
  3. Configure and verify the conversion hosts by running the Ansible playbooks. See Section 2.3.2.3, “Configuring and verifying the conversion hosts”.

2.3.2.1. Creating the extra_vars.yml file

  1. Log in to the Manager machine using SSH.
  2. Install the ovirt-ansible-v2v-conversion-host package:

    # yum install ovirt-ansible-v2v-conversion-host
  3. Create the extra_vars.yml file with the following parameters:

    ---
    v2v_host_type: rhv
    
    # Transport methods to configure on the conversion host. Valid values: "vddk", "ssh"
    v2v_transport_methods:
      - vddk 1
    
    # Maximum number of concurrent conversions per host. Default is "10".
    v2v_max_concurrent_conversions: 10 2
    
    # File name of VDDK package
    v2v_vddk_package_name: "VMware-vix-disklib-version.x86_64.tar.gz" 3
    
    # URL of VDDK package
    v2v_vddk_package_url: "http://path_to_vddk_package/{{ v2v_vddk_package_name }}" 4
    
    # Name of the CloudForms provider to which the conversion host belongs
    manageiq_provider_name: RHV
    
    # Base URL of CloudForms machine
    manageiq_url: "https://CloudForms_FQDN" 5
    
    # Whether to validate certificate of CloudForms server. Default is "true".
    manageiq_validate_certs: true 6
    
    manageiq_zone_id: "42000000000001"' 7
    
    # Empty vmware_hosts variable for conversion_host_disable.yml
    vmware_hosts: ""
    
    # List of infrastructure providers
    # Each provider is a dictionary with 3 attributes: "name", "hostname", and "connection_configurations"
    manageiq_providers:
      - name: "RHV"
        hostname: Manager_FQDN_or_IP_address
        connection_configurations:
          - endpoint:
              role: "default"
              certificate_authority: | 8
                -----BEGIN CERTIFICATE-----
                MIIDoDCCAoigAwIBAgIBATANBgkqhkiG9w0BAQsFADA9MRswGQYDVQ....
                -----END CERTIFICATE-----
1
Select a transformation method, VDDK or SSH.
2
v2v_max_concurrent_conversions is the maximum number of concurrent conversions per host. The default value is 10. If you are using VDDK transformation, do not set this value higher than 20.
3
Update the v2v_vddk_package_name with the correct version.
4
v2v_vddk_package_url is the path to the VDDK archive file that you downloaded.
5
manageiq_url is the FQDN of the CloudForms machine.
6
You can set manageiq_validate_certs to false if you do not want to validate the CloudForms CA certificate. The default value is true.
7
To obtain the manageiq_zone_id, enter this command on the CloudForms machine:
# curl -sk -u admin \'https://CloudForms_FQDN/api/zones/?filter\[\]=name=RHV&expand=resources&attributes=zone
8
The certificate_authority is stored as /etc/pki/ovirt-engine/apache-ca.pem on the Manager machine.

2.3.2.2. Creating the secure_vars.yml file

  1. Create an encrypted secure_vars.yml file:

    # ansible-vault create secure_vars.yml
  2. Add the following parameters and save the file:

    ---
    # CloudForms "admin" user, for connecting to CloudForms
    manageiq_username: "username"
    
    # CloudForms "admin" password:
    manageiq_password: "password"
    
    # SSH private key to connect to VMware hypervisors for SSH transformation 1
    v2v_ssh_private_key: |
      -----BEGIN RSA PRIVATE KEY-----
      b3BlbnNzaC1rZXktdjEAAAAABG5vbmUAAAAEbm9uZQAAAAAAAAABAAAAlwAAAAdzc2gtcn....
      -----END RSA PRIVATE KEY-----
1
This is the private key of the SSH key pair that you created when you enabled SSH access on the VMware hypervisors.
Important

If the Red Hat Virtualization conversion host has an existing SSH private key, the v2v_ssh_private_key value does not overwrite it. You must delete the old key manually in /var/lib/vdsm/.ssh/id_rsa before running the conversion_host_enable playbook.

2.3.2.3. Configuring and verifying the conversion hosts

  1. Run the conversion_host_enable.yml playbook to configure each conversion host:

    # ansible-playbook -i conversion_host, -b \ 1
        -e "ansible_ssh_private_key_file=/etc/pki/ovirt-engine/keys/engine_id_rsa" \
        -e @extra_vars.yml -e @secure_vars.yml --ask-vault-pass \
        /usr/share/ovirt-ansible-v2v-conversion-host/playbooks/conversion_host_enable.yml
    1
    conversion_host is the FQDN or IP address of the conversion host.
    Caution

    Running the conversion_host_enable.yml playbook more than once on the same host creates multiple entries in the CloudForms database for that host.

    If you need to run the conversion_host_enable.yml playbook again, remove the host’s existing database entry by running conversion_host_disable.yml on the host:

    # ansible-playbook /usr/share/ovirt-ansible-v2v-conversion-host/playbooks/conversion_host_disable.yml
  2. Run the conversion_host_check.yml playbook to verify the configuration:

    # ansible-playbook --ask-vault-pass -i conversion_host, -c local \ 1
        -e @extra_vars.yml -e @secure_vars.yml conversion_host_check.yml
    1
    conversion_host is the FQDN or IP address of the conversion host.

If you are using VDDK, you can migrate your virtual machines. (Optionally) You can verify the conversion hosts in a browser. See Section 2.3.6, “Verifying the conversion hosts in a browser”.

If you are using SSH, copy the VMware keys to the conversion hosts. See Section 2.3.3, “Copying the VMware SSH keys to the conversion hosts”.

2.3.3. Copying the VMware SSH keys to the conversion hosts

You can copy the VMware SSH public keys to the conversion hosts.

The VMware keys can be collected either when you configure the VMware hypervisors for SSH transformation (see Section 2.3.3.1, “Copying keys collected during VMware hypervisor configuration”) or by using ssh-keyscan (see Section 2.3.3.2, “Copying keys collected with ssh-keyscan).

2.3.3.1. Copying keys collected during VMware hypervisor configuration

  1. Copy the VMware keys to /var/lib/vdsm/.ssh/known_hosts on each conversion host.
  2. Verify the SSH connection by connecting to each VMware hypervisor as vdsm:

    # sudo -u vdsm ssh root@_esx1.example.com_ 1
    1
    esx1.example.com is the host name of the VMware hypervisor.
    Important

    If the connection fails, check that the VMware hypervisor is configured for SSH and that you have used the correct keys.

2.3.3.2. Copying keys collected with ssh-keyscan

Caution

You must run ssh-keyscan for each VMware hypervisor. Otherwise your conversion hosts will not have all the VMware hypervisor keys and the migration will fail.

  1. Run ssh-keyscan for each VMware hypervisor and copy its public key to known_hosts, as in the following example:

    # ssh-keyscan esx1_IP > /var/lib/vdsm/.ssh/known_hosts 1
    # ssh-keyscan esx2_IP >> /var/lib/vdsm/.ssh/known_hosts
    # ssh-keyscan esx3_IP >> /var/lib/vdsm/.ssh/known_hosts
    1
    You must use the IP address, not the host name, of the VMware hypervisor.
  2. Change the ownership of the known_hosts file to vdsm user and kvm group:

    # chown 36:36 /var/lib/vdsm/.ssh/known_hosts
  3. Verify the SSH connection by connecting to each VMware hypervisor as vdsm:

    # sudo -u vdsm ssh root@_esx1.example.com_ 1
    1
    esx1.example.com is the host name of the VMware hypervisor.
    Important

    If the connection fails, check that the VMware hypervisor is configured for SSH and that you have used the correct keys.

After you have copied all VMware keys to all conversion hosts, you can configure secure remote login to the VMware hypervisors.

2.3.4. Configuring secure remote login to the VMware hypervisors

You can configure secure remote login to the VMware hypervisors by using ssh-agent and ssh-add.

Important

You must configure and verify secure remote login for each VMware hypervisor.

Perform the following procedure on the Manager machine:

  1. Create an ssh-agent session for the vdsm user:

    # sudo -u vdsm ssh-agent

    ssh-agent creates the SSH_AUTH_SOCK and SSH_AGENT_PID environment variables:

    SSH_AUTH_SOCK=/tmp/ssh-socket_number/agent.pid; export SSH_AUTH_SOCK;
    SSH_AGENT_PID=139150; export SSH_AGENT_PID;
    echo Agent pid 139150;
  2. Copy the SSH_AUTH_SOCK variable from the output.
  3. Run ssh-add to authorize the vdsm user’s private key for the ssh-agent session:

    # sudo -u vdsm SSH_AUTH_SOCK=SSH_AUTH_SOCK_value ssh-add 1
    1
    SSH_AUTH_SOCK_value is the SSH_AUTH_SOCK environment variable.
  4. Connect to a VMware hypervisor to verify the secure remote login:

    # sudo -u vdsm \
        SSH_AUTH_SOCK=SSH_AUTH_SOCK_value ssh root@esx1.example.com 1 2
    1
    SSH_AUTH_SOCK_value is the SSH_AUTH_SOCK variable.
    2
    esx1.example.com is the host name of a VMware hypervisor.
    Important

    You must test the SSH connection to all VMware hypervisors.

    If a connection is unsuccessful, all migrations from this VMware hypervisor using SSH transformation will fail.

If the connection is successful, authenticate the conversion hosts in CloudForms. See Section 2.3.5, “Authenticating the conversion hosts in CloudForms”.

2.3.5. Authenticating the conversion hosts in CloudForms

Perform the following procedure for each conversion host:

  1. Click ComputeInfrastructureHosts and select a Red Hat Virtualization conversion host.
  2. Click the Configuration drop-down button and select Edit Selected items.
  3. In the Default tab of the Endpoints section, enter the Username and the Password for root.
  4. Click Validate and wait for validation to complete.
  5. Click Save.

You can migrate your virtual machines. (Optionally) You can verify the conversion hosts in a browser. See Section 2.3.6, “Verifying the conversion hosts in a browser”.

2.3.6. Verifying the conversion hosts in a browser

You can verify your conversion hosts in a browser by using the CloudForms API:

  1. In the address bar of a browser, enter the following:

    https://CloudForms_FQDN/api/conversion_hosts 1
    1
    CloudForms_FQDN is the FQDN of the CloudForms machine.

    A log-in screen is displayed.

  2. Enter your CloudForms Username and Password and click Sign in.

    The conversion hosts and their IDs are displayed in JSON format:

    {"name":"conversion_hosts","count":3,"subcount":3,"pages":1,"resources":[{"href":"https://cloudforms.example.com/api/conversion_hosts/10000000000001"},{"href":"https://cloudforms.example.com/api/conversion_hosts/10000000000002"},{"href":"https://cloudforms.example.com/api/conversion_hosts/10000000000003"}],"actions":[{"name":"create","method":"post","href":"https://cloudforms.example.com/api/conversion_hosts"},{"name":"edit","method":"post","href":"https://cloudforms.example.com/api/conversion_hosts"},{"name":"delete","method":"post","href":"https://cloudforms.example.com/api/conversion_hosts"}],"links":{"self":"https://cloudforms.example.com/api/conversion_hosts?offset=0","first":"https://cloudforms.example.com/api/conversion_hosts?offset=0","last":"https://cloudforms.example.com/api/conversion_hosts?offset=0"}}

Chapter 3. Migrating the virtual machines

Migrating the virtual machines involves the following key tasks:

  1. Mapping the resources of your VMware and Red Hat Virtualization environments. See Section 3.1, “Creating an infrastructure mapping”.
  2. Checking for migration conditions with prerequisites. See Section 3.2, “Checking for migration prerequisites”.
  3. Creating and running a migration plan. See Section 3.3, “Creating and running a migration plan”.

    (Optional) You can change the maximum number of concurrent migrations for conversion hosts or providers to control the migration process. See Section 3.3.5, “Changing the maximum number of concurrent migrations”.

3.1. Creating an infrastructure mapping

The infrastructure mapping maps the resources of your VMware and Red Hat Virtualization environments.

Important

If you add or remove providers or provider objects from an infrastructure mapping, the mapping will have missing resource errors. You must delete the infrastructure mapping and create a new one.

Procedure

  1. Click ComputeMigrationInfrastructure Mappings.
  2. Click Create Infrastructure Mapping.
  3. Perform the following procedures in the Create Infrastructure Mapping wizard:

Creating infrastructure mapping

ScreenProcedure

General

  1. Enter the infrastructure mapping Name and (optional) Description.
  2. Select the Target Provider.
  3. Click Next.

Map Compute

  1. Select a Source Provider \ Datacenter \ Cluster and a Target Provider \ Datacenter \ Cluster.

    If the target cluster does not contain a conversion host, a warning icon ( warning ) appears. You can create and save an infrastructure mapping, but you must configure the conversion hosts before running a migration plan.

  2. Click Add Mapping. You can map additional clusters.
  3. Click Next.

Map Storage

  1. Select a source cluster whose datastores you want to map.
  2. Select a Source Provider \ Datacenter \ Datastore and Target Datastores.
  3. Click Add Mapping. You can map additional datastores.
  4. Click Next.

Map Networks

  1. Select a source cluster from the drop-down list.
  2. Select one or more networks from Source Provider \ Datacenter \ Network and Target Project \ Network.
  3. Click Add Mapping. You can map the networks of additional source clusters.
  4. Click Create.

Results

Click Close. Your infrastructure mapping is saved in ComputeMigrationInfrastructure Mappings.

You can click an infrastructure mapping element to view its details:

Infrastructure Mappings list

Infra mappings list

After you have created an infrastructure mapping, you can check for migration prerequisites. If these conditions do not apply, you can create a migration plan. See Section 3.3, “Creating and running a migration plan”.

3.2. Checking for migration prerequisites

Check your migration for the following conditions, which have prerequisites:

You are migrating previously migrated virtual machines

You must add previously migrated machines to the migration plan with a CSV file. A CSV file is also recommended for large migrations.

See Section 3.2.1, “Creating a CSV file to add virtual machines to the migration plan”.

You are using Ansible playbooks for premigration/postmigration tasks

You must create an Ansible repository and add credentials and playbooks to CloudForms.

See Section 3.2.2, “Adding Ansible playbooks to CloudForms for premigration and postmigration tasks”.

You are migrating virtual machines running RHEL or other Linux operating system

You must create a RHEL premigration playbook to preserve IP addresses after migration. This playbook is selected when you create a migration plan.

See Section 3.2.2, “Adding Ansible playbooks to CloudForms for premigration and postmigration tasks” and Section 3.2.3, “Creating a RHEL premigration playbook for RHEL/Linux source virtual machines”.

3.2.1. Creating a CSV file to add virtual machines to the migration plan

If you are migrating virtual machines that were migrated in the past, you should create a CSV file to add the virtual machines to the migration plan, because the migration plan cannot discover them automatically.

Note

A CSV file is recommended for large migrations because it is faster than manually selecting individual virtual machines.

Table 3.1. CSV file fields

FieldComments

Name

Virtual machine name. Required

Host

Optional. Only required if virtual machines have identical Name fields.

Provider

Optional. Only required if virtual machines have identical Name and Host fields.

CSV file example

Name,Host,Provider
vm01,host1,vSphere3
vm02,host1,vSphere3
vm03,host1,vSphere3

3.2.2. Adding Ansible playbooks to CloudForms for premigration and postmigration tasks

You can add Ansible playbooks to CloudForms to perform automated premigration and postmigration tasks on specific virtual machines, for example:

  • Removing webservers from a load-balancing pool before migration and returning them to the pool after migration
  • Running fstrim after migration to reduce the space required by virtual machines migrating to Red Hat OpenStack Platform with Ceph storage

Procedure

  1. Enable the Embedded Ansible server role in CloudForms. See Enabling the Embedded Ansible Server Role in Red Hat CloudForms: Managing Providers.
  2. Add an Ansible playbook repository. See Adding a Playbook Repository in Red Hat CloudForms: Managing Providers.
  3. Add the credentials of each virtual machine that you are migrating. See Credentials in Red Hat CloudForms: Managing Providers.
  4. Add your playbook as an Ansible service catalog item. See Creating an Ansible Playbook Service Catalog Item in Red Hat CloudForms: Provisioning Virtual Machines and Instances.

You will select the playbooks and the virtual machines on which they run in the Advanced Options screen when you create the migration plan.

3.2.3. Creating a RHEL premigration playbook for RHEL/Linux source virtual machines

If you are migrating virtual machines running RHEL or other Linux operating system, you can create a RHEL premigration playbook to ensure that the IP addresses are accessible after migration. The RHEL premigration playbook calls the Ansible ims.rhel_premigration role.

To install the role with Ansible Galaxy, see ims_rhel_pre_migration. This role is not included in the IMS installation.

The ims.rhel_premigration role performs the following tasks on the VMware virtual machines:

  • Preserves the static IP address configuration by creating udev rules to associate the virtual machine’s MAC address with its interface name
  • Installs the Red Hat Virtualization guest agent. The guest agent reports the new virtual machine’s IP address and installed applications to the Manager.
Note

The ims.rhel_premigration role assumes that either the rhel-6-server-rpms or the rhel-7-server-rpms repository is enabled in the source virtual machine when it installs qemu-guest-agent. If you have disabled the repository, re-enable it in the RHEL premigration playbook.

RHEL premigration playbook example

---
- hosts: all
  roles:
    - role: ims.rhel_pre_migration

You can create a migration plan. See Section 3.3, “Creating and running a migration plan”.

3.3. Creating and running a migration plan

Before you perform a large migration, you should perform several test migrations with different maximum numbers of concurrent migrations for your conversion hosts or providers. This will enable you to assess the capabilities of your environment’s infrastructure.

You can create and run a migration plan in CloudForms with the following options:

Note

A CSV file is optional, but recommended, for large migrations because it is faster than manually selecting each virtual machine.

  1. Click ComputeMigrationMigration Plans.
  2. Click Create Migration Plan.
  3. Perform the following procedures in the Create Migration Plan wizard:

Create Migration Plan screen

ScreenProcedure

General

  1. Select an infrastructure mapping from the drop-down list.
  2. Enter the migration plan Name and (optional) Description.
  3. Select a virtual machine discovery method:

    • Choose from a list of VMs discovered in the selected infrastructure mapping

      If the virtual machines cannot be discovered, check that the source datastores and networks in the infrastructure mapping are correct.

    • Import a CSV file with a list of VMs to be migrated.

      A CSV file is required for previously migrated source virtual machines and recommended for large migrations.

  4. Click Next.

VMs

  • If you selected Choose from a list of VMs discovered in the selected infrastructure mapping:

    • Select the virtual machines for migration.

      You can search for virtual machines by VM Name, Data Center, Cluster, and Folder.

  • If you selected Import a CSV file with a list of VMs to be migrated:

    1. Click Import.
    2. Browse to the CSV file and click Open.

      If the virtual machines cannot be added to the migration plan, check the CSV file format and fields for errors.

      If the Create Migration Plan wizard freezes, refresh the web page, check the CSV file for errors (for example, virtual machines with duplicate Name fields and no other fields to distinguish them), and create a new migration plan.

    3. Click Next.

Schedule

  1. Select a schedule option:

    • Save migration plan to run later

      The migration plan is saved in Migration Plans Not Started and will not run unless you schedule it or click Migrate to run the scheduled migration plan immediately.

    • Start migration immediately

      The migration plan may take some time to complete. Progress bars indicate the amount of transferred data, the number of migrated virtual machines, and the elapsed time. See Section 3.3.2, “Viewing a migration plan in progress” for details.

      Optionally, you can cancel a migration plan in progress.

  2. Click Create.

Results

Click Close.

When the migration plan has finished, click Migration Plans Complete to view the status of the migration plan. The completed migration plan shows the status of the migrated virtual machines.

In the migration plans list, you can click the More Actions icon ( 7 ) to archive, edit, or delete a migration plan.

3.3.1. Scheduling a saved migration plan

To schedule a saved migration plan to run in the future:

  1. Click Migration Plans Not Started.
  2. Click the Schedule button of a migration plan.
  3. In the Schedule Migration Plan window, select a date and time and click Schedule. The plan’s status is Migration Scheduled with the date and time.

3.3.2. Viewing a migration plan in progress

To view the progress of a migration plan:

  1. Click Migration Plans in Progress.
  2. Click a migration plan name to view its details, including the status of the migrating virtual machines.
Note

The counter in ComputeMigrationMigration Plans may be a few seconds ahead of the counter in the migration plan details view. This is because the Migration Plans counter displays the total time for running the migration plan, while the details counter displays the time for migrating the virtual machines.

3.3.3. Canceling a migration plan in progress

To cancel a migration plan in progress:

  1. Click Migration Plans in Progress.
  2. Select a migration plan and click Cancel Migration.
  3. Click Cancel Migrations to confirm the cancellation. The canceled migration appears in Migration Plans Complete with a red x indicating that the plan did not complete successfully.

3.3.4. Retrying a failed migration plan

To retry a migration plan that failed because of external circumstances (for example, power outage):

  1. Delete all objects created by the failed migration plan:

    • Delete newly created virtual machines to avoid name conflicts with migrating VMware virtual machines.
    • Delete converted disks to free up space.
  2. Click ComputeMigrationMigration Plans.
  3. Click Migration Plans Complete.
  4. Click the Retry button beside the failed migration plan.

3.3.5. Changing the maximum number of concurrent migrations

You can change the maximum number of concurrent migrations for conversion hosts or providers to control the impact of the migration process on your infrastructure.

The provider setting has priority over the conversion host setting. For example, if the maximum number of concurrent migrations is 20 for a provider and 3 for five conversion hosts, the maximum number of concurrent migrations is 20, not 15 (5 conversion hosts x 3 concurrent migrations per host).

An increase in the maximum number of concurrent migrations affects all migration plans immediately. Virtual machines that are queued to migrate will migrate in greater numbers.

A decrease maximum number of concurrent migrations affects only future migration plans. Migration plans that are in progress will use the limit that was set when the plan was created.

Caution

If you are using VDDK transformation, the number of concurrent migrations must not exceed 20. Otherwise, network overload will cause the migration to fail.

Changing the maximum number of concurrent migrations for all conversion hosts
  1. In CloudForms, click ComputeMigrationMigration Settings.
  2. Select a new Maximum concurrent migrations per conversion host. The default is 10.

    Note

    In the current release, the Maximum concurrent migrations per conversion host interface control does not work.

Changing the maximum number of concurrent migrations for a provider
  1. Log in to the CloudForms machine using SSH.
  2. Enter the following command:

    # vmdb
    # rails console
    irb(main):001:0> $evm = MiqAeMethodService::MiqAeService.new(MiqAeEngine::MiqAeWorkspaceRuntime.new)
    irb(main):002:0> $evm.vmdb(:ext_management_system).find_by(:name => "RHV").custom_set("Max Transformation Runners", 30) 1
    1
    Max Transformation Runners is the maximum number of concurrent migrations. The default value is 20 for a provider.
Getting the maximum number of concurrent migrations for a provider
  1. Log in to the CloudForms machine using SSH.
  2. Enter the following command:

    # vmdb
    # rails console
    irb(main):001:0> $evm = MiqAeMethodService::MiqAeService.new(MiqAeEngine::MiqAeWorkspaceRuntime.new)
    irb(main):002:0> $evm.vmdb(:ext_management_system).find_by(:name => "RHV").custom_get("Max Transformation Runners")

Chapter 4. Troubleshooting

To identify errors, you can review the migration logs. See Section 4.1, “Migration logs”.

You can check these common issues and mistakes:

Section 4.7, “Known Issues” provides information about issues that will be addressed in a future release.

4.1. Migration logs

You can check the following logs identify the cause of a migration error:

The conversion host logs and the CloudForms migration log can help with troubleshooting.

4.2. Accessing the conversion host logs

When disk migration starts, two logs are created in the conversion host:

  • virt-v2v: Debug output from virt-v2v itself. This log tracks the core of the virtual machine migration process, including libguestfs traces and disk migration details. You can download access this log on the conversion host or download it in CloudForms.
  • virt-v2v-wrapper: Log of the daemonizing wrapper for virt-v2v. This log traces the orchestration of the virtual machine conversion on the conversion host, including disk migration percentages and virt-v2v error reporting. You can access this log on the conversion host.
Important

If you need to open a Red Hat Support call, you must submit both the migration (virt-v2v) log and virt-v2v-wrapper log for analysis.

To access the virt-v2v and virt-v2v-wrapper logs on the conversion host:

  1. Log in to the conversion host using SSH.

    If you are not sure which conversion host to log in to, click the information icon ( 20 ) of a virtual machine in the migration plan details view.

  2. Go to /var/log/vdsm/import/ to access the logs for each migration:

    • virt-v2v log: v2v-import-date-log_number.log
    • virt-v2v-wrapper log: v2v-import-date-log_number-wrapper.log

To download the virt-v2v log in CloudForms:

  1. Click ComputeMigrationMigration Plans.
  2. Click a completed migration plan to view its details.
  3. Click Download LogMigration Log.

4.2.1. Accessing the CloudForms migration log

This log traces the orchestration of the virtual machine migration in CloudForms.

To access the CloudForms migration log:

  1. Log in to the CloudForms machine using SSH.
  2. The migration log is /var/www/miq/vmdb/log/automation.log.

4.3. Infrastructure mapping errors

  • Networks missing, Datastores missing, and Clusters missing error messages: If you create an infrastructure mapping and then change a provider or refresh the Red Hat Virtualization hosts, the provider’s object IDs change. Delete the infrastructure mapping and create a new one.

4.4. Migration plan errors

  • If the virtual machines are being migrated for the first time and are not discovered by the migration plan, check the source datastores and networks in the infrastructure mapping.
  • If the virtual machines have been migrated in the past, they cannot be discovered by the migration plan. Use a CSV file to add the virtual machines to the migration plan.
  • If the virtual machines cannot be added to the migration plan with a CSV file, check the CSV file format and fields. Create a new migration plan with the updated CSV file.
  • Create Migration Plan wizard hangs while importing a CSV file. This error is caused by an invalid CVS file (for example, virtual machines with a duplicate Name field and no Host/Provider field to distinguish them, or with a duplicate Name field and duplicate Host/Provider fields). Refresh the web page.
  • Denied State error (IMS 1.1). If a migration plan fails immediately and the migration plan displays a Denied State error message, check that you have created and configured the conversion hosts correctly. Cancel the migration plan and run it again.
  • Unable to migrate VMs because no conversion host was configured at the time of the attempted migration. See the product documentation for information on configuring conversion hosts. (IMS 1.2) You can create and save a migration plan whose infrastructure mapping does not contain conversion hosts, but you cannot run the migration plan without conversion hosts. Cancel the migration plan, create the conversion hosts, and run the migration plan again.

4.5. IP address errors

  • If the IP address of a migrated RHEL (or other Linux-based operating system) virtual machine is not accessible, you must create a RHEL premigration playbook and add it to the migration plan.
  • If a migrated virtual machine does not have an IP address:

    • Check that you installed VMware Tools on the VMware virtual machine before migration.

4.6. Environment configuration errors

VMware

  • A VMware virtual machine cannot be migrated if it has any of the following conditions:

    • Mounted ISO/CDROM disk
    • Encrypted disk
    • Invalid name, containing spaces or special characters
  • VDDK transformation only: To perform more than ten concurrent migrations from a single VMware hypervisor, you must increase the hypervisor’s NFC service memory.

Red Hat Virtualization

  • Name conflict: VMware virtual machine has the same name as a Red Hat Virtualization virtual machine.
  • MAC address conflict: VMware virtual machine has the same MAC address as a Red Hat Virtualization virtual machine in a MAC address pool.
  • SSH transformation only:

    • If you are using SSSD with single sign-on, you must reinstall ipa-client without OpenSSH.
    • Check that the conversion host does not have an existing private SSH key in /var/lib/vdsm/.ssh/id_rsa. Conversion host configuration does not overwrite old SSH keys. They must be deleted manually.
    • Check that you enabled SSH access on the VMware hypervisors and correctly configured your conversion hosts for SSH transformation.

4.7. Known Issues

The following issues will be addressed in a future release:

Part II. Migrating from VMware to Red Hat OpenStack Platform

The migration process involves the following tasks:

  1. Planning the migration. See Chapter 5, Planning the migration.
  2. Preparing the VMware, Red Hat OpenStack Platform, and CloudForms environments. See Chapter 6, Preparing the environment for migration.
  3. Migrating the virtual machines. See Chapter 7, Migrating the virtual machines.
  4. Troubleshooting, if necessary. See Chapter 8, Troubleshooting.

Chapter 5. Planning the migration

During the planning phase, you will formulate a specific migration goal, for example, "I want to migrate 2000 virtual machines, with 200 TB of data, in less than 6 months".

Review the following information to plan your migration:

The migration workflow describes the migration process in greater detail.

Figure 5.1. VMware to Red Hat OpenStack Platform migration workflow

vmware to osp migration workflow

25 You create and run a migration plan in CloudForms.

25 CloudForms uses the migration plan to locate the source virtual machines.

25 If you are using VDDK transformation to migrate your virtual machines, CloudForms captures the ESXi host fingerprint for authentication during the virtual machine conversion process.

25 Using the attributes defined for the Red Hat OpenStack Platform environment, CloudForms initiates communication with the conversion hosts (Red Hat OpenStack Platform instances created from a conversion host appliance, with virt-v2v and virt-v2v-wrapper installed).

25 virt-v2v-wrapper connects to the source datastore through the ESXi host. virt-v2v streams the source disks to the target data domain and converts the source disks.

25 After the source disks are converted, virt-v2v detaches the volumes from the conversion host, migrates the volumes to the destination project, and creates the network ports defined in the infrastructure mapping.

25 virt-v2v-wrapper creates the target Red Hat OpenStack Platform instance with the flavor and security group defined in the migration plan. virt-v2v attaches the newly created network ports and the disks mapped in the block storage to the instance and the instance is powered on.

The migration process is complete and the migration plan’s status is displayed in CloudForms.

5.1. Questions to ask before migration

The following questions can help you to estimate the resources and time required for migration.

What am I migrating?
  • Identify the VMware virtual machines that you will be migrating.
What is the maximum number of disks or virtual machines that I can migrate?
  • There is no maximum number of disks or virtual machines that you can migrate. However, you may not want to migrate all your virtual machines at the same time, in order to minimize the impact on your users.

    Important

    If you exceed the capabilities of your environment, the migrations will fail. This situation could affect existing applications running on virtual machines attached to the network and storage.

What operating systems can I migrate?
What am I missing?
  • Identify resource gaps, such as bandwidth, storage, licenses, or a suitable maintenance window, before you begin the migration.
What impact will the migration have on my users?
  • Assess the effects the migration may have on a production environment.

    It may be possible to migrate your applications in phases, without downtime at the application layer, if the applications are distributed in a high-availability architecture.

  • Check whether users will lose access to critical applications.
How long will the migration take?

There is no formula to estimate how long the actual migration will take. This is determined on a case-by-case basis.

The following example is provided as a guide:

  • Duration of migration: 2:13:00 (hh:mm:ss)
  • 20 virtual machines
  • 2 conversion hosts, maximum of 10 concurrent conversions
  • Total data migrated, using VDDK: 1000 GB
  • Hardware:

    • Strong host (40 cores, 500 GB RAM)
    • Fast SSD XtremIO storage
    • Fibre Channel 8 interface for host-to-storage connection
    • 10 GbE network interface cards for all other connections
How many conversion hosts do I need?

The number of conversion hosts you create depends on the size of your migration. All the virtual machines in a migration plan are migrated at the same time, in parallel. The number of virtual machines that you can migrate simultaneously depends on your infrastructure capabilities. Each migration requires a certain amount of network bandwidth, I/O throughput, and processing power for the conversion process.

Multiple conversion hosts provide load-balancing and better performance, even for small migrations.

Conversion hosts are limited to a maximum of ten concurrent migrations, unless you change the default values.

You should test your environment thoroughly before the migration to determine how many migrations it can support without negative effects, for example, five conversion hosts, each running ten concurrent migrations.

Should I migrate my virtual machines with VDDK or SSH?

You can configure your conversion hosts to migrate the virtual machines by using either the VMware Virtual Disk Development Kit (VDDK) or SSH. The choice is determined by the number of virtual machines you are migrating and the capabilities of your infrastructure.

The following table compares the options.

Table 5.1. VDDK and SSH comparison

 ProsCons

VDDK

  • Recommended because it is approximately twice as fast as SSH
  • Compiled with nbdkit, which provides direct connectivity to the virtual machine disks
  • Requires VDDK package download
  • Requires VDDK package location for conversion host configuration
  • Limited to 20 concurrent migrations per conversion host
  • Limited to 10 concurrent migrations per VMware hypervisor unless you increase the hypervisor’s NFC service memory

SSH

  • Better suited for large-scale migrations than VDDK (not affected by hypervisor’s NFC service memory limitation)
  • Does not require additional downloads, packages, tools, or compilation
  • Slower than VDDK
  • Requires VMware hypervisors to have SSH access enabled
  • Requires VMware hypervisors and conversion hosts to have shared SSH keys

5.2. Recommendations and best practices

The following recommendations will help to minimize the impact of the migration on your environment.

Scheduling the migration

  • Schedule your migration carefully, to minimize the impact on your users.
  • Prepare your users for downtime.

    Important

    Currently, IMS supports only cold migration. Virtual machines are powered off gracefully as part of the migration process.

  • Stagger the migration schedules.
  • Move critical applications during maintenance windows.

Distributing the migration workload

  • Create migration groups, so that you are not migrating all of your virtual machines at the same time, keeping in mind the following considerations:

    • How are the virtual machines grouped now?
    • Which virtual machines should be migrated together?
    • Which workloads or linked applications should be migrated together?
    • What applications must remain available?
  • Consider which parts of the workload to migrate first:

    • Databases
    • Applications
    • Web servers
    • Load balancers

Deploying the conversion hosts

  • Create a sufficient number of conversion hosts for your migration, with sufficient resources.
  • Choose the best transformation option (SSH or VDDK) for your environment.

Controlling the migration process

  • Create multiple migration plans for finer control.
  • Perform test migrations with different maximum numbers of concurrent migrations to assess the capabilities of your environment’s infrastructure.

Chapter 6. Preparing the environment for migration

6.1. Preparing the VMware environment

Preparing the VMware environment for migration involves the following tasks:

  1. Extending the VMware network. See Section 6.1.1, “Preparing the VMware network”.
  2. Preparing the VMware virtual machines. See Section 6.1.2, “Preparing the VMware virtual machines”.
  3. Configuring the VMware hypervisors if you are using SSH transformation. See Section 6.1.3, “Preparing the VMware hypervisors for SSH transformation”.
  4. Configuring a VMware hypervisor if you are using VDDK and performing more than ten concurrent migrations from that hypervisor. See Section 6.1.4, “Configuring a VMware hypervisor for more than ten concurrent VDDK migrations”.

6.1.1. Preparing the VMware network

Important
  • The network configuration must not be changed in any way during the migration.
  • IP addresses, VLANs, and other network configuration must not be changed before or after migration because the conversion process preserves the source virtual machine MAC addresses.

You can prepare the VMware virtual machines. See Section 6.1.2, “Preparing the VMware virtual machines”.

6.1.2. Preparing the VMware virtual machines

Perform the following steps on each VMware virtual machine that you are migrating:

  1. Install VMware Tools to capture IP addresses.

    To download and install VMware Tools, see VMware Workstation 5.0: Installing VMware Tools.

  2. Unmount mounted ISO/CDROM disks.
  3. Ensure that attached disks are not encrypted.
  4. Ensure that each NIC has no more than one IPv4 and/or one IPv6 address.
  5. Ensure that the virtual machine names contain only upper- or lower-case letters, numbers, underscores (_), hyphens (-), or periods (.).

    Note

    International characters and spaces are not permitted.

  6. Ensure that the virtual machine names do not duplicate names of virtual machines in the Red Hat OpenStack Platform tenant.

If you are using SSH transformation, you can configure the VMware hypervisors. See Section 6.1.2, “Preparing the VMware virtual machines”.

If you are using VDDK transformation and performing more than ten concurrent migrations from a VMware hypervisor, you must configure the hypervisor to support the additional connections. See Section 6.1.4, “Configuring a VMware hypervisor for more than ten concurrent VDDK migrations”.

6.1.3. Preparing the VMware hypervisors for SSH transformation

If you are using SSH transformation, you can configure the VMware hypervisors for passwordless access by sharing a conversion host’s public SSH key with the hypervisors.

Important

A single SSH key pair is recommended because the key pair is used only for virtual machine conversion and it simplifies conversion host management.

If you wish to use a dedicated SSH key pair for each conversion host, you can copy the public key of each conversion host to all the VMware hypervisors.

Procedure

  1. Enable SSH access on each VMware hypervisor:

    For instructions, navigate to VMware vSphere Documentation. In the navigation pane, click vSphere versionESXi and vCenter ServerVMware ESXi Installation and SetupInstalling and Setting Up ESXiSetting Up ESXiEnable ESXi Shell and SSH Access with the Direct Console User Interface.

    Note

    If your security policies allow you to collect the SSH public keys of the VMware hypervisors at this stage, save them for copying to the conversion hosts.

  2. Generate an SSH key pair without a passphrase:

    # ssh-keygen -N ''
  3. Copy the public SSH key to /etc/ssh/keys-root/authorized_keys on each VMware hypervisor.

    You will use the private SSH key to configure the conversion hosts.

6.1.4. Configuring a VMware hypervisor for more than ten concurrent VDDK migrations

If you are performing more than ten concurrent migrations from a VMware hypervisor using VDDK transformation, the migration will fail because the hypervisor’s NFC service memory buffer is limited to ten parallel connections. See VMware vSphere 6.5 NFC session connection limits and Virt-v2v. VDDK: ESXi NFC service memory limits for details.

You can increase the hypervisor’s NFC service memory to enable additional connections for migrations.

Procedure

  1. Log in to a VMware hypervisor.
  2. Change the value of maxMemory to 1000000000 in /etc/vmware/hostd/config.xml:

          <nfcsvc>
             <path>libnfcsvc.so</path>
             <enabled>true</enabled>
             <maxMemory>1000000000</maxMemory>
             <maxStreamMemory>10485760</maxStreamMemory>
          </nfcsvc>
  3. Restart hostd:

    # /etc/init.d/hostd restart

    You do not need to reboot the VMware hypervisor.

6.2. Preparing the Red Hat OpenStack Platform environment

Preparing the Red Hat OpenStack Platform environment involves the following key steps:

  1. Installing and configuring Red Hat OpenStack Platform 13 or 14. See Section 6.2.1, “Installing and configuring Red Hat OpenStack Platform 13 or 14”.
  2. Installing and configuring CloudForms 4.7.3. See Section 6.2.2, “Installing and configuring CloudForms 4.7.3”.
  3. Creating the conversion hosts. See Section 6.2.3, “Creating the conversion hosts”.

Prerequisites

The following prerequisites apply:

  • BIOS settings of physical hosts adjusted for optimal performance (rather than power-saving), according to the vendor’s recommendations
  • C1E halt state disabled, if applicable
  • Compatible software versions:

    Table 6.1. Software compatibility

    SoftwareVersion

    VMware

    5.5 or later

    CloudForms

    4.7.3, with CFME 5.10.3

    CFME 5.10.4 does not support migration.

    Red Hat OpenStack Platform

    13 or 14

    RHOSP V2V Image for Red Hat OpenStack Director

    14.0.2

6.2.1. Installing and configuring Red Hat OpenStack Platform 13 or 14

  1. Install Red Hat OpenStack Platform 13 or 14. See Red Hat OpenStack Platform Director Installation and Usage 13 or Red Hat OpenStack Platform Director Installation and Usage 14.
  2. Create provider networks for the target instances to preserve the IP addresses of the source virtual machines. See Create a network in the Red Hat OpenStack Platform Networking Guide.
  3. Create a project for the conversion hosts and whatever destination projects you require for the target instances. See Create a Project in the Red Hat OpenStack Platform Users and Identity Management Guide.
  4. Ensure that the admin user has member and admin roles in the conversion and destination projects. See Edit a Project in the Red Hat OpenStack Platform Users and Identity Management Guide.
  5. Set at least one volume type for the target block storage. See Create a Volume and Changing a Volume’s Type (Volume Re-typing) in the Red Hat OpenStack Platform Storage Guide. Otherwise, CloudForms cannot detect the storage when you create the infrastructure mapping.
  6. Ensure that the storage backends have sufficient space for the migrated virtual machines.

    Important

    If you are using Red Hat Ceph Storage, you will require three times the space of the source virtual machines for the migrated virtual machines. A Ceph storage cluster, by default, creates two copies of an object in a replicated storage pool, for a total of three copies. See Data Copies in the Red Hat Ceph Storage Architecture Guide.

    The migrated disks use all of the space because it is preallocated. For example, a source virtual machine with a 100 GB disk requires 300 GB of storage, regardless of how much data the disk actually contains. To save storage space, you can use the fstrim command on the migrated virtual machines as a postmigration task or playbook.

  7. Configure security groups with the following ports enabled:

    • For the conversion hosts and CloudForms: port 22 (SSH)
    • For CloudForms: port 443 (HTTPS)

      Note

      Outbound traffic is enabled by default. If you have changed this setting, enable ports 902 (CloudForms to VMware) and 5480 (conversion hosts to vCenter).

  8. Create flavors for the source virtual machines. If you do not create custom flavors, CloudForms will try to map each source virtual machine to an existing flavor.

6.2.2. Installing and configuring CloudForms 4.7.3

  1. Install Red Hat CloudForms 4.7.3 (CFME 5.10.3). See Installing Red Hat CloudForms on Red Hat OpenStack Platform.

    Note

    CFME 5.10.4 does not support migration.

  2. Add VMware to CloudForms as a provider. See Adding a VMware vCenter Provider in Red Hat CloudForms: Managing Providers.
  3. Add Red Hat OpenStack Platform to CloudForms as a provider. See Adding an OpenStack Infrastructure Provider in Red Hat CloudForms: Managing Providers.

    Note

    Removing or changing a provider before, during, or after the migration will cause errors in the infrastructure mappings and migration plans.

6.2.3. Creating the conversion hosts

You can create the Red Hat OpenStack Platform conversion hosts with the conversion appliance (RHOSP V2V Image for Red Hat OpenStack Director). The number of conversion hosts you deploy depends on your migration size and infrastructure capabilities.

Note

For optimal performance:

To create a Red Hat OpenStack Platform conversion host:

  1. Navigate to Red Hat Product Downloads.
  2. In the A-Z tab, click Red Hat OpenStack Platform.
  3. Click the green Download Latest button to go to the Download Red Hat OpenStack Platform page.
  4. In the Product Software tab, locate RHOSP V2V Image for Red Hat OpenStack Director 14.0.2 (x86_64), click the Download Now button, and save the image.

    Important

    Versions 14.0.3 or later cannot be used with IMS 1.1.

  5. Upload the image to Glance storage.
  6. Launch the image as a conversion host instance, with the following resources:

    • 4 vCPUs
    • 1 GB RAM for each concurrent migration, starting at 10 GB RAM. The default maximum number of concurrent migrations per conversion host is 10. If you raise the number of concurrent migrations, you should increase the RAM accordingly. If you lower the number of concurrent conversions, you should not go below 8 GB RAM.
    • /tmp (1 GB for each concurrent migration)
    • /var/tmp (1 GB for each concurrent migration)
    • /var/logs (5 GB)

      See Launch an Instance in the Red Hat OpenStack Platform Instances and Images Guide.

  7. Increase the disk space of the instance to accommodate its file system. See Resize an Instance in the Red Hat OpenStack Platform Instances and Images Guide.

    The instance is created from an image, but the disk space defined in the image will not be sufficient. You can either extend the partition (and subsequently, extend the physical volume in the volume group) to the required size, or you can create a new partition and add it as a physical volume to the volume group.

    Note

    You must resize lv_root to use all available disk space because the image will not use it by default.

Add the conversion host’s private SSH key to CloudForms:

  1. In CloudForms, click ComputeCloudsProviders.
  2. Select the Red Hat OpenStack Platform provider, click Configuration and select Edit Selected Cloud Provider.
  3. Click the RSA key pair tab in the Endpoints section.
  4. Enter the Username cloud-user and the Private Key of the conversion host instance. If you deploy multiple conversion hosts, they must use the same key pair.
  5. Click Save.

6.3. Configuring the conversion hosts for VDDK or SSH transformation

Perform the following steps to configure your conversion hosts for VDDK or SSH transformation:

VDDKSSH
  1. Configure the conversion hosts.

    Section 6.3.2, “Configuring the conversion hosts with Ansible playbooks”

  2. Copy the VMware SSH keys to the conversion hosts.

    Section 6.3.3, “Copying the VMware SSH keys to the conversion hosts”

  3. Configure secure remote login to the VMware hypervisors.

    Section 2.3.4, “Configuring secure remote login to the VMware hypervisors”

  4. (Optional) Verify the conversion hosts in a browser.

    Section 6.3.4, “Verifying the conversion hosts in a browser”

6.3.1. Downloading VDDK

Download and save the VMware Virtual Disk Development Kit:

  1. Navigate to VMware Documentation.
  2. Click VMware SDK & API Product Documentation to expand.
  3. Click VMware Virtual Disk Development Kit (VDDK).
  4. Click Latest Releases and select the latest VDDK release.
  5. Click Download SDKs to download the VDDK archive file.
  6. Save the VDDK archive file in an HTTP-accessible location and record its path.

You can configure the conversion hosts with Ansible playbooks.

6.3.2. Configuring the conversion hosts with Ansible playbooks

You can configure your conversion hosts for VDDK or SSH transformation with Ansible playbooks:

  1. Create the extra_vars.yml file for the conversion host parameters. See Section 6.3.2.1, “Creating the extra_vars.yml file”.
  2. Create the secure_vars.yml file for secure variables. See Section 6.3.2.2, “Creating the secure_vars.yml file”.
  3. Configure and verify the conversion hosts by running the Ansible playbooks. See Section 6.3.2.3, “Configuring and verifying the conversion hosts”.

6.3.2.1. Creating the extra_vars.yml file

  1. Log in to a conversion host.
  2. In /usr/share/ovirt-ansible-v2v-conversion-host/playbooks, create an extra_vars.yml file with the following parameters:

    ---
    v2v_host_type: openstack
    
    # Transport methods to configure on the conversion host. Valid values: "vddk", "ssh"
    v2v_transport_methods:
      - vddk 1
    
    # Maximum number of concurrent conversions per host. Default is "10".
    v2v_max_concurrent_conversions: 10 2
    
    # File name of VDDK package
    v2v_vddk_package_name: "VMware-vix-disklib-version.x86_64.tar.gz" 3
    
    # URL of VDDK package
    v2v_vddk_package_url: "http://path/to/downloaded_vddk_package/{{ v2v_vddk_package_name }}" 4
    
    manageiq_provider_name: OpenStack
    
    # Base URL of CloudForms machine
    manageiq_url: "https://CloudForms_FQDN" 5
    
    # Whether to validate certificate of CloudForms server. Default is "true".
    manageiq_validate_certs: false 6
    
    manageiq_zone_id: "42000000000001" 7
    
    # Empty vmware_hosts variable for conversion_host_disable.yml
    vmware_hosts: ""
    
    # List of cloud providers
    # Each provider is a dictionary with 3 attributes: "name", "hostname", and "connection_configurations"
    manageiq_providers:
      - name: "OpenStack"
        hostname: controller_node_FQDN_or_IP_address
        connection_configurations:
          - endpoint:
              role: "default"
              security_protocol: "ssl" 8
              certificate_authority: | 9
                -----BEGIN TRUSTED CERTIFICATE-----
                MIIDNzCCAh8CAQEwDQYJKoZIhvcNAQELBQAwYjELMAkGA1UEBhMCVV....
                -----END TRUSTED CERTIFICATE-----
                -----BEGIN TRUSTED CERTIFICATE-----
                MIIDlzCCAn+gAwIBAgIJAOP7AaT7dsLYMA0GCSqGSIb3DQEBCwUAMG....
                -----END TRUSTED CERTIFICATE-----
1
Select a transformation method, VDDK or SSH.
2
v2v_max_concurrent_conversions is the maximum number of concurrent conversions per host. The default is 10. If you are using VDDK transformation, do not set this number higher than 20.
3
Update the v2v_vddk_package_name with the correct version.
4
v2v_vddk_package_url is the path to the VDDK archive file that you downloaded.
5
manageiq_url is the FQDN of the CloudForms machine.
6
You can set manageiq_validate_certs to false if you do not want to validate the CloudForms CA certificate. The default value is true.
7
To obtain the manageiq_zone_id, enter this command on the CloudForms machine:
# curl -sk -u admin \'https://CloudForms_FQDN/api/zones/?filter\[\]=name=RHV&expand=resources&attributes=zone
8
You can specify the connection security: non-ssl, ssl-without-validation, or ssl. If you choose ssl, add the CA chain (certificate_authority).
9
The CA chain (certificate_authority) is a concatenation of two CA files:
  • /etc/pki/ca-trust/source/anchors/undercloud-cacert.pem on the undercloud server
  • /etc/pki/ca-trust/anchors/overcloud-cacert.pem on one of the overcloud controllers

    If you deploy your own CA chain, use the chain that signs the Red Hat OpenStack Platform API certificates. See SSL/TLS Certificate Configuration in Red Hat OpenStack Platform Director Installation and Usage.

6.3.2.2. Creating the secure_vars.yml file

  1. Create an encrypted secure_vars.yml file:

    # ansible-vault create secure_vars.yml
  2. Add the following parameters and save the file:

    ---
    # CloudForms "admin" user, for connecting to CloudForms
    manageiq_username: "username"
    
    # CloudForms "admin" password:
    manageiq_password: "password"
    
    # SSH private key to connect to VMware hypervisors for SSH transformation 1
    v2v_ssh_private_key: |
      -----BEGIN RSA PRIVATE KEY-----
      b3BlbnNzaC1rZXktdjEAAAAABG5vbmUAAAAEbm9uZQAAAAAAAAABAAAAlwAAAAdzc2gtcn....
      -----END RSA PRIVATE KEY-----
1
This is the private key of the SSH key pair that you created when you enabled SSH access on the VMware hypervisors.

6.3.2.3. Configuring and verifying the conversion hosts

  1. Run the conversion_host_enable.yml playbook to configure each conversion host:

    # ansible-playbook -i conversion_host, -c local -b \ 1
        -e @extra_vars.yml -e @secure_vars.yml --ask-vault-pass \
        /usr/share/ovirt-ansible-v2v-conversion-host/playbooks/conversion_host_enable.yml
    1
    conversion_host is the FQDN or IP address of the conversion host.
    Caution

    Running the conversion_host_enable.yml playbook more than once on the same host creates multiple entries in the CloudForms database for that host.

    If you need to run the conversion_host_enable.yml playbook again, remove the host’s existing database entry by running conversion_host_disable.yml on the host:

    # ansible-playbook /usr/share/ovirt-ansible-v2v-conversion-host/playbooks/conversion_host_disable.yml
  2. Run the conversion_host_check.yml playbook to verify the configuration:

    # ansible-playbook --ask-vault-pass -i conversion_host, -c local \ 1
        -e @extra_vars.yml -e @secure_vars.yml conversion_host_check.yml
    1
    conversion_host is the FQDN or IP address of the conversion host.

If you are using VDDK, you can migrate your virtual machines. (Optionally) You can verify the conversion hosts in a browser. See Section 6.3.4, “Verifying the conversion hosts in a browser”.

If you are using SSH, copy the VMware keys to the conversion hosts. See Section 6.3.3, “Copying the VMware SSH keys to the conversion hosts”.

6.3.3. Copying the VMware SSH keys to the conversion hosts

You can copy the VMware SSH public keys to the conversion hosts.

The VMware keys can be collected either when you configure the VMware hypervisors for SSH transformation (see Section 6.3.3.1, “Copying keys collected during VMware hypervisor configuration”) or by using ssh-keyscan (see Section 6.3.3.2, “Copying keys collected with ssh-keyscan).

6.3.3.1. Copying keys collected during VMware hypervisor configuration

  1. Copy the VMware keys to /root/.ssh/known_hosts on each conversion host.
  2. On each conversion host, verify the SSH connection by connecting to each VMware hypervisor as cloud-user:

    # sudo -u cloud-user ssh root@esx1.example.com 1
    1
    esx1.example.com is the host name of the VMware hypervisor.
    Important

    If the connection fails, check that the VMware hypervisor is configured for SSH and that you have used the correct keys.

6.3.3.2. Copying keys collected with ssh-keyscan

Caution

You must run ssh-keyscan for each VMware hypervisor. Otherwise your conversion hosts will not have all the VMware hypervisor keys and the migration will fail.

  1. Run ssh-keyscan for each VMware hypervisor and copy its public key to known_hosts, as in the following example:

    # ssh-keyscan esx1_IP > /root/.ssh/known_hosts 1
    # ssh-keyscan esx2_IP >> /root/.ssh/known_hosts
    # ssh-keyscan esx3_IP >> /root/.ssh/known_hosts
    1
    You must use the IP address, not the host name, of the VMware hypervisor.
  2. On each conversion host, verify the SSH connection by connecting to each VMware hypervisor as cloud-user:

    # sudo -u cloud-user ssh root@esx1.example.com 1
    1
    esx1.example.com is the host name of the VMware hypervisor.
    Important

    If the connection fails, check that the VMware hypervisor is configured for SSH and that you have used the correct keys.

You can migrate your virtual machines. (Optionally) You can verify the conversion hosts in a browser. See Section 6.3.4, “Verifying the conversion hosts in a browser”.

6.3.4. Verifying the conversion hosts in a browser

You can verify your conversion hosts in a browser by using the CloudForms API:

  1. In the address bar of a browser, enter the following:

    https://CloudForms_FQDN/api/conversion_hosts 1
    1
    CloudForms_FQDN is the FQDN of the CloudForms machine.

    A log-in screen is displayed.

  2. Enter your CloudForms Username and Password and click Sign in.

    The conversion hosts and their IDs are displayed in JSON format:

    {"name":"conversion_hosts","count":3,"subcount":3,"pages":1,"resources":[{"href":"https://cloudforms.example.com/api/conversion_hosts/10000000000001"},{"href":"https://cloudforms.example.com/api/conversion_hosts/10000000000002"},{"href":"https://cloudforms.example.com/api/conversion_hosts/10000000000003"}],"actions":[{"name":"create","method":"post","href":"https://cloudforms.example.com/api/conversion_hosts"},{"name":"edit","method":"post","href":"https://cloudforms.example.com/api/conversion_hosts"},{"name":"delete","method":"post","href":"https://cloudforms.example.com/api/conversion_hosts"}],"links":{"self":"https://cloudforms.example.com/api/conversion_hosts?offset=0","first":"https://cloudforms.example.com/api/conversion_hosts?offset=0","last":"https://cloudforms.example.com/api/conversion_hosts?offset=0"}}

Chapter 7. Migrating the virtual machines

Migrating the virtual machines involves the following key tasks:

  1. Mapping the resources of your VMware and Red Hat OpenStack Platform environments. See Section 7.1, “Creating an infrastructure mapping”.
  2. Checking for migration conditions with prerequisites. See Section 7.2, “Checking for migration prerequisites”.
  3. Creating and running a migration plan. See Section 7.3, “Creating and running a migration plan”.

    (Optional) You can change the maximum number of concurrent migrations for conversion hosts or providers to control the migration process. See Section 7.3.5, “Changing the maximum number of concurrent migrations”.

7.1. Creating an infrastructure mapping

The infrastructure mapping maps the resources of your VMware and Red Hat OpenStack Platform environments.

Important

If you add or remove providers or provider objects from an infrastructure mapping, the mapping will have missing resource errors. You must delete the infrastructure mapping and create a new one.

Procedure

  1. Click ComputeMigrationInfrastructure Mappings.
  2. Click Create Infrastructure Mapping.
  3. Perform the following procedures in the Create Infrastructure Mapping wizard:

Creating infrastructure mapping

ScreenProcedure

General

  1. Enter the infrastructure mapping Name and (optional) Description.
  2. Select the Target Provider.
  3. Click Next.

Map Compute

  1. Select a Source Provider \ Datacenter \ Cluster source cluster and a Target Provider \ Project.

    If the target project does not contain a conversion host, a warning icon ( warning ) appears. You can create and save an infrastructure mapping, but you must configure the conversion hosts before running a migration plan.

  2. Click Add Mapping. You can map additional projects.
  3. Click Next.

Map Storage

  1. Select a source cluster whose datastores you want to map.
  2. Select a Source Provider \ Datacenter \ Datastore and Target Provider \ Volume Type.

    If the volume type is missing, check that the volume type has been set. Block storage requires at least one volume type. See Create a Volume and Changing a Volume’s Type (Volume Re-typing) in the Red Hat OpenStack Platform Storage Guide.

  3. Click Add Mapping. You can map additional datastores.
  4. Click Next.

Map Networks

  1. Select a source cluster from the drop-down list.
  2. Select one or more networks from Source Provider \ Datacenter \ Network and Target Project \ Network.

    IMS supports both provider and tenant networks.

  3. Click Add Mapping. You can map the networks of additional source clusters.
  4. Click Create.

Results

Click Close. Your infrastructure mapping is saved in ComputeMigrationInfrastructure Mappings.

You can click an infrastructure mapping element to view its details:

Infrastructure Mappings list

Infra mappings list

After you have created an infrastructure mapping, you can check for migration prerequisites. If these conditions do not apply, you can create a migration plan. See Section 7.3, “Creating and running a migration plan”.

7.2. Checking for migration prerequisites

Check your migration for the following conditions, which have prerequisites:

You are migrating previously migrated virtual machines

You must add previously migrated machines to the migration plan with a CSV file. A CSV file is also recommended for large migrations.

See Section 7.2.1, “Creating a CSV file to add virtual machines to the migration plan”.

You are using Ansible playbooks for premigration/postmigration tasks

You must create an Ansible repository and add credentials and playbooks to CloudForms.

See Section 7.2.2, “Adding Ansible playbooks to CloudForms for premigration and postmigration tasks”.

You are migrating virtual machines running RHEL or other Linux operating system

You must create a RHEL premigration playbook to preserve IP addresses after migration. This playbook is selected when you create a migration plan.

See Section 7.2.2, “Adding Ansible playbooks to CloudForms for premigration and postmigration tasks” and Section 7.2.3, “Creating a RHEL premigration playbook for RHEL/Linux source virtual machines”.

7.2.1. Creating a CSV file to add virtual machines to the migration plan

If you are migrating virtual machines that were migrated in the past, you should create a CSV file to add the virtual machines to the migration plan, because the migration plan cannot discover them automatically.

Note

A CSV file is recommended for large migrations because it is faster than manually selecting the security group and flavor of each virtual machine.

Table 7.1. CSV file fields

FieldComments

Name

Virtual machine name. Required

Host

Optional. Only required if virtual machines have identical Name fields.

Provider

Optional. Only required if virtual machines have identical Name and Host fields.

Security Group

Optional. The default is Default.

Flavor

Optional If you do not create flavors for the migration or if you leave this field blank, CloudForms tries to map the source virtual machines to existing flavors.

CSV file example

Name,Host,Provider,Security Group,Flavor
vm01,host1,vSphere3,webservers,x1.medium
vm02,host1,vSphere3,webservers,x1.medium
vm03,host1,vSphere3,webservers,x1.medium

7.2.2. Adding Ansible playbooks to CloudForms for premigration and postmigration tasks

You can add Ansible playbooks to CloudForms to perform automated premigration and postmigration tasks on specific virtual machines, for example:

  • Removing webservers from a load-balancing pool before migration and returning them to the pool after migration
  • Running fstrim after migration to reduce the space required by virtual machines migrating to Red Hat OpenStack Platform with Ceph storage

Procedure

  1. Enable the Embedded Ansible server role in CloudForms. See Enabling the Embedded Ansible Server Role in Red Hat CloudForms: Managing Providers.
  2. Add an Ansible playbook repository. See Adding a Playbook Repository in Red Hat CloudForms: Managing Providers.
  3. Add the credentials of each virtual machine that you are migrating. See Credentials in Red Hat CloudForms: Managing Providers.
  4. Add your playbook as an Ansible service catalog item. See Creating an Ansible Playbook Service Catalog Item in Red Hat CloudForms: Provisioning Virtual Machines and Instances.

You will select the playbooks and the virtual machines on which they run in the Advanced Options screen when you create the migration plan.

7.2.3. Creating a RHEL premigration playbook for RHEL/Linux source virtual machines

If you are migrating virtual machines running RHEL or other Linux operating system, you can create a RHEL premigration playbook to ensure that the IP addresses are accessible after migration. The RHEL premigration playbook calls the Ansible ims.rhel_premigration role.

To install the role with Ansible Galaxy, see ims_rhel_pre_migration. This role is not included in the IMS installation.

The ims.rhel_premigration role performs the following tasks on the VMware virtual machines:

  • Preserves the static IP address configuration by creating udev rules to associate the virtual machine’s MAC address with its interface name
Note

The ims.rhel_premigration role assumes that either the rhel-6-server-rpms or the rhel-7-server-rpms repository is enabled in the source virtual machine when it installs qemu-guest-agent. If you have disabled the repository, re-enable it in the RHEL premigration playbook.

RHEL premigration playbook example

---
- hosts: all
  roles:
    - role: ims.rhel_pre_migration

You can create a migration plan. See Section 7.3, “Creating and running a migration plan”.

7.3. Creating and running a migration plan

Before you perform a large migration, you should perform several test migrations with different maximum numbers of concurrent migrations for your conversion hosts or providers. This will enable you to assess the capabilities of your environment’s infrastructure.

You can create and run a migration plan in CloudForms with the following options:

Note

A CSV file is optional, but recommended, for large migrations because it is faster than manually selecting the security group and flavor of each virtual machine.

  1. Click ComputeMigrationMigration Plans.
  2. Click Create Migration Plan.
  3. Perform the following procedures in the Create Migration Plan wizard:

Create Migration Plan screen

ScreenProcedure

General

  1. Select an infrastructure mapping from the drop-down list.
  2. Enter the migration plan Name and (optional) Description.
  3. Select a virtual machine discovery method:

    • Choose from a list of VMs discovered in the selected infrastructure mapping

      If the virtual machines cannot be discovered, check that the source datastores and networks in the infrastructure mapping are correct.

    • Import a CSV file with a list of VMs to be migrated.

      A CSV file is required for previously migrated source virtual machines and recommended for large migrations.

  4. Click Next.

VMs

  • If you selected Choose from a list of VMs discovered in the selected infrastructure mapping:

    • Select the virtual machines for migration.

      You can search for virtual machines by VM Name, Data Center, Cluster, and Folder.

  • If you selected Import a CSV file with a list of VMs to be migrated:

    1. Click Import.
    2. Browse to the CSV file and click Open.

      If the virtual machines cannot be added to the migration plan, check the CSV file format and fields for errors.

      If the Create Migration Plan wizard freezes, refresh the web page, check the CSV file for errors (for example, virtual machines with duplicate Name fields and no other fields to distinguish them), and create a new migration plan.

    3. Click Next.

Instance Properties

  1. Click the pencil icon to edit the network or flavor of each selected virtual machine.

    Flavors that are too small for the virtual machine are marked with an asterisk (*). If you have not created flavors for the migration, CloudForms tries to map the source virtual machines to existing flavors.

  2. Click Next.

Advanced Options

  1. Select a premigration and/or postmigration playbook service from the dropdown lists.
  2. Select the virtual machines on which to run the playbook services.
  3. Click Next.

Schedule

  1. Select a schedule option:

    • Save migration plan to run later

      The migration plan is saved in Migration Plans Not Started and will not run unless you schedule it or click Migrate to run the scheduled migration plan immediately.

    • Start migration immediately

      The migration plan may take some time to complete. Progress bars indicate the amount of transferred data, the number of migrated virtual machines, and the elapsed time. See Section 7.3.2, “Viewing a migration plan in progress” for details.

      Optionally, you can cancel a migration plan in progress.

  2. Click Create.

Results

Click Close.

When the migration plan has finished, click Migration Plans Complete to view the status of the migration plan. The completed migration plan shows the status of the migrated virtual machines.

In the migration plans list, you can click the More Actions icon ( 7 ) to archive, edit, or delete a migration plan.

7.3.1. Scheduling a saved migration plan

To schedule a saved migration plan to run in the future:

  1. Click Migration Plans Not Started.
  2. Click the Schedule button of a migration plan.
  3. In the Schedule Migration Plan window, select a date and time and click Schedule. The plan’s status is Migration Scheduled with the date and time.

7.3.2. Viewing a migration plan in progress

To view the progress of a migration plan:

  1. Click Migration Plans in Progress.
  2. Click a migration plan name to view its details, including the status of the migrating virtual machines.
Note

The counter in ComputeMigrationMigration Plans may be a few seconds ahead of the counter in the migration plan details view. This is because the Migration Plans counter displays the total time for running the migration plan, while the details counter displays the time for migrating the virtual machines.

7.3.3. Canceling a migration plan in progress

To cancel a migration plan in progress:

  1. Click Migration Plans in Progress.
  2. Select a migration plan and click Cancel Migration.
  3. Click Cancel Migrations to confirm the cancellation. The canceled migration appears in Migration Plans Complete with a red x indicating that the plan did not complete successfully.

7.3.4. Retrying a failed migration plan

To retry a migration plan that failed because of external circumstances (for example, power outage):

  1. Delete all objects created by the failed migration plan:

    • Delete newly created instances to avoid name conflicts with migrating VMware virtual machines.
    • Delete network ports of failed instances.
  2. Click ComputeMigrationMigration Plans.
  3. Click Migration Plans Complete.
  4. Click the Retry button beside the failed migration plan.

7.3.5. Changing the maximum number of concurrent migrations

You can change the maximum number of concurrent migrations for conversion hosts or providers to control the impact of the migration process on your infrastructure.

The provider setting has priority over the conversion host setting. For example, if the maximum number of concurrent migrations is 20 for a provider and 3 for five conversion hosts, the maximum number of concurrent migrations is 20, not 15 (5 conversion hosts x 3 concurrent migrations per host).

An increase in the maximum number of concurrent migrations affects all migration plans immediately. Virtual machines that are queued to migrate will migrate in greater numbers.

A decrease maximum number of concurrent migrations affects only future migration plans. Migration plans that are in progress will use the limit that was set when the plan was created.

Caution

Red Hat OpenStack Platform conversion hosts require an additional 1 GB RAM for each additional concurrent migration above 10.

If you are using VDDK transformation, the number of concurrent migrations must not exceed 20. Otherwise, network overload will cause the migration to fail.

Changing the maximum number of concurrent migrations for all conversion hosts
  1. In CloudForms, click ComputeMigrationMigration Settings.
  2. Select a new Maximum concurrent migrations per conversion host. The default is 10.

    Note

    In the current release, the Maximum concurrent migrations per conversion host interface control does not work.

Changing the maximum number of concurrent migrations for a provider
  1. Log in to the CloudForms machine using SSH.
  2. Enter the following command:

    # vmdb
    # rails console
    irb(main):001:0> $evm = MiqAeMethodService::MiqAeService.new(MiqAeEngine::MiqAeWorkspaceRuntime.new)
    irb(main):002:0> $evm.vmdb(:ext_management_system).find_by(:name => "OpenStack").custom_set("Max Transformation Runners", 30) 1
    1
    Max Transformation Runners is the maximum number of concurrent migrations. The default value is 20 for a provider.
Getting the maximum number of concurrent migrations for a provider
  1. Log in to the CloudForms machine using SSH.
  2. Enter the following command:

    # vmdb
    # rails console
    irb(main):001:0> $evm = MiqAeMethodService::MiqAeService.new(MiqAeEngine::MiqAeWorkspaceRuntime.new)
    irb(main):002:0> $evm.vmdb(:ext_management_system).find_by(:name => "OpenStack").custom_get("Max Transformation Runners")

Chapter 8. Troubleshooting

To identify errors, you can review the migration logs. See Section 8.1, “Migration logs”.

You can check these common issues and mistakes:

Section 8.7, “Known Issues” provides information about issues that will be addressed in a future release.

8.1. Migration logs

You can check the following logs identify the cause of a migration error:

The conversion host logs and the CloudForms migration log can help with troubleshooting.

8.2. Accessing the conversion host logs

When disk migration starts, two logs are created in the conversion host:

  • virt-v2v: Debug output from virt-v2v itself. This log tracks the core of the virtual machine migration process, including libguestfs traces and disk migration details. You can download access this log on the conversion host or download it in CloudForms.
  • virt-v2v-wrapper: Log of the daemonizing wrapper for virt-v2v. This log traces the orchestration of the virtual machine conversion on the conversion host, including disk migration percentages and virt-v2v error reporting. You can access this log on the conversion host.
Important

If you need to open a Red Hat Support call, you must submit both the migration (virt-v2v) log and virt-v2v-wrapper log for analysis.

To access the virt-v2v and virt-v2v-wrapper logs on the conversion host:

  1. Log in to the conversion host using SSH.

    If you are not sure which conversion host to log in to, click the information icon ( 20 ) of a virtual machine in the migration plan details view.

  2. Go to /var/log/vdsm/import/ to access the logs for each migration:

    • virt-v2v log: v2v-import-date-log_number.log
    • virt-v2v-wrapper log: v2v-import-date-log_number-wrapper.log

To download the virt-v2v log in CloudForms:

  1. Click ComputeMigrationMigration Plans.
  2. Click a completed migration plan to view its details.
  3. Click Download LogMigration Log.

8.2.1. Accessing the CloudForms migration log

This log traces the orchestration of the virtual machine migration in CloudForms.

To access the CloudForms migration log:

  1. Log in to the CloudForms machine using SSH.
  2. The migration log is /var/www/miq/vmdb/log/automation.log.

8.3. Infrastructure mapping errors

  • Networks missing, Datastores missing, and Clusters missing error messages: If you create an infrastructure mapping and then change a provider, the provider’s object IDs change. Delete the infrastructure mapping and create a new one.
  • Storage volume type not detected: Check that you have set at least one volume type. See Group Volume Settings with Volume Types in the Red Hat OpenStack Platform Storage Guide for the storage.

8.4. Migration plan errors

  • If the virtual machines are being migrated for the first time and are not discovered by the migration plan, check the source datastores and networks in the infrastructure mapping.
  • If the virtual machines have been migrated in the past, they cannot be discovered by the migration plan. Use a CSV file to add the virtual machines to the migration plan.
  • If the virtual machines cannot be added to the migration plan with a CSV file, check the CSV file format and fields. Create a new migration plan with the updated CSV file.
  • Create Migration Plan wizard hangs while importing a CSV file. This error is caused by an invalid CVS file (for example, virtual machines with a duplicate Name field and no Host/Provider field to distinguish them, or with a duplicate Name field and duplicate Host/Provider fields). Refresh the web page.
  • Denied State error (IMS 1.1). If a migration plan fails immediately and the migration plan displays a Denied State error message, check that you have created and configured the conversion hosts correctly. Cancel the migration plan and run it again.
  • Unable to migrate VMs because no conversion host was configured at the time of the attempted migration. See the product documentation for information on configuring conversion hosts. (IMS 1.2) You can create and save a migration plan whose infrastructure mapping does not contain conversion hosts, but you cannot run the migration plan without conversion hosts. Cancel the migration plan, create the conversion hosts, and run the migration plan again.

8.5. IP address errors

  • If the IP address of a migrated RHEL (or other Linux-based operating system) virtual machine is not accessible, you must create a RHEL premigration playbook and add it to the migration plan.
  • If a migrated virtual machine does not have an IP address:

    • Check that you installed VMware Tools on the VMware virtual machine before migration.
    • Check the VMware virtual machine for an interface configuration file mapped to a non-existent interface (for example, /etc/sysconfig/network-scripts/ifcfg-eth1 exists, but eth1 interface does not). Log example:

      CalledProcessError: Command '['openstack', u'--os-username=admin', u'--os-identity-api-version=3', u'--os-user-domain-name=default', u'--os-auth-url=http://osp.example.com:5000/v3', u'--os-project-name=admin', u'--os-password=******, u--os-project-id=0123456789abcdef0123456789abcdef', 'port', 'create', '--format', 'json', '--network', u'01234567-89ab-cdef-0123-456789abcdef', '--mac-address', u'00:50:56:01:23:45', '--enable', u'port_0', '--fixed-ip', 'ip-address=None'"]' returned non-zero exit status 1
      date time:ERROR: Command output:
      BadRequestException: Unknown errors

8.6. Environment configuration errors

VMware

  • A VMware virtual machine cannot be migrated if it has any of the following conditions:

    • Mounted ISO/CDROM disk
    • Encrypted disk
    • Invalid name, containing spaces or special characters
    • Powered off during migration
  • VDDK transformation only: To perform more than ten concurrent migrations from a single VMware hypervisor, you must increase the hypervisor’s NFC service memory.

Red Hat OpenStack Platform

  • SSH transformation only: Check that you enabled SSH access on the VMware hypervisors and correctly configured your conversion hosts for SSH transformation.
  • disallowed by policy error: The Red Hat OpenStack Platform admin user in CloudForms does not have admin role privileges within the target project. Add the admin user as member and admin to your target project. See Edit a Project in the Red Hat OpenStack Platform Users and Identity Management Guide.

    ERROR: Command exited with non-zero return code 1, output: HttpException: 403: Client Error for url: https://FQDN:13696/v2.0/ports, {"NeutronError": {"message": "((rule:create_port and rule:create_port:mac_address) and rule:create_port:fixed_ips) is disallowed by policy", "type": "PolicyNotAuthorized", "detail": ""}}

8.7. Known Issues

The following issues will be addressed in a future release: