Red Hat Training

A Red Hat training course is available for Red Hat CloudForms

Migrating to Red Hat CloudForms 4.2

Red Hat CloudForms 4.2

Upgrading your system from earlier versions of Red Hat CloudForms Management Engine

Red Hat CloudForms Documentation Team

Abstract

This document describes the process of migrating your Red Hat CloudForms 4.0 or 4.1 environment to Red Hat CloudForms 4.2 (CFME 5.7). It also describes methods for applying minor updates to your CloudForms 4.2 appliances.
If you have a suggestion for improving this guide or have found an error, please submit a Bugzilla report at http://bugzilla.redhat.com against Red Hat CloudForms Management Engine for the Documentation component. Please provide specific details, such as the section number, guide name, and CloudForms version so we can easily locate the content.

This document describes the process of migrating an older Red Hat CloudForms environment to Red Hat CloudForms 4.2 (CFME 5.7). Section 3, “Updating CloudForms” also provides instructions for applying minor updates (errata) to your CloudForms appliances.

You can migrate directly to Red Hat CloudForms 4.2 from the following versions:

  • Red Hat CloudForms 4.1 (CFME 5.6)
  • Red Hat CloudForms 4.0 (CFME 5.5)
Note

Migrating from Cloudforms 3.2 (CFME 5.4) directly to Red Hat CloudForms 4.2 (CFME 5.7) is not supported. You must first upgrade to Red Hat CloudForms 4.0 (CFME 5.5) to perform a migration to CloudForms 4.2.

See Migrating and Updating Red Hat CloudForms / Red Hat CloudForms Management Engine for articles on migrating to CloudForms versions prior to 4.2.

1. Migrating from CloudForms 4.1 (CFME 5.6) to CloudForms 4.2 (CFME 5.7)

1.1. Overview

This procedure describes the process of migrating your Red Hat CloudForms 4.1 (CFME 5.6) to Red Hat CloudForms 4.2 (CFME 5.7). This procedure does not necessarily include migration of all possible customer modifications, so it is recommended that you fully test any modifications before migrating your environment.

Important

Read through all of the steps in this procedure before beginning the migration process.

You can classify the migration into three groups of appliances. A VMDB appliance is defined as a CloudForms server hosting the Virtual Machine Database (VMDB).

Red Hat recommends migrating your appliances in the following order:

  1. Master or global database appliance. In Red Hat CloudForms 4.2, this is now considered the global appliance.
  2. Subordinate database appliances that replicate to the master database.
  3. Non-database appliances.

Migration Workflow Summary

In summary, the migration process from CFME 5.6 to CFME 5.7 follows this workflow:

Note

Appliances must be offline during migration; ensure you plan for downtime when migrating.

  1. Back up appliances (optional but recommended).

  2. Prepare appliances:

    1. Disable older CloudForms repositories and enable new repositories.
    2. Shut down non-VMDB appliances.
    3. On subordinate appliances, stop database synchronization and shut down the evmserver process.
    4. Remove any existing rubyrep configuration.
    5. Shut down evmserver on the master or global appliance.

  3. Migrate appliances:

    1. Migrate the global VMDB appliance: update CFME packages, migrate the database and Automate Model, upgrade PostgreSQL and edit postgresql.conf, then enable and restart the PostgreSQL and CFME services.
    2. Migrate the subordinate database appliances using the same steps as the global VMDB appliance.
    3. Finally, migrate non-database appliances by updating CFME packages, then restart the CFME service.
  4. Configure replication after the migration process is complete and appliances are running once again.

1.2. Back Up Current Appliances

These steps will not affect the operations of your CloudForms infrastructure. However, they will help ensure that you are able to roll back if required and replicate the network settings.

  1. Back up the database of your CFME 5.6 appliance. Take a snapshot if possible.

  2. Back up the following files for disaster recovery, noting which appliance each comes from:

    • /var/www/miq/vmdb/GUID
    • /var/www/miq/vmdb/REGION
  3. If you plan to reuse the hostnames and IP addresses, keep track of that information. You may need these after the CFME 5.7 appliances are created. This information is available on the summary screen of the appliance console.

  4. During the upgrade, the iptables configuration file ( /etc/sysconfig/iptables) is removed. If you have changed the iptables configuration from the default (run iptables --list -n to see the current configuration), use the following command to back up the iptables configuration: 

    # iptables-save > /etc/iptables.conf

    You can restore your iptables configuration file with the following command: 

    # iptables-restore < /etc/iptables.conf

    Alternatively, add this command to /etc/rc.local to reload the rules at every reboot.

Note

For 5.6 appliances with the User Interface server role: Before migration, ensure that the Web Services role is enabled (it is enabled by default in CFME 5.6). If the Web Services role is disabled, it will not be turned on during the migration process. This role is required in CFME 5.7 to be able to log in to the user interface.

1.3. Preparing the Appliances for Migration

  1. Disable the CloudForms 4.1 (CFME 5.6) repositories:

  2. Enable the CloudForms 4.2 (CFME 5.7) repositories.

  3. Shut down all non-VMDB appliances:

    1. Connect to each appliance using SSH.

    2. Log in and shut down the evmserver process: 

      # systemctl stop evmserverd
  4. Repeat for each VMDB appliance.

1.3.1. Additional Preparation for VMDB Appliances

Complete the following steps only on VMDB appliances:

  1. This step must be completed before migrating the master VMDB appliance. On the subordinate region VMDB appliances, stop the Database Synchronization server role.

    1. In the CloudForms user interface, navigate to the specific server’s page under Settings → Configuration → Settings → Server.
    2. On the Server tab under Server Control, uncheck the Database Synchronization role.
    3. Verify that the Database Synchronization role has stopped by going to Settings → Configuration → Diagnostics → Region.
    4. Click the Replication tab. The number for Current Backlog should be increasing.
  2. Connect to the VMDB appliance using SSH.

  3. Shut down the evmserver process on the subordinate or remote database: 

    # systemctl stop evmserverd

  4. Enter the VMDB console: 

    # vmdb

  5. Run the following to remove any installed rubyrep configuration. This will prevent errors when setting up pglogical after migration: 

    # bin/rake evm:dbsync:uninstall

  6. Shut down the evmserver process on the master or global region: 

    # systemctl stop evmserverd

1.4. Migrating from CFME 5.6 to 5.7

To migrate each appliance from CFME 5.6 to 5.7, stop the CFME service, update to the latest CFME package, then restart the CFME service. Additionally for VMDB appliances, migrate the database to the 5.7 level.

1.4.1. Migrating VMDB Appliances from CFME 5.6 to 5.7

Perform the following steps on your CFME 5.6 VMDB appliances to migrate to CFME 5.7. Ensure you wait for each command to finish before going to the next step:

  1. Connect to the appliance using SSH.

  2. Update packages: 

    # yum update

  3. Log out of the appliance, then log back in to fully reload Ruby.

    Important

    Failure to log out of the old shell environment and back into a new one results in an error reporting the Ruby gem bundle is locked.

    To resolve errors from running bundle install or failing to log in and out of the old shell environment: log out of the current shell and log back in, then run yum reinstall cfme-gemset.

  4. Enter the VMDB console: 

    # vmdb

  5. Run the following to migrate everything in the database from 5.6 to work with the latest 5.7 configuration: 

    # bin/rake db:migrate

  6. Migrate the Automate Model to the latest version. This resets the ManageIQ and Red Hat domains (base domains) to a new and upgraded version. 

    # bin/rake evm:automate:reset
    Important

    Updates to the base domains can affect changes made in higher level domains. If you have custom code in any higher level domain that overrides the code in the base domain, it must be investigated and verified to ensure there are not significant changes in the update that could break or invalidate the custom code.

  7. Run the PostgreSQL upgrade script: 

    # /usr/bin/miq_postgres_upgrade.sh
    Important

    When you run the script, you are asked to confirm whether to proceed with the operation. Your response must be either a capital Y or a capital N; all other values are rejected.

  8. Before starting the new server, edit the /var/opt/rh/rh-postgresql95/lib/pgsql/data/postgresql.conf file to ensure the configuration works correctly with PostgreSQL 9.5.

    Note that some lines below are intentionally added in a commented-out fashion; these indicate the default values for particular parameters and are shown for informational purposes. See /var/opt/rh/rh-postgresql95/lib/pgsql/data/postgresql.conf for further documentation about configuring this file.

    1. Under Checkpoints, remove and add the following lines as shown: 

      -checkpoint_segments = 15   # MIQ Value;
-#checkpoint_segments = 3    # in logfile segments, min 1, 16MB each
+#max_wal_size = 1GB
+#min_wal_size = 80MB

    2. Under Kernel Resource Usage, edit the following line as shown: 

      -shared_preload_libraries = 'pglogical'   # MIQ Value (change requires restart)
+shared_preload_libraries = 'pglogical,repmgr_funcs'   # MIQ Value (change requires restart)

    3. Under Archiving, add the following line: 

      +wal_log_hints = on

  9. Enable and start PostgreSQL: 

    # systemctl enable rh-postgresql95-postgresql && systemctl start rh-postgresql95-postgresql

  10. Start the evmserver process: 

    # systemctl start evmserverd

1.4.2. Migrating Non-Database Appliances from CFME 5.6 to 5.7

Perform the following steps on your CFME 5.6 non-database appliances to migrate to CFME 5.7. Ensure you wait for each command to finish before going to the next step.

Important

Before migrating non-database appliances, ensure the VMDB appliances for the region are fully up and running.

  1. Connect to the appliance using SSH.

  2. Update packages: 

    # yum update

  3. Log out of the appliance, then log back in to fully reload Ruby.

    Important

    Failure to log out of the old shell environment and back into a new one results in an error reporting the Ruby gem bundle is locked.

    To resolve errors from running bundle install or failing to log in and out of the old shell environment: log out of the current shell and log back in, then run yum reinstall cfme-gemset.

  4. Enter the VMDB console: 

    # vmdb

  5. Start the evmserver process: 

    # systemctl start evmserverd
Note

To configure replication, see Configuring Database Replication in General Configuration.

2. Migrating from CloudForms 4.0 (CFME 5.5) to CloudForms 4.2 (CFME 5.7)

2.1. Overview

This procedure describes the process of migrating your Red Hat CloudForms 4.0 (CFME 5.5) to Red Hat CloudForms 4.2 (CFME 5.7). This procedure does not necessarily include migration of all possible customer modifications, so it is recommended that you fully test any modifications before migrating your environment.

Important

Read through all of the steps in this procedure before beginning the migration process.

You can classify the migration into three groups of appliances. A VMDB appliance is defined as a CloudForms server hosting the Virtual Machine Database (VMDB).

Red Hat recommends migrating your appliances in the following order:

  1. Master or global database appliance. In Red Hat CloudForms 4.2, this is now considered the global appliance.
  2. Subordinate database appliances that replicate to the master database.
  3. Non-database appliances.

Migration Workflow Summary

In summary, the migration process from CFME 5.5 to CFME 5.7 follows this workflow:

Note

Appliances must be offline during migration; ensure you plan for downtime when migrating.

  1. Back up appliances (optional but recommended).

  2. Prepare appliances:

    1. Disable older CloudForms repositories and enable new repositories.
    2. Shut down non-VMDB appliances.
    3. On subordinate appliances, stop database synchronization and shut down the evmserver process.
    4. Remove any existing rubyrep configuration.
    5. Shut down evmserver on the master or global appliance.

  3. Migrate appliances:

    1. Migrate the global VMDB appliance: update CFME packages, migrate the database and Automate Model, upgrade PostgreSQL, edit PostgreSQL configuration, then enable and start the PostgreSQL service, rearrange any mismatched database tables, then start the CFME service.
    2. Migrate the subordinate database appliances using the same steps as the global VMDB appliance.
    3. Finally, migrate non-database appliances by updating CFME packages, then restart the CFME service.
  4. Configure replication after the migration process is complete and appliances are running once again.

2.2. Back Up Current Appliances

These steps will not affect the operations of your CloudForms infrastructure. However, they will help ensure that you are able to roll back if required and replicate the network settings.

  1. Back up the database of your CFME 5.5 appliance. Take a snapshot if possible.

  2. Back up the following files for disaster recovery, noting which appliance each comes from:

    • /var/www/miq/vmdb/GUID
    • /var/www/miq/vmdb/REGION
  3. If you plan to reuse the hostnames and IP addresses, keep track of that information. You may need these after the CFME 5.7 appliances are created. This information is available on the summary screen of the appliance console.

  4. During the upgrade, the iptables configuration file ( /etc/sysconfig/iptables) is removed. If you have changed the iptables configuration from the default (run iptables --list -n to see the current configuration), use the following command to back up the iptables configuration: 

    # iptables-save > /etc/iptables.conf

    You can restore your iptables configuration file with the following command: 

    # iptables-restore < /etc/iptables.conf

    Alternatively, add this command to /etc/rc.local to reload the rules at every reboot.

Note

For 5.6 appliances with the User Interface server role: Before migration, ensure that the Web Services role is enabled (it is enabled by default in CFME 5.6). If the Web Services role is disabled, it will not be turned on during the migration process. This role is required in CFME 5.7 to be able to log in to the user interface.

2.3. Preparing the Appliances for Migration

  1. Disable the CloudForms 4.0 (CFME 5.5) repositories.

  2. Enable the CloudForms 4.2 (CFME 5.7) repositories.

  3. Shut down all non-VMDB appliances:

    1. Connect to each appliance using SSH.

    2. Stop the evmserver process: 

      # systemctl stop evmserverd
  4. Repeat for each VMDB appliance.

2.3.1. Additional Preparation for VMDB Appliances

Complete the following steps only on VMDB appliances:

  1. This step must be completed before migrating the master VMDB appliance. On the subordinate region VMDB appliances, stop the Database Synchronization server role.

    1. In the CloudForms user interface, navigate to the specific server’s page under Settings → Configuration → Settings → Server.
    2. On the Server tab under Server Control, uncheck the Database Synchronization role.
    3. Click Save.
    4. Verify that the Database Synchronization role has stopped by going to Configure → Configuration → Diagnostics → Region.
    5. Click the Replication tab. The number for Current Backlog should be increasing.
  2. Connect to the VMDB appliance using SSH.

  3. Shut down the evmserver process on the subordinate or remote database: 

    # systemctl stop evmserverd

  4. Enter the VMDB console: 

    # vmdb

  5. Run the following to remove any installed rubyrep configuration. This will prevent errors when setting up pglogical after migration: 

    # bin/rake evm:dbsync:uninstall

  6. Shut down the evmserver process on the master or global region: 

    # systemctl stop evmserverd

2.4. Migrating from CFME 5.5 to 5.7

To migrate each appliance from CFME 5.5 to 5.7, stop the CFME service, update to the latest CFME package, then restart the CFME service. Additionally for VMDB appliances, migrate the database to the 5.7 level.

2.4.1. Migrating VMDB Appliances from CFME 5.5 to 5.7

Perform the following steps on your CFME 5.5 VMDB appliances to migrate to CFME 5.7. Ensure you wait for each command to finish before going to the next step:

  1. Connect to the appliance using SSH.

  2. Update packages: 

    # yum update

  3. Log out of the appliance, then log back in to fully reload Ruby.

    Important

    Failure to log out of the old shell environment and back into a new one results in an error reporting the Ruby gem bundle is locked.

    To resolve errors from running bundle install or failing to log in and out of the old shell environment: log out of the current shell and log back in, then run yum reinstall cfme-gemset.

  4. Enter the VMDB console: 

    # vmdb

  5. Run the following to migrate everything in the database from 5.5 to work with the latest 5.7 configuration: 

    # bin/rake db:migrate

  6. Migrate the Automate Model to the latest version. This resets the ManageIQ and Red Hat domains (base domains) to a new and upgraded version. 

    # bin/rake evm:automate:reset
    Important

    Updates to the base domains can affect changes made in higher level domains. If you have custom code in any higher level domain that overrides the code in the base domain, it must be investigated and verified to ensure there are not significant changes in the update that could break or invalidate the custom code.

  7. Run the PostgreSQL upgrade script: 

    # /usr/bin/miq_postgres_upgrade.sh
    Important

    When you run the script, you are asked to confirm whether to proceed with the operation. Your response must be either a capital Y or a capital N; all other values are rejected.

  8. Before starting the new server, edit the /var/opt/rh/rh-postgresql95/lib/pgsql/data/postgresql.conf file to ensure the configuration works correctly with PostgreSQL 9.5.

    Note that some lines below are intentionally added in a commented-out fashion; these indicate the default values for particular parameters and are shown for informational purposes. See /var/opt/rh/rh-postgresql95/lib/pgsql/data/postgresql.conf for further documentation about configuring this file.

    1. Under Checkpoints, remove and add the following lines as shown: 

      -checkpoint_segments = 15   # MIQ Value;
-#checkpoint_segments = 3    # in logfile segments, min 1, 16MB each
+#max_wal_size = 1GB
+#min_wal_size = 80MB

-wal_level = minimal     # minimal, archive, or hot_standby
+wal_level = 'logical'    # MIQ Value (pglogical)

-max_wal_senders = 0        # max number of walsender processes
+max_wal_senders = 10        # MIQ Value (pglogical) max number of walsender processes

+max_worker_processes = 10   # MIQ Value (pglogical)
+max_replication_slots = 10  # MIQ Value (pglogical)

    2. Under Kernel Resource Usage, edit the following line as shown: 

      -shared_preload_libraries = 'pglogical'   # MIQ Value (change requires restart)
+shared_preload_libraries = 'pglogical,repmgr_funcs'   # MIQ Value (change requires restart)

    3. Under Archiving, add the following line: 

      +wal_log_hints = on

  9. Add the following line in the /var/opt/rh/rh-postgresql95/lib/pgsql/data/pg_ident.conf file: 

    usermap    postgres    root

  10. Add the following line in the /var/opt/rh/rh-postgresql95/lib/pgsql/data/pg_hba.conf file: 

    host  replication  all  all  md5

  11. Enable and start PostgreSQL: 

    # systemctl enable rh-postgresql95-postgresql && systemctl start rh-postgresql95-postgresql

  12. After the migration, some columns contained by tables (such as computer_systems, event_streams and security_contexts) in the database may no longer match. Run the following to list and check all tables for mismatch: 

    # rake evm:db:check_schema

  13. Rearrange the table columns to match the new schema. Run the following, replacing <table_name> with each table listed in the previous command: 

    # rails r tools/fix_column_ordering.rb <table name>

  14. Start the evmserver process: 

    # systemctl start evmserverd

2.4.2. Migrating Non-Database Appliances from CFME 5.5 to 5.7

Perform the following steps on your CFME 5.6 non-database appliances to migrate to CFME 5.7. Ensure you wait for each command to finish before going to the next step.

Important

Before migrating non-database appliances, ensure the VMDB appliances for the region are fully up and running.

  1. Connect to the appliance using SSH.

  2. Update packages: 

    # yum update

  3. Log out of the appliance, then log back in to fully reload Ruby.

    Important

    Failure to log out of the old shell environment and back into a new one results in an error reporting the Ruby gem bundle is locked.

    To resolve errors from running bundle install or failing to log in and out of the old shell environment: log out of the current shell and log back in, then run yum reinstall cfme-gemset.

  4. Enter the VMDB console: 

    # vmdb

  5. Start the evmserver process: 

    # systemctl start evmserverd
Note

To configure replication, see Configuring Database Replication in General Configuration.

3. Updating CloudForms

This chapter details applying software package minor updates (referred to as errata) to CloudForms 4.2 appliances.

Appliances must be registered to Red Hat Subscription Manager and subscribed to the update channels required by CloudForms in order to access updates. See Registering Red Hat CloudForms in General Configuration to register and subscribe the appliance.

3.1. Updating the CloudForms Application

An important part of securing CloudForms is to ensure your appliances use the latest software packages.

The Red Hat Updates tab in the CloudForms user interface enables you to check for updates and update registered appliances. Any services requiring a restart to apply updates are automatically restarted as part of the Red Hat Updates process.

Important

Using the Red Hat Updates tab only applies software updates for the CloudForms application. To run the update from the command line, run yum update cfme*. See Section 3.2, “Updating All Packages on the Appliance” for instructions on applying Red Hat errata. To upgrade your appliance to CloudForms 4.2 from an earlier version, see Section 1, “Migrating from CloudForms 4.1 (CFME 5.6) to CloudForms 4.2 (CFME 5.7)” and Section 2, “Migrating from CloudForms 4.0 (CFME 5.5) to CloudForms 4.2 (CFME 5.7)”.

To apply updates to the CloudForms application:

  1. From the settings menu, select Configuration.
  2. Select Region in the accordion menu and click the Red Hat Updates tab.
  3. Click Check For Updates to search the Content Delivery Network (CDN) for any updated CloudForms packages. If an appliance update is available, it will be listed with the available version.
  4. Click Apply CFME Update to install and update CloudForms packages. The CloudForms service will be automatically restarted as needed.
Note

If the appliance is registered to Red Hat Satellite, you can use content views to manage updates for CloudForms. For more information, see Creating Content Views in the Red Hat Satellite 6 Content Management Guide.

The following options are available in the Appliance Updates section of Red Hat Updates:

Table 1. Options Available in Appliance Updates

OptionUse

Refresh List

Refreshes the list of appliances.

Check for Updates

Checks for all available CloudForms updates using yum.

Register

Attempts to register the appliance if it is not already registered. CloudForms subscribes to the cf-me-5.7-for-rhel-7-rpms rhel-server-rhscl-7-rpms repositories, and to the products designated by Red Hat product certification for subscription-manager registered appliances. The Red Hat Enterprise Linux channels are enabled by default on registration. In addition, CloudForms automatically checks for updates after registering.

Apply CFME Update

Applies updates to CloudForms packages only. Specifically, this option runs the yum -y update cfme-appliance command. This command installs every package listed in the dependency tree if it is not already installed. If a specific version of a package is required, that version of the package is installed or upgraded. No other packages, such as PostgreSQL or Red Hat Enterprise Linux, are updated. The appliance may be rebooted automatically during this process.

3.2. Updating All Packages on the Appliance

You can apply updates to the appliance using the yum command or Red Hat Satellite. This updates all RPMs on the appliance, not just the CloudForms packages. Yum can be used at any time to update any single package or the entire appliance to any new or updated packages available on the subscription.

Warning

Updates to the the operating system, CloudForms application or dependent packages may introduce incompatibilities in customized environments. Before applying updates to the appliance, back up the appliance or take a snapshot so that changes can be reverted in production environments if needed.

Important

Scheduled downtime is required while updating system packages for the following reasons:

  • Some updates may interrupt CloudForms operations.
  • Updates for the PostgreSQL database server suspend CloudForms operations.
  • System updates may require a reboot of the CloudForms appliance.

To update all packages on the appliance using yum, follow the procedure below. To update packages on the appliance using Red Hat Satellite, see Viewing and Applying Errata and Configuring and Running Remote Commands in the Red Hat Satellite 6 documentation for more information.

  1. Log into each appliance console as the root user and perform the following steps:

    1. Stop the CloudForms application (the evmserver process) with the following command: 

      # systemctl stop evmserverd

    2. Apply the software updates: 

      # yum update
      Important

      Do not reboot or restart yet.

  2. Log into each server containing an internal database and perform the following steps:

    1. Stop the database with the following command: 

      # systemctl stop rh-postgresql95-postgresql.service

    2. Apply the software updates: 

      # yum update

    3. Reboot the server unless the errata or the command needs-restarting advises a restart is safe: 

      # systemctl restart rh-postgresql95-postgresql.service

  3. Log into the appliance console on each appliance as the root user and perform the following steps:

    1. Reboot the server unless the errata or the command needs-restarting advises a restart is safe: 

      # reboot