Upgrading and Updating Red Hat Satellite

Red Hat Satellite 6.6

Upgrading and updating Red Hat Satellite Server and Capsule Server

Red Hat Satellite Documentation Team

Abstract

This guide describes upgrading and updating Red Hat Satellite Server, Capsule Server, and hosts.

Chapter 1. Upgrade Overview

This chapter details the prerequisites and available upgrade paths to Red Hat Satellite 6.6. Review this information before upgrading your current Red Hat Satellite 6 installation.

In this guide, the terms update, upgrade, and migrate have the following meanings:

Upgrading
The process of advancing your Satellite Server and Capsule Server installations from a y-stream release to the next, for example Satellite 6.5 to Satellite 6.6.
Updating
The process of advancing your Satellite Server and Capsule Server installations from a z-stream release to the next, for example Satellite 6.6.0 to Satellite 6.6.1.
Migrating
The process of moving an existing Satellite installation to another Red Hat Enterprise Linux server.

For interactive upgrade instructions, you can also use the Red Hat Satellite Upgrade Helper on the Red Hat Customer Portal. This application provides you with an exact guide to match your current version number. You can find instructions that are specific to your upgrade path, as well as steps to prevent known issues. For more information, see Satellite Upgrade Helper on the customer portal.

Important

The Red Hat Satellite Server and Capsule Server y-stream versions must match. For example, a 6.5 Satellite Server is not compatible with a 6.6 Capsule Server and a 6.6 Satellite Server is not compatible with a 6.5 Capsule Server. Mismatching Satellite Server and Capsule Server versions will result in the Capsule Server failing silently. However, a Capsule Server using one z-stream version older than the Satellite Server is supported. For example, a Satellite 6.6.1 Satellite Server is compatible with a 6.6.0 Capsule Server.

1.1. Prerequisites

Upgrading to Satellite 6.6 affects your entire Satellite infrastructure. Before proceeding, complete the following:

Warning

If you customize configuration files, manually or use a tool such as Hiera, these customizations are overwritten when the installation script runs during upgrading or updating. You can use the --noop option with the satellite-installer script to test for changes. For more information, see the Red Hat Knowledgebase solution How to use the noop option to check for changes in Satellite config files during an upgrade.

1.2. Upgrade Paths

You can upgrade to Red Hat Satellite 6.6 from Red Hat Satellite 6.5. Satellite Servers and Capsule Servers on earlier versions must first be upgraded to Satellite 6.5. For more details, see the Satellite 6.5 Upgrading and Updating Red Hat Satellite guide.

Figure 1.1. Overview of Satellite 6.6 Upgrade Paths

Overview of Satellite 6.6 Upgrade Paths
Warning

Upgrading from the Beta to GA version is not supported.

The high level steps in upgrading to Satellite 6.6 are as follows.

  1. Clone your existing Satellite Servers. For more information, see Chapter 2, Cloning Satellite Server.
  2. Upgrade Satellite Server and all Capsule Servers to Satellite 6.6. For more information, see Section 3.1, “Upgrading Satellite Server”.
  3. Upgrade the Satellite tools on all Satellite clients. For more information, see Section 3.4, “Upgrading Satellite Clients”.

Self-Registered Satellites

You cannot upgrade a self-registered Satellite. You must migrate a self-registered Satellite to the Red Hat Content Delivery Network (CDN) and then perform the upgrade. To migrate a self-registered Satellite to the CDN, see Migrating Self-Registered Satellites in the Satellite 6.3 Upgrading and Updating Red Hat Satellite guide.

1.3. Following the Progress of the Upgrade

Because of the lengthy upgrade time, use a utility such as screen to suspend and reattach a communication session. You can then check the upgrade progress without staying connected to the command shell continuously. For more information about using the screen command, see How do I use the screen command? article in the Red Hat Knowledge Base. You can also see the screen manual page for more information.

If you lose connection to the command shell where the upgrade command is running you can see the logs in /var/log/foreman-installer/satellite.log to check if the process completed successfully.

Chapter 2. Cloning Satellite Server

When you upgrade Satellite Server, you can optionally create a clone of your Satellite to ensure that you do not lose any data while you upgrade. After your upgrade is complete, you can then decommission the earlier version of Satellite Server.

Use the following procedures to clone your Satellite instances to preserve your environments in preparation for upgrade.

The Satellite clone tool does not support migrating a Capsule Server to Red Hat Enterprise Linux 7. Instead you must backup the existing Capsule Server, restore it on Red Hat Enterprise Linux 7, then reconfigure the Capsule Server.

Terminology

Ensure that you understand the following terms:

Source server: the server that you clone

Target server: the new server that you copy files to and clone the source server to.

2.1. Cloning Process Overview

  1. Back up the source server.
  2. Clone the source server to the target server.
  3. Power off the source server.
  4. Update the network configuration on the target server to match the target server’s IP address with its new host name.
  5. Restart goferd in Content hosts and Capsules to refresh the connection.
  6. Test the new target server.

2.2. Prerequisites

To clone Satellite Server, ensure that you have the following resources available:

  • A minimal install of Red Hat Enterprise Linux 7 server to become the target server. Do not install Red Hat Enterprise Linux 7 software groups, or third-party applications. Ensure that your server complies with all the specifications of Preparing your Environment for Installation in Installing Satellite Server from a Connected Network.
  • A backup from Satellite 6.5 that you make using the satellite-maintain backup script. You can use a backup with or without Pulp data.
  • A Satellite subscription for the target server.

Before you begin cloning, ensure the following conditions exist:

  • The target server is on an isolated network. This avoids unwanted communication with Capsule Servers and hosts
  • The target server has the capacity to store all your backup files from the source server.

Customized configuration files

If you have any customized configurations on your source server that are not managed by the satellite-installer tool or Satellite backup process, you must manually back up these files.

2.3. Pulp Data Considerations

You can clone Satellite server without including Pulp data. However, for your cloned environment to work, you do require Pulp data. If the target server does not have Pulp data. it is not a fully working Satellite.

To transfer Pulp data to a target server, you have two options:

  • Clone using backup with Pulp data
  • Clone using backup without Pulp data and copy /var/lib/pulp manually from source server.

If your pulp_data.tar file is greater than 500 GB, or if you use a slow storage system, such as NFS, and your pulp_data.tar file is greater than 100 GB, do not include pulp_data.tar in the backup because this can cause memory errors during extraction. Copy the pulp_data.tar file from the source server to the target server.

To back up without Pulp data

Follow the steps in the procedure in Section 2.4, “Cloning Satellite Server” and replace the steps that involve cloning with Pulp data with the following steps:

  1. Perform a backup with MongoDB and PostgreSQL databases active excluding the Pulp data:

    # satellite-maintain backup offline --skip-pulp-content \
    --assumeyes /var/backup
  2. Stop and disable the satellite-maintain services

    # satellite-maintain service stop
    # satellite-maintain service disable
  3. Copy the Pulp data to the target server:

    # rsync --archive --partial --progress --compress \
    /var/lib/pulp target_server.example.com:/var/lib/pulp

Proceed to Section 2.4.2, “Cloning to the Target Server”.

2.4. Cloning Satellite Server

Use the following procedures to clone Satellite Server. Note that because of the high volume of data that you must copy and transfer as part of these procedures, it can take a significant amount of time to complete.

2.4.1. Preparing the source server for cloning

On the source server, complete the following steps:

  1. Verify the Pool ID of your Satellite subscription:

    # subscription-manager list --consumed \
    --matches 'Red Hat Satellite'|grep "Pool ID:"|awk '{print $3}'

    Note the Pool ID for later use.

  2. Remove the Red Hat Satellite subscription.

    # subscription-manager remove --serial=$(subscription-manager list \
    --consumed \
    --matches 'Red Hat Satellite'|grep "Serial:"|awk '{print $2}')
  3. Determine the size of the Pulp data:

    # du -sh /var/lib/pulp/
  4. If you have less than 500 GB of Pulp data, perform a backup with MongoDB and PostgreSQL databases active including the Pulp data. If you have more than 500 GB of Pulp data, skip the following steps and complete the steps in Section 2.3, “Pulp Data Considerations” before you continue.

    # satellite-maintain backup offline --assumeyes /var/backup
  5. Stop and disable the satellite-maintain services:

    # satellite-maintain service stop
    # satellite-maintain service disable

Proceed to Section 2.4.2, “Cloning to the Target Server”.

2.4.2. Cloning to the Target Server

To clone your server, complete the following steps on your target server:

  1. The satellite-clone tool defaults to using /backup/ as the backup folder. If you copy to a different folder, update the backup_dir variable in the /etc/satellite-clone/satellite-clone-vars.yml file.
  2. Place the backup files from the source Satellite in the /backup/ folder on the target server. You can either mount the shared storage or copy the backup files to the /backup/ folder on the target server.
  3. Power off the source server.
  4. Enter the following commands to register to the Customer Portal, attach subscriptions, and enable only the required subscriptions:

    # subscription-manager register your_customer_portal_credentials
    # subscription-manager attach --pool=pool_ID
    # subscription-manager repos --disable=*
    # subscription-manager repos \
    --enable=rhel-7-server-rpms \
    --enable=rhel-server-rhscl-7-rpms \
    --enable=rhel-7-server-satellite-maintenance-6-rpms \
    --enable=rhel-7-server-satellite-6.5-rpms
  5. Install the satellite-clone package

    # satellite-maintain packages install satellite-clone

    After you install the satellite-clone tool, you can adjust any configuration to suit your own deployment in the /etc/satellite-clone/satellite-clone-vars.yml file.

  6. Run the satellite-clone tool.

    # satellite-clone
  7. Reconfigure DHCP, DNS, TFTP and remote execution services. The cloning process disables these services on the target Satellite Server to avoid conflict with the source Satellite Server.
  8. Reconfigure and enable DHCP, DNS, TFTP in the Satellite web UI. For more information, see Configuring DNS, DHCP, and TFTP on Satellite Server in Installing Satellite Server from a Connected Network.
  9. Enable remote execution:

    # satellite-installer --scenario satellite \
    --enable-foreman-plugin-remote-execution \
    --enable-foreman-proxy-plugin-remote-execution-ssh
  10. Log on to the Satellite web UI, with the username admin and the password changeme. Immediately update the admin password to secure credentials.
  11. Ensure that the correct organization is selected.
  12. Navigate to Content > Subscriptions, then click Manage Manifest.
  13. Click the Refresh button, then click Close to return to the list of subscriptions.
  14. Verify that the available subscriptions are correct.
  15. Follow the instructions in the /usr/share/satellite-clone/logs/reassociate_capsules.txt file to restore the associations between Capsules and their lifecycle environments.
  16. Update your network configuration, for example, DNS, to match the target server’s IP address with its new host name. The satellite-clone tool changes the hostname to the source server’s hostname. If you want to change the hostname to something different, you can use the satellite-change-hostname tool. For more information, see Renaming a Satellite or Capsule Server in Administrating Red Hat Satellite.
  17. If the source server uses the virt-who daemon, install and configure it on the target server. Copy all the virt-who configuration files in the /etc/virt-who.d/ directory from the source server to the same directory on the target server. For more information, see Configuring Virtual Machine Subscriptions in Red Hat Satellite.

After you perform an upgrade using the following chapters, you can safely decommission the source server.

Chapter 3. Upgrading Red Hat Satellite

Warning

If you have Satellite 6 installed in a high availability configuration, contact Red Hat Support before upgrading to Satellite 6.6.

Use the following procedures to upgrade your existing Red Hat Satellite to Red Hat Satellite 6.6:

Before upgrading, see Section 1.1, “Prerequisites”.

3.1. Upgrading Satellite Server

This section describes how to upgrade Satellite Server from 6.5 to 6.6. You can upgrade from any minor version of Red Hat Satellite Server 6.5.

Before You Begin

  • Review and update your firewall configuration prior to upgrading your Satellite Server. For more information, see Ports and Firewalls Requirements in Installing Satellite Server from a Connected Network.
  • Ensure that you do not delete the manifest from the Customer Portal or in the Satellite Web UI because this removes all the entitlements of your content hosts.
  • Back up and remove all Foreman hooks before upgrading. Restore any hooks only after Satellite is known to be working after the upgrade is complete.
  • If you have edited any of the default templates, back up the files either by cloning or exporting them. Cloning is the recommended method because that prevents them being overwritten in future updates or upgrades. To confirm if a template has been edited, you can view its History before you upgrade or view the changes in the audit log after an upgrade. In the web UI, Navigate to Monitor > Audits and search for the template to see a record of changes made. If you use the export method, restore your changes by comparing the exported template and the default template, manually applying your changes.

Capsule Considerations

  • If you use Content Views to control updates to a Capsule Server’s base operating system, or for the Capsule Server repository, you must publish updated versions of those Content Views.
Warning

If you implemented custom certificates, you must retain the content of both the /root/ssl-build directory and the directory in which you created any source files associated with your custom certificates.

Failure to retain these files during an upgrade causes the upgrade to fail. If these files have been deleted, they must be restored from a backup in order for the upgrade to proceed.

Configuring the BASH shell

The BASH shell stores the location of a binary in a hash table. During the upgrade, the location of the satellite-maintain script is changed, but BASH does not register this change, and satellite-maintain fails if it calls the script after the change.

  • Optional: Before the upgrade, users of the BASH shell can set the checkhash option temporarily to ensure satellite-maintain works after the installer completes. Enter a command as follows in your BASH shell:

    # shopt -s checkhash
  • After a successful or failed upgrade, in all currently running BASH shells, enter the following command:

    # hash -d satellite-maintain 2> /dev/null

Upgrade Scenarios

You cannot upgrade a self-registered Satellite. You must migrate a self-registered Satellite to the Red Hat Content Delivery Network (CDN) and then perform the upgrade. To migrate a self-registered Satellite to the CDN, see Migrating Self-Registered Satellites in the Satellite 6.3 Upgrading and Updating Red Hat Satellite guide.

FIPS mode

You cannot upgrade Satellite Server from a RHEL base system that is not operating in FIPS mode to a RHEL base system that is operating in FIPS mode.

To run Satellite Server on a RHEL base system operating in FIPS mode, you must install Satellite on a freshly provisioned RHEL base system operating in FIPS mode. For more information, see System Requirements in Installing Satellite Server from a Connected Network.

3.1.1. Upgrading a Connected Satellite Server

Use this procedure for a Satellite Server connected to the Red Hat Content Delivery Network.

Warning

If you customize configuration files, manually or using a tool such as Hiera, these changes are overwritten when the installation script runs during upgrading or updating. You can use the --noop option with the satellite-installer script to test for changes. For more information, see the Red Hat Knowledgebase solution How to use the noop option to check for changes in Satellite config files during an upgrade.

Upgrade Satellite Server

  1. Create a backup.

  2. Back up the DNS and DHCP configuration files /etc/zones.conf and /etc/dhcp/dhcpd.conf as the installer only supports one domain or subnet, and therefore restoring changes from these backups might be required.
  3. If you have made manual edits to DNS or DHCP configuration files and do not want to overwrite the changes, enter the following command:

    # satellite-installer --foreman-proxy-dns-managed=false \
    --foreman-proxy-dhcp-managed=false
  4. In the Satellite web UI, navigate to Hosts > Discovered hosts. On the Discovered Hosts page, power off and then delete the discovered hosts. From the Select an Organization menu, select each organization in turn and repeat the process to power off and delete the discovered hosts. Make a note to reboot these hosts when the upgrade is complete.
  5. Clean yum cache:

    # yum clean all
  6. Ensure that the rubygem-foreman_maintain package that provides satellite-maintain is installed and up to date:

    # yum install rubygem-foreman_maintain
  7. Check the available versions to confirm the version you want is listed:

    # satellite-maintain upgrade list-versions
  8. Use the health check option to determine if the system is ready for upgrade. When prompted, enter the hammer admin user credentials to configure satellite-maintain with hammer credentials. These changes are applied to the /etc/foreman-maintain/foreman-maintain-hammer.yml file.

    # satellite-maintain upgrade check --target-version 6.6

    Review the results and address any highlighted error conditions before performing the upgrade.

  9. Because of the lengthy upgrade time, use a utility such as screen to suspend and reattach a communication session. You can then check the upgrade progress without staying connected to the command shell continuously. For more information about using the screen command, see How do I use the screen command? article in the Red Hat Knowledge Base.

    If you lose connection to the command shell where the upgrade command is running you can see the logged messages in the /var/log/foreman-installer/satellite.log file to check if the process completed successfully.

  10. Perform the upgrade:

    # satellite-maintain upgrade run --target-version 6.6
  11. Check when the kernel packages were last updated:

    # rpm -qa --last | grep kernel
  12. Optional: If a kernel update occurred since the last reboot, stop the satellite-maintain services and reboot the system:

    # satellite-maintain service stop
    # reboot
  13. If using a BASH shell, after a successful or failed upgrade, enter:

    # hash -d satellite-maintain service 2> /dev/null
  14. Check and restore any changes required to the DNS and DHCP configuration files using the backups that you make.
  15. If you make changes in the previous step, restart the satellite-maintain services.

    # satellite-maintain service restart
  16. If you have the OpenSCAP plug-in installed, but do not have the default OpenSCAP content available, enter the following command:

    # foreman-rake foreman_openscap:bulk_upload:default

3.1.2. Upgrading a Disconnected Satellite Server

Use this procedure for a Satellite Server not connected to the Red Hat Content Delivery Network.

Warning

If you customize configuration files, manually or using a tool such as Hiera, these changes are overwritten when you enter the satellite-maintain command during upgrading or updating. You can use the --noop option with the satellite-installer command to review the changes that are applied during upgrading or updating. For more information, see the Red Hat Knowledgebase solution How to use the noop option to check for changes in Satellite config files during an upgrade.

Before You Begin

  • Review and update your firewall configuration before upgrading your Satellite Server. For more information, see Ports and Firewalls Requirements in Installing Satellite Server from a Disconnected Network.
  • Ensure that you do not delete the manifest from the Customer Portal or in the Satellite Web UI because this removes all the entitlements of your content hosts.
  • Back up and remove all Foreman hooks before upgrading. Reinstate hooks only after Satellite is known to be working after the upgrade is complete.
Warning

If you implemented custom certificates, you must retain the content of both the /root/ssl-build directory and the directory in which you created any source files associated with your custom certificates.

Failure to retain these files during an upgrade causes the upgrade to fail. If these files have been deleted, they must be restored from a backup in order for the upgrade to proceed.

Upgrade Disconnected Satellite Server

  1. Create a backup.

    • On a virtual machine, take a snapshot.
    • On a physical machine, create a backup.
  2. A pre-upgrade script is available to detect conflicts and list hosts which have duplicate entries in Satellite Server that can be unregistered and deleted after upgrade. In addition, it will detect hosts which are not assigned to an organization. If a host is listed under Hosts > All hosts without an organization association and if a content host with same name has an organization already associated with it then the content host will automatically be unregistered. This can be avoided by associating such hosts to an organization before upgrading.

    Run the pre-upgrade check script to get a list of hosts that can be deleted after upgrading. If any unassociated hosts are found, associating them to an organization before upgrading is recommended.

    # foreman-rake katello:upgrade_check

    If the upgrade check reports a failure due to running tasks, then it is recommended that you wait for the tasks to complete. It is possible to cancel some tasks, but you should follow the guidance in the Red Hat Knowledgebase solution How to manage paused tasks on Red Hat Satellite 6 to understand which tasks are safe to cancel and which are not safe to cancel.

  3. Back up the DNS and DHCP configuration files /etc/zones.conf and /etc/dhcp/dhcpd.conf as the installer only supports one domain or subnet, and therefore restoring changes from these backups might be required.
  4. If you have made manual edits to DNS or DHCP configuration files and do not want to overwrite the changes, run the installer script as follows:

    # satellite-installer --foreman-proxy-dns-managed=false \
    --foreman-proxy-dhcp-managed=false
  5. In the Satellite web UI, navigate to Hosts > Discovered hosts. If there are discovered hosts available, turn them off and then delete all entries under the Discovered hosts page. Select all other organizations in turn using the organization setting menu and repeat this action as required. Reboot these hosts after the upgrade has completed.
  6. Make sure all external Capsule Servers are assigned to an organization, otherwise they might get unregistered due to host-unification changes.
  7. Remove old repositories:

    # rm /etc/yum.repos.d/*
  8. Stop the satellite-maintain services.

    # satellite-maintain service stop
  9. Obtain the latest ISO files by following the Downloading the Binary DVD Images procedure in the Installing Satellite Server from a Disconnected Network guide.
  10. Create directories to serve as a mount point, mount the ISO images, and configure the rhel7-server repository by following the Configuring the Base System with Offline Repositories procedure in the Installing Satellite Server from a Disconnected Network guide. Do not install or update any packages at this stage.
  11. Configure the Satellite 6.6 repository from the ISO file.

    1. Copy the ISO file’s repository data file for the Red Hat Satellite packages:

      # cp /media/sat6/media.repo /etc/yum.repos.d/sat6.repo
    2. Edit the /etc/yum.repos.d/sat6.repo file:

      # vi /etc/yum.repos.d/sat6.repo
      1. Change the default InstallMedia repository name to Satellite-6.6:

        [Satellite-6.6]
      2. Add the baseurl directive:

        baseurl=file:///media/sat6/
  12. Configure the Red Hat Software Collections repository from the ISO file.

    1. Copy the ISO file’s repository data file for Red Hat Software Collections packages:

      # cp /media/sat6/RHSCL/media.repo /etc/yum.repos.d/RHSCL.repo
    2. Edit the /etc/yum.repos.d/RHSCL.repo file:

      # vi /etc/yum.repos.d/RHSCL.repo
      1. Change the default InstallMedia repository name to RHSCL:

        [RHSCL]
      2. Add the baseurl directive:

        baseurl=file:///media/sat6/RHSCL/
  13. Configure the Red Hat Satellite Maintenance repository from the ISO file.

    1. Copy the ISO file’s repository data file for Red Hat Satellite Maintenance packages:

      # cp /media/sat6/sat-maintenance/media.repo /etc/yum.repos.d/sat-maintenance.repo
    2. Edit the /etc/yum.repos.d/sat-maintenance.repo file:

      # vi /etc/yum.repos.d/sat-maintenance.repo
      1. Change the default InstallMedia repository name to Satellite-Maintenance:

        [Satellite-Maintenance]
      2. Add the baseurl directive:

        baseurl=file:///media/sat6/sat-maintenance/
  14. Optional: If you have applied custom Apache server configurations, note that the custom configurations are reverted to the installation defaults when you perform the upgrade.

    To preview the changes that are applied during the upgrade, enter the satellite-installer command with the --noop (no operation) option. These changes are applied when you enter the satellite-maintain upgrade command in a following step.

    1. Add the following line to the /etc/httpd/conf/httpd.conf configuration file.

      Include /etc/httpd/conf.modules.d/*.conf
    2. Restart the httpd service.

      # systemctl restart httpd
    3. Start the postgresql and rh-mongodb34-mongod database services.

      # systemctl start postgresql
      # systemctl start rh-mongodb34-mongod
    4. Enter the satellite-installer command with the --noop option:

      # satellite-installer --scenario satellite --upgrade --verbose --noop

      Review the /var/log/foreman-installer/satellite.log to preview the changes that are applied during the upgrade. Locate the +++ and --- symbols that indicate the changes to the configurations files. Although entering the satellite-installer command with the --noop option does not apply any changes to your Satellite, some Puppet resources in the module expect changes to be applied and might display failure messages.

    5. Stop the satellite-maintain services:

      # satellite-maintain service stop
  15. Because of the lengthy upgrade time, use a utility such as screen to suspend and reattach a communication session. You can then check the upgrade progress without staying connected to the command shell continuously. For more information about using the screen command, see How do I use the screen command? article in the Red Hat Knowledge Base.

    If you lose connection to the command shell where the upgrade command is running you can see the logs in /var/log/foreman-installer/satellite.log to check if the process completed successfully.

  16. Clean yum cache:

    # yum clean all
  17. Ensure that the rubygem-foreman_maintain package that provides satellite-maintain is installed and up to date:

    # yum install rubygem-foreman_maintain
  18. Check the available versions to confirm the version you want is listed:

    # satellite-maintain upgrade list-versions
  19. Use the health check option to determine if the system is ready for upgrade. When prompted, enter the hammer admin user credentials to configure satellite-maintain with hammer credentials. These changes are applied to the /etc/foreman-maintain/foreman-maintain-hammer.yml file.

    # satellite-maintain upgrade check --target-version 6.6 \
    --whitelist="repositories-validate,repositories-setup"

    Review the results and address any highlighted error conditions before performing the upgrade.

  20. Perform the upgrade:

    # satellite-maintain upgrade run --target-version 6.6 \
    --whitelist="repositories-validate,repositories-setup"
    Warning

    If you run the command from a directory containing a config subdirectory, you will encounter the following error:

    ERROR: Scenario (config/satellite.yaml) was not found, can not continue.

    In such a case, change directory, for example to the root user’s home directory, and run the command again.

    If the script fails due to missing or outdated packages, you must download and install these separately. For more information, see the Downloading Packages Manually section in the Installing Satellite Server from a Disconnected Network guide.

  21. If using a BASH shell, after a successful or failed upgrade, enter:

    # hash -d satellite-maintain service 2> /dev/null
  22. Check when the kernel packages were last updated:

    # rpm -qa --last | grep kernel
  23. Optional: If a kernel update occurred since the last reboot, stop the satellite-maintain services and reboot the system:

    # satellite-maintain service stop
    # reboot
  24. Check and restore any changes required to the DNS and DHCP configuration files using the backups that you make.
  25. If you make changes in the previous step, restart the satellite-maintain services.

    # satellite-maintain service restart
  26. If you have the OpenSCAP plug-in installed, but do not have the default OpenSCAP content available, enter the following command.

    # foreman-rake foreman_openscap:bulk_upload:default
  27. In the Satellite web UI, go to Configure > Discovery Rules and associate selected organizations and locations with discovery rules.

3.2. Synchronizing the New Repositories

You must enable and synchronize the new 6.6 repositories before you can upgrade Capsule Servers and Satellite clients.

Procedure

  1. In the Satellite web UI, navigate to Content > Red Hat Repositories.
  2. Toggle the Recommended Repositories switch to the On position.
  3. From the list of results, expand the following repositories and click the Enable icon to enable the repositories:

    • To upgrade Satellite clients, enable the Red Hat Satellite Tools 6.6 repositories for all Red Hat Enterprise Linux versions that clients use.
    • If you have Capsule Servers, to upgrade them, enable the following repositories too:

      Red Hat Satellite Capsule 6.6 (for RHEL 7 Server) (RPMs)

      Red Hat Satellite Maintenance 6 (for RHEL 7 Server) (RPMs)

      Red Hat Ansible Engine 2.8 RPMs for Red Hat Enterprise Linux 7 Server

      Red Hat Software Collections RPMs for Red Hat Enterprise Linux 7 Server

    Note

    If the 6.6 repositories are not available, refresh the Subscription Manifest. Navigate to Content > Subscriptions, click Manage Manifest, then click Refresh.

  4. Navigate to Content > Sync Status.
  5. Click the arrow next to the product to view the available repositories.
  6. Select the repositories for 6.6.
  7. Click Synchronize Now.

    Important

    If an error occurs when you try to synchronize a repository, refresh the manifest. If the problem persists, raise a support request. Do not delete the manifest from the Customer Portal or in the Satellite web UI; this removes all the entitlements of your content hosts.

  8. If you use Content Views to control updates to the base operating system of Capsule Server, update those Content Views with new repositories, publish, and promote their updated versions. For more information, see Managing Content Views in the Content Management Guide.

3.3. Upgrading Capsule Servers

This section describes how to upgrade Capsule Servers from 6.5 to 6.6.

Before You Begin

  • You must upgrade Satellite Server before you can upgrade any Capsule Servers.
  • Ensure the Red Hat Satellite Capsule 6.6 repository is enabled in Satellite Server and synchronized.
  • Ensure that you synchronize the required repositories on Satellite Server. For more information, see Section 3.2, “Synchronizing the New Repositories”.
  • If you use Content Views to control updates to the base operating system of Capsule Server, update those Content Views with new repositories and publish their updated versions. For more information, see Managing Content Views in the Content Management Guide.
  • Ensure the Capsule’s base system is registered to the newly upgraded Satellite Server.
  • Ensure the Capsule has the correct organization and location settings in the newly upgraded Satellite Server.
  • Review and update your firewall configuration prior to upgrading your Capsule Server. For more information, see Ports and Firewalls Requirements in Installing Capsule Server.
Warning

If you implemented custom certificates, you must retain the content of both the /root/ssl-build directory and the directory in which you created any source files associated with your custom certificates.

Failure to retain these files during an upgrade causes the upgrade to fail. If these files have been deleted, they must be restored from a backup in order for the upgrade to proceed.

Upgrading Capsule Servers

  1. Create a backup.

  2. Back up the DNS and DHCP configuration files /etc/zones.conf and /etc/dhcp/dhcpd.conf as the installer only supports one domain or subnet, and therefore restoring changes from these backups might be required.
  3. If you have made manual edits to DNS or DHCP configuration files and do not want to overwrite the changes, enter the following command:

    # satellite-installer --foreman-proxy-dns-managed=false \
    --foreman-proxy-dhcp-managed=false
  4. Disable all repositories:

    # subscription-manager repos --disable "*"
  5. Enter the following command to enable new repositories:

    The Red Hat Software Collections repository provides a later version of Ruby required by some Red Hat Satellite features, including the Remote Execution feature. The Satellite tools repository provides katello-host-tools and katello-agent which provide communication services for managing Errata.

    # subscription-manager repos \
    --enable rhel-7-server-rpms \
    --enable rhel-7-server-satellite-capsule-6.6-rpms \
    --enable rhel-server-rhscl-7-rpms \
    --enable rhel-7-server-satellite-tools-6.6-rpms \
    --enable rhel-7-server-satellite-maintenance-6-rpms \
    --enable rhel-7-server-ansible-2.8-rpms
  6. In the Satellite web UI, go to Hosts > Discovered hosts. If there are discovered hosts available, power off the hosts and then delete all entries under the Discovered hosts page. Select all other organizations in turn using the organization setting menu and repeat this action as required. Reboot these hosts after the upgrade has completed.
  7. Clear the repository cache.

    # yum clean all
  8. Stop the satellite-maintain services.

    # satellite-maintain service stop
  9. Until BZ#1649764 is resolved, update the gofer package:

    # yum update gofer
  10. Restart goferd.

    # systemctl restart goferd
  11. Update all packages.

    # yum update
  12. If you plan to use Capsule Server as a proxy for discovered hosts, install the Discovery plug-in.

    # yum install rubygem-smart_proxy_discovery.noarch
  13. On the Capsule Server, verify that the foreman_url setting is correct.

    # grep foreman_url /etc/foreman-proxy/settings.yml

    The fully qualified domain name of the Satellite Server should display.

  14. Perform the upgrade by running the installer script with the --upgrade option:

    # satellite-installer --scenario capsule --upgrade
    Warning

    If you run the command from a directory containing a config subdirectory, you will encounter the following error:

    ERROR: Scenario (config/capsule.yaml) was not found, can not continue.

    In such a case, change directory, for example to the root user’s home directory, and run the command again.

  15. Check when the kernel packages were last updated:

    # rpm -qa --last | grep kernel
  16. Optional: If a kernel update occurred since the last reboot, stop the satellite-maintain services and reboot the system:

    # satellite-maintain service stop
    # reboot
  17. Check and restore any changes required to the DNS and DHCP configuration files using the backups made earlier.
  18. If you use custom repositories, ensure that you enable these custom repositories after the upgrade completes.
  19. Upgrade the foreman-discovery package on Satellite Server and turn on the hosts that were shut down prior to the upgrade.

3.4. Upgrading Satellite Clients

The Satellite tools repository provides katello-agent and katello-host-tools, which provide communication services for managing Errata.

Currently, the Satellite 6.5 version of katello-agent and other client libraries in the Satellite Tools repository are not formally tested or supported against Satellite 6.6.

For deployments using katello-agent and goferd, update all clients to the new version of katello-agent. For deployments not using katello-agent and goferd, update all clients to the new version of katello-host-tools. Complete this action as soon as possible so that your clients are fully compatible with Satellite Server.

Prerequisites

  • You must have upgraded Satellite Server.
  • You must have enabled the new Satellite Tools repositories on the Satellite.
  • You must have synchronized the new repositories in the Satellite.
  • If you have not previously installed katello-agent on your clients and you want to install, use the manual method. For more information, see Upgrade Satellite Clients Manually.
Warning

If you implemented custom certificates, you must retain the content of both the /root/ssl-build directory and the directory in which you created any source files associated with your custom certificates.

Failure to retain these files during an upgrade causes the upgrade to fail. If these files have been deleted, they must be restored from a backup in order for the upgrade to proceed.

Upgrade Satellite Clients Using the Bulk Repository Set UI:

  1. In the Satellite web UI, navigate to Hosts > Content Hosts and select the Content Hosts that you want to upgrade.
  2. From the Select Action list, select Manage Repository Sets.
  3. From the Repository Sets Management list, select the Red Hat Satellite Tools 6.5 check box.
  4. From the Select Action list, select Override to Disabled, and click Done.
  5. When the process completes, on the same set of hosts from the previous steps, from the Select Action list Manage Repository Sets.
  6. From the Repository Sets Management list, select the Red Hat Satellite Tools 6.6 check box.
  7. From the Select Action list, select Override to Enabled, and click Done.
  8. When the process completes, on the same set of hosts from the previous steps, from the Select Action list, select Manage Packages.
  9. In the Package search field, enter one of the following options depending on your configuration:

    • If your deployment uses katello-agent and goferd, enter katello-agent.
    • If your deployment does not use katello-agent and goferd, enter katello-host-tools.
  10. Until BZ#1649764 is resolved, from the Update list, you must select via remote execution. This is required because if you update the package using the Katello agent, the package update disrupts the communication between the client and Satellite or Capsule Server, which causes the update to fail. For more information, see Running Jobs on Hosts in the Managing Hosts guide.

Upgrade Satellite Clients Manually

  1. Log into the client system.
  2. Disable the repositories for the previous version of Satellite.

    # subscription-manager repos \
    --disable rhel-7-server-satellite-tools-6.5-rpms
  3. Enable the Satellite tools repository for this version of Satellite.

    # subscription-manager repos \
    --enable=rhel-7-server-satellite-tools-6.6-rpms
  4. Depending on your configuration, complete one of the following steps:

    • If your deployment uses katello-agent and goferd, enter the following command to install or upgrade katello-agent:

      # yum install katello-agent
    • If your deployment does not use katello-agent and goferd, enter the following command to install or upgrade katello-host-tools:

      # yum install katello-host-tools

Chapter 4. Post-Upgrade Tasks

Some of the procedures in this section are optional. You can choose to perform only those procedures that are relevant to your installation.

If you use the PXE-based discovery process, then you must complete the discovery upgrade procedure on Satellite and on any Capsule Server with hosts that you want to be listed in Satellite on the Hosts > Discovered hosts page.

4.1. Upgrading Discovery

This section describes updating the PXELinux template and the boot image passed to hosts that use PXE booting to register themselves with Satellite Server.

From Satellite 6.4, provisioning templates now have a separate association with a subnet, and do not default to using the TFTP Capsule for that subnet. If you create subnets after the upgrade, you must specifically enable the Satellite or a Capsule to provide a proxy service for discovery templates and then configure all subnets with discovered hosts to use a specific template Capsule.

During the upgrade, for every subnet with a TFTP proxy enabled, the template Capsule is set to be the same as the TFTP Capsule. After the upgrade, check all subnets to verify this was set correctly.

These procedures are not required if you do not use PXE booting of hosts to enable Satellite to discover new hosts.

4.1.1. Upgrading Discovery on Satellite Server

  1. Update the Discovery template in the Satellite web UI:

    1. Navigate to Hosts > Provisioning templates.
    2. On the PXELinux global default line, click Clone.
    3. Enter a new name for the template in the Name field, for example ACME PXE global default.
    4. In the template editor field, change the line ONTIMEOUT local to ONTIMEOUT discovery and click Submit.
    5. Navigate to Administer > Settings.
    6. Locate Global default PXELinux template and click on its Value.
    7. Select the name of the newly created template from the menu and click the tick button.
    8. Navigate to Hosts > Provisioning templates.
    9. Click Build PXE Default, then click OK.
  2. In the Satellite web UI, go to Configure > Discovery Rules and associate selected organizations and locations with discovery rules.

4.1.2. Upgrading Discovery on Capsule Servers

  1. Verify that the Foreman Discovery package is current on Satellite Server.

    # yum upgrade tfm-rubygem-foreman_discovery
  2. If an update occurred in the previous step, restart the satellite-maintain services.

    # satellite-maintain service restart
  3. Upgrade the Discovery image on the Satellite Capsule that is either connected to the provisioning network with discovered hosts or provides TFTP services for discovered hosts.

    # yum upgrade foreman-discovery-image
  4. On the same instance, install the package which provides the Proxy service, and then restart foreman-proxy service.

    # yum install rubygem-smart_proxy_discovery
    # service foreman-proxy restart
  5. In the Satellite web UI, go to Infrastructure > Capsules and verify that the relevant Capsule lists Discovery in the features column. Select Refresh from the Actions drop-down menu if necessary.
  6. Go to Infrastructure > Subnets and for each subnet on which you want to use discovery:

    1. Click the subnet name.
    2. On the Capsules tab, ensure the Discovery Capsule is set to a Capsule you configured above.

4.1.3. Verifying Subnets have a Template Capsule

Ensure all subnets with discovered hosts have a template Capsule:

  1. In the Satellite web UI, navigate to Infrastructure > Subnets.
  2. Select the subnet you want to check.
  3. On the Capsules tab, ensure a Template Capsule has been set for this subnet.

For more information about configuring subnets with template Capsules, see Configuring Discovery Subnets in the Red Hat Satellite Managing Hosts guide.

4.2. Updating Ansible Playbooks to Remove foreman_params

In previous versions of Satellite, Ansible playbooks used the hash foreman_params to fetch parameters, for example:

- hosts: all
  tasks:
    - name: Print Remote Execution SSH keys
      debug:
        var: foreman_params['remote_execution_ssh_keys']

This caused problems when users wanted to import Ansible playbooks because they had to update all Ansible playbooks to use the foreman_params hash. In Satellite 6.6, the Ansible variable functionality has been enhanced and the foreman_params hash has been removed.

After you upgrade to Satellite 6.6, you must update your Ansible playbooks to remove the foreman_params hash.

Ensure that your Ansible playbooks reference variables in the following manner:

- hosts: all
  tasks:
    - name: Print Remote Execution SSH keys
      debug:
        var: remote_execution_ssh_keys

4.3. Upgrading virt-who

If virt-who is installed on Satellite Server or a Capsule Server, it will be upgraded when they are upgraded. No further action is required. If virt-who is installed elsewhere, it must be upgraded manually.

Before You Begin

If virt-who is installed on a host registered to Satellite Server or a Capsule Server, first upgrade the host to the latest packages available in the Satellite Tools repository. For information about upgrading hosts, see Section 3.4, “Upgrading Satellite Clients”.

Upgrade virt-who Manually

  1. Upgrade virt-who.

    # yum upgrade virt-who
  2. Restart the virt-who service so the new version is activated.

    # systemctl restart virt-who.service

4.4. Removing the Previous Version of the Satellite Tools Repository

After completing the upgrade to Satellite 6.6, the Red Hat Satellite Tools 6.5 repository can be removed from Content Views and then disabled.

Disable Version 6.5 of the Satellite Tools Repository:

  1. In the Satellite web UI, navigate to Content > Red Hat Repositories.
  2. In the Enabled Repositories area, locate Red Hat Satellite Tools 6.5 for RHEL 7 Server RPMs x86_64.
  3. Click the Disable icon to the right.

If the repository is still contained in a Content View then you cannot disable it. Packages from a disabled repository are removed automatically by a scheduled task.

4.5. Upgrading the MongoDB Storage Engine

When you complete the upgrade, you can optionally upgrade the MongoDB storage engine to WiredTiger. Note that if you already use WiredTiger, you do not have to perform this procedure after you upgrade. If you want to use WiredTiger, you must repeat the following procedure on Satellite Server and all Capsule Servers. For more information about the WiredTiger storage engine, see WiredTiger Storage Engine in the MongoDB Manual.

Prerequisites

Before upgrading the storage engine, ensure that the following conditions exist:

  • Create a backup of the MongoDB storage.
  • Ensure that the /var/tmp directory has storage space that is at least twice the size of the /var/lib/mongodb directory.
  • Optional: On high traffic Satellite environments, use MongoDB repair to reclaim disk space. For more information, see the KCS article How to compact MongoDB files and/or reclaim disk space in "/var/lib/mongodb" in Satellite 6?.
  • Optional: On high traffic Satellite environments, use MongoDB compact to reclaim disk space. For more information, see compact in MongoDB Manual.
  • Optional: If you want to verify what version of MongoDB you currently use, enter the following command:

    # mongo pulp_database --eval "db.serverStatus().storageEngine"

Procedure

To upgrade the MongoDB storage engine, enter the following command on Satellite Server and all Capsule Servers:

# satellite-installer --upgrade-mongo-storage-engine

4.6. Reclaiming PostgreSQL Space After an Upgrade

When you complete the upgrade, you can perform a full database vacuum for PostgreSQL on Satellite Server to reclaim space on the migrated databases.

Procedure

On Satellite Server, to reclaim space on a PostgreSQL database, complete the following steps:

  1. To stop all services, except for the postgresql service, enter the following command:

    # satellite-maintain service stop --exclude postgresql
  2. To switch to the postgres user and reclaim space on the database, enter the following command:

    # su - postgres -c 'vacuumdb --full --dbname=foreman'
  3. To start the other services when the vacuum completes, enter the following command:

    # satellite-maintain service start

4.7. Updating Templates, Parameters, Lookup Keys and Values

During the upgrade process, Satellite attempts to locate macros that are deprecated for Satellite 6.6 and converts old syntax to new syntax for the default Satellite templates, parameters, and lookup keys and values. However, Satellite does not convert old syntax in the custom templates that you have created and in the cloned templates.

The process uses simple text replacement, for example:

@host.params['parameter1'] -> host_param('parameter1')
@host.param_true?('parameter1') -> host_param_true?('parameter1')
@host.param_false?('parameter1') -> host_param_false?('parameter1')
@host.info['parameters'] -> host_enc['parameters']
Warning

If you use cloned templates in Satellite, verify whether the cloned templates have diverged from the latest version of the original templates in Satellite. The syntax for the same template can differ between versions of Satellite. If your cloned templates contain outdated syntax, update the syntax to match the latest version of the template.

To ensure that this text replacement does not break or omit any variables in your files during the upgrade, check all templates, parameters, and lookup keys and values for the old syntax and replace manually.

The following error occurs because of old syntax remaining in files after the upgrade:

 undefined method '#params' for Host::Managed::Jail

Fixing the outdated subscription_manager_registration snippet

Satellite 6.4 onwards uses the redhat_register snippet instead of the subscription_manager_registration snippet.

If you upgrade from Satellite 6.3 and earlier, ensure to replace the subscription_manager_registration snippet in your custom templates as follows:

<%= snippet "subscription_manager_registration" %>
               ↓
<%= snippet 'redhat_register' %>

Chapter 5. Updating Satellite Server, Capsule Server, and Content Hosts

Use this chapter to update your existing Satellite Server, Capsule Server, and Content Hosts to a new minor version, for example, from 6.6.0 to 6.6.1.

Updates patch security vulnerabilities and minor issues discovered after code is released, and are often fast and non-disruptive to your operating environment.

Before updating, back up your Satellite Server and all Capsule Servers. For more information, see Backing Up and Restoring Satellite Server and Capsule Server in the Administering Red Hat Satellite guide.

5.1. Updating Satellite Server

Prerequisites

  • Ensure that you have synchronized Satellite Server repositories for Satellite, Capsule, and Satellite Tools.
  • Ensure each external Capsule and Content Host can be updated by promoting the updated repositories to all relevant Content Views.
Warning

If you customize configuration files, manually or use a tool such as Hiera, these customizations are overwritten when the installation script runs during upgrading or updating. You can use the --noop option with the satellite-installer script to test for changes. For more information, see the Red Hat Knowledgebase solution How to use the noop option to check for changes in Satellite config files during an upgrade.

Updating Satellite Server to the Next Minor Version

To Update Satellite Server:

  1. Ensure the Satellite Maintenance repository is enabled:

    # subscription-manager repos --enable \
    rhel-7-server-satellite-maintenance-6-rpms
  2. Clean yum cache:

    # yum clean all
  3. Ensure that the rubygem-foreman_maintain package that provides satellite-maintain is installed and up to date:

    # yum install rubygem-foreman_maintain
  4. Check the available versions to confirm the next minor version is listed:

    # satellite-maintain upgrade list-versions
  5. Use the health check option to determine if the system is ready for upgrade. On first use of this command, satellite-maintain prompts you to enter the hammer admin user credentials and saves them in the /etc/foreman-maintain/foreman-maintain-hammer.yml file.

    # satellite-maintain upgrade check --target-version 6.6.z

    Review the results and address any highlighted error conditions before performing the upgrade.

  6. Because of the lengthy update time, use a utility such as screen to suspend and reattach a communication session. You can then check the upgrade progress without staying connected to the command shell continuously. For more information about using the screen command, see How do I use the screen command? article in the Red Hat Knowledge Base.

    If you lose connection to the command shell where the upgrade command is running, you can see the logged messages in the /var/log/foreman-installer/satellite.log file to check if the process completed successfully.

  7. Perform the upgrade:

    # satellite-maintain upgrade run --target-version 6.6.z
  8. Check when the kernel packages were last updated:

    # rpm -qa --last | grep kernel
  9. Optional: If a kernel update occurred since the last reboot, stop the satellite-maintain services and reboot the system:

    # satellite-maintain service stop
    # reboot

5.2. Updating Capsule Server

Updating Capsule Servers to the Next Minor Version

Procedure

To update Capsule Server, complete the following steps:

  1. On Capsule Server, list the enabled repositories:

    # subscription-manager repos --list-enabled
  2. Ensure that only the following repositories are enabled:

    rhel-7-server-rpms
    rhel-7-server-satellite-capsule-6.6-rpms
    rhel-server-rhscl-7-rpms
    rhel-7-server-satellite-tools-6.6-rpms
    rhel-7-server-satellite-maintenance-6-rpms
    rhel-7-server-ansible-2.8-rpms

    For more information about disabling and enabling repositories, see Configuring Repositories in Installing Capsule Server. The rhel-7-server-satellite-tools-6.6-rpms repository provides Katello Agent. For more information, see Installing the katello Agent in Installing Capsule Server. The Red Hat Software Collections repository is optional, it is required for the Remote Execution feature.

  3. Stop the satellite-maintain services:

    # satellite-maintain service stop
  4. Until BZ#1649764 is resolved, update the gofer package:

    # yum update gofer
  5. Restart goferd.

    # systemctl restart goferd
  6. Update all packages:

    # yum update

    If a kernel update occurs, make a note to reboot after the upgrade is complete. Do not reboot at this point.

  7. Perform the update by running the installer script with the --upgrade option.

    # satellite-installer --scenario capsule --upgrade
  8. Optional: If a kernel update occurred during the yum update step, stop the satellite-maintain services and reboot the system:

    # satellite-maintain service stop
    # reboot

5.3. Updating Content Hosts

Updating Content Hosts to the Next Minor Version

To Update a Content Host, enter the following commands:

  1. Until BZ#1649764 is resolved, update the gofer package:

    # yum update gofer
  2. Restart goferd:

    # systemctl restart goferd
  3. Update all packages:

    # yum update
  4. Optional: If a kernel update occurred since the last reboot, stop the satellite-maintain services and reboot the system:

    # satellite-maintain service stop
    # reboot

Legal Notice

Copyright © 2020 Red Hat, Inc.
The text of and illustrations in this document are licensed by Red Hat under a Creative Commons Attribution–Share Alike 3.0 Unported license ("CC-BY-SA"). An explanation of CC-BY-SA is available at http://creativecommons.org/licenses/by-sa/3.0/. In accordance with CC-BY-SA, if you distribute this document or an adaptation of it, you must provide the URL for the original version.
Red Hat, as the licensor of this document, waives the right to enforce, and agrees not to assert, Section 4d of CC-BY-SA to the fullest extent permitted by applicable law.
Red Hat, Red Hat Enterprise Linux, the Shadowman logo, the Red Hat logo, JBoss, OpenShift, Fedora, the Infinity logo, and RHCE are trademarks of Red Hat, Inc., registered in the United States and other countries.
Linux® is the registered trademark of Linus Torvalds in the United States and other countries.
Java® is a registered trademark of Oracle and/or its affiliates.
XFS® is a trademark of Silicon Graphics International Corp. or its subsidiaries in the United States and/or other countries.
MySQL® is a registered trademark of MySQL AB in the United States, the European Union and other countries.
Node.js® is an official trademark of Joyent. Red Hat is not formally related to or endorsed by the official Joyent Node.js open source or commercial project.
The OpenStack® Word Mark and OpenStack logo are either registered trademarks/service marks or trademarks/service marks of the OpenStack Foundation, in the United States and other countries and are used with the OpenStack Foundation's permission. We are not affiliated with, endorsed or sponsored by the OpenStack Foundation, or the OpenStack community.
All other trademarks are the property of their respective owners.