Chapter 3. Upgrading Red Hat Satellite

Warning

If you have Satellite 6 installed in a high availability configuration, contact Red Hat Support before upgrading to Satellite 6.5.

Use the following procedures to upgrade your existing Red Hat Satellite to Red Hat Satellite 6.5:

Before upgrading, see Section 1.1, “Prerequisites”.

3.1. Upgrading Satellite Server

This section describes how to upgrade Satellite Server from 6.4 to 6.5. You can upgrade from any minor version of Red Hat Satellite Server 6.4.

Before You Begin

  • Review and update your firewall configuration prior to upgrading your Satellite Server. For more information, see Ports and Firewalls Requirements in Installing Satellite Server from a Connected Network.
  • Ensure that you do not delete the manifest from the Customer Portal or in the Satellite Web UI because this removes all the entitlements of your content hosts.
  • Back up and remove all Foreman hooks before upgrading. Restore any hooks only after Satellite is known to be working after the upgrade is complete.
  • If you have edited any of the default templates, back up the files either by cloning or exporting them. Cloning is the recommended method because that prevents them being overwritten in future updates or upgrades. To confirm if a template has been edited, you can view its History before you upgrade or view the changes in the audit log after an upgrade. In the web UI, Navigate to Monitor > Audits and search for the template to see a record of changes made. If you use the export method, restore your changes by comparing the exported template and the default template, manually applying your changes.

Capsule Considerations

  • If you use Content Views to control updates to a Capsule Server’s base operating system, or for the Capsule Server repository, you must publish updated versions of those Content Views.
Warning

If you implemented custom certificates, you must retain the content of both the /root/ssl-build directory and the directory in which you created any source files associated with your custom certificates.

Failure to retain these files during an upgrade causes the upgrade to fail. If these files have been deleted, they must be restored from a backup in order for the upgrade to proceed.

Configuring the BASH shell

The BASH shell stores the location of a binary in a hash table. During the upgrade, the location of the foreman-maintain script is changed, but BASH does not register this change, and foreman-maintain fails if it calls the script after the change.

  • Optional: Before the upgrade, users of the BASH shell can set the checkhash option temporarily to ensure foreman-maintain works after the installer completes. Enter a command as follows in your BASH shell:

    # shopt -s checkhash
  • After a successful or failed upgrade, in all currently running BASH shells, enter the following command:

    # hash -d foreman-maintain 2> /dev/null

Upgrade Scenarios

You cannot upgrade a self-registered Satellite. You must migrate a self-registered Satellite to the Red Hat Content Delivery Network (CDN) and then perform the upgrade. To migrate a self-registered Satellite to the CDN, see Migrating Self-Registered Satellites in the Satellite 6.3 Upgrading and Updating Red Hat Satellite guide.

FIPS mode

You cannot upgrade Satellite Server from a RHEL base system that is not operating in FIPS mode to a RHEL base system that is operating in FIPS mode.

To run Satellite Server on a RHEL base system operating in FIPS mode, you must install Satellite on a freshly provisioned RHEL base system operating in FIPS mode. For more information, see System Requirements in Installing Satellite Server from a Connected Network.

3.1.1. Upgrading a Connected Satellite Server

Use this procedure for a Satellite Server connected to the Red Hat Content Delivery Network.

Warning

If you customize configuration files, manually or using a tool such as Hiera, these changes are overwritten when the installation script runs during upgrading or updating. You can use the --noop option with the satellite-installer script to test for changes. For more information, see the Red Hat Knowledgebase solution How to use the noop option to check for changes in Satellite config files during an upgrade.

Upgrade Satellite Server

  1. Create a backup.

  2. Back up the DNS and DHCP configuration files /etc/zones.conf and /etc/dhcp/dhcpd.conf as the installer only supports one domain or subnet, and therefore restoring changes from these backups might be required.
  3. If you have made manual edits to DNS or DHCP configuration files and do not want to overwrite the changes, enter the following command:

    # satellite-installer --foreman-proxy-dns-managed=false \
    --foreman-proxy-dhcp-managed=false
  4. In the Satellite web UI, navigate to Hosts > Discovered hosts. On the Discovered Hosts page, power off and then delete the discovered hosts. From the Select an Organization menu, select each organization in turn and repeat the process to power off and delete the discovered hosts. Make a note to reboot these hosts when the upgrade is complete.
  5. Refresh your subscription:

    # subscription-manager refresh
  6. Enable the Satellite Maintenance and Red Hat Enterprise Linux Ansible repositories:

    # subscription-manager repos \
    --enable rhel-7-server-satellite-maintenance-6-rpms \
    --enable rhel-7-server-ansible-2.6-rpms
  7. Enter the following command to install foreman-maintain or to update it to the latest version:

    # yum install rubygem-foreman_maintain
  8. Check the available versions to confirm the version you want is listed:

    # foreman-maintain upgrade list-versions
  9. Use the health check option to determine if the system is ready for upgrade. When prompted, enter the hammer admin user credentials to configure foreman-maintain with hammer credentials. These changes are applied to the /etc/foreman-maintain/foreman-maintain-hammer.yml file.

    # foreman-maintain upgrade check --target-version 6.5

    Review the results and address any highlighted error conditions before performing the upgrade.

  10. Because of the lengthy upgrade time, use a utility such as screen to suspend and reattach a communication session. You can then check the upgrade progress without staying connected to the command shell continuously. For more information about using the screen command, see How do I use the screen command? article in the Red Hat Knowledge Base.

    If you lose connection to the command shell where the upgrade command is running you can see the logged messages in the /var/log/foreman-installer/satellite.log file to check if the process completed successfully.

  11. Perform the upgrade:

    # foreman-maintain upgrade run --target-version 6.5
  12. Check when the kernel packages were last updated:

    # rpm -qa --last | grep kernel
  13. If a kernel update occurred since the last reboot, reboot the system:

    # reboot
  14. If using a BASH shell, after a successful or failed upgrade, enter:

    # hash -d foreman-maintain service 2> /dev/null
  15. Check and restore any changes required to the DNS and DHCP configuration files using the backups that you make.
  16. If you make changes in the previous step, restart the foreman-maintain services.

    # foreman-maintain service restart
  17. If you have the OpenSCAP plug-in installed, but do not have the default OpenSCAP content available, enter the following command:

    # foreman-rake foreman_openscap:bulk_upload:default

3.1.2. Upgrading a Disconnected Satellite Server

Use this procedure for a Satellite Server not connected to the Red Hat Content Delivery Network.

Warning

If you customize configuration files, manually or using a tool such as Hiera, these changes are overwritten when the installation script runs during upgrading or updating. You can use the --noop option with the satellite-installer script to test for changes. For more information, see the Red Hat Knowledgebase solution How to use the noop option to check for changes in Satellite config files during an upgrade.

Before You Begin

  • Review and update your firewall configuration before upgrading your Satellite Server. For more information, see Ports and Firewalls Requirements in Installing Satellite Server from a Disconnected Network.
  • Ensure that you do not delete the manifest from the Customer Portal or in the Satellite Web UI because this removes all the entitlements of your content hosts.
  • Back up and remove all Foreman hooks before upgrading. Reinstate hooks only after Satellite is known to be working after the upgrade is complete.
Warning

If you implemented custom certificates, you must retain the content of both the /root/ssl-build directory and the directory in which you created any source files associated with your custom certificates.

Failure to retain these files during an upgrade causes the upgrade to fail. If these files have been deleted, they must be restored from a backup in order for the upgrade to proceed.

Upgrade Disconnected Satellite Server

  1. Create a backup.

    • On a virtual machine, take a snapshot.
    • On a physical machine, create a backup.
  2. A pre-upgrade script is available to detect conflicts and list hosts which have duplicate entries in Satellite Server that can be unregistered and deleted after upgrade. In addition, it will detect hosts which are not assigned to an organization. If a host is listed under Hosts > All hosts without an organization association and if a content host with same name has an organization already associated with it then the content host will automatically be unregistered. This can be avoided by associating such hosts to an organization before upgrading.

    Run the pre-upgrade check script to get a list of hosts that can be deleted after upgrading. If any unassociated hosts are found, associating them to an organization before upgrading is recommended.

    # foreman-rake katello:upgrade_check

    If the upgrade check reports a failure due to running tasks, then it is recommended that you wait for the tasks to complete. It is possible to cancel some tasks, but you should follow the guidance in the Red Hat Knowledgebase solution How to manage paused tasks on Red Hat Satellite 6 to understand which tasks are safe to cancel and which are not safe to cancel.

  3. Back up the DNS and DHCP configuration files /etc/zones.conf and /etc/dhcp/dhcpd.conf as the installer only supports one domain or subnet, and therefore restoring changes from these backups might be required.
  4. If you have made manual edits to DNS or DHCP configuration files and do not want to overwrite the changes, run the installer script as follows:

    # satellite-installer --foreman-proxy-dns-managed=false \
    --foreman-proxy-dhcp-managed=false
  5. In the Satellite web UI, navigate to Hosts > Discovered hosts. If there are discovered hosts available, turn them off and then delete all entries under the Discovered hosts page. Select all other organizations in turn using the organization setting menu and repeat this action as required. Reboot these hosts after the upgrade has completed.
  6. Make sure all external Capsule Servers are assigned to an organization, otherwise they might get unregistered due to host-unification changes.
  7. Remove old repositories:

    # rm /etc/yum.repos.d/*
  8. Stop the foreman-maintain services.

    # foreman-maintain service stop
  9. Obtain the latest ISO files by following the Downloading the Binary DVD Images procedure in the Installing Satellite Server from a Disconnected Network guide.
  10. Create directories to serve as a mount point, mount the ISO images, and configure the rhel7-server repository by following the Configuring the Base System with Offline Repositories procedure in the Installing Satellite Server from a Disconnected Network guide. Do not install or update any packages at this stage.
  11. Configure the Satellite 6.5 repository from the ISO file.

    1. Copy the ISO file’s repository data file for the Red Hat Satellite packages:

      # cp /media/sat6/media.repo /etc/yum.repos.d/sat6.repo
    2. Edit the /etc/yum.repos.d/sat6.repo file:

      # vi /etc/yum.repos.d/sat6.repo
      1. Change the default InstallMedia repository name to Satellite-6.5:

        [Satellite-6.5]
      2. Add the baseurl directive:

        baseurl=file:///media/sat6/
  12. Configure the Red Hat Software Collections repository from the ISO file.

    1. Copy the ISO file’s repository data file for Red Hat Software Collections packages:

      # cp /media/sat6/RHSCL/media.repo /etc/yum.repos.d/RHSCL.repo
    2. Edit the /etc/yum.repos.d/RHSCL.repo file:

      # vi /etc/yum.repos.d/RHSCL.repo
      1. Change the default InstallMedia repository name to RHSCL:

        [RHSCL]
      2. Add the baseurl directive:

        baseurl=file:///media/sat6/RHSCL/
  13. Configure the Red Hat Satellite Maintenance repository from the ISO file.

    1. Copy the ISO file’s repository data file for Red Hat Satellite Maintenance packages:

      # cp /media/sat6/sat-maintenance/media.repo /etc/yum.repos.d/sat-maintenance.repo
    2. Edit the /etc/yum.repos.d/sat-maintenance.repo file:

      # vi /etc/yum.repos.d/sat-maintenance.repo
      1. Change the default InstallMedia repository name to Satellite-Maintenance:

        [Satellite-Maintenance]
      2. Add the baseurl directive:

        baseurl=file:///media/sat6/sat-maintenance/
  14. If you have custom Apache server configurations, they will be reverted to the installation defaults in the next step. If you want to see what will be changed when you perform the upgrade, you can enter the upgrade command with the --noop (no operation) option and review the changes that will be applied when you enter the upgrade command in the following step. If you choose not to do this test, skip to the next step now. Alternatively, proceed as follows:

    1. Add the following line to the /etc/httpd/conf/httpd.conf configuration file.

      Include /etc/httpd/conf.modules.d/*.conf
    2. Restart the httpd service.

      # systemctl restart httpd
    3. Start the postgresql and rh-mongodb34-mongod database services.

      # systemctl start postgresql
      # systemctl start rh-mongodb34-mongod
    4. Run the installer script with the --noop option:

      # satellite-installer --scenario satellite --upgrade --verbose --noop

      Review the /var/log/foreman-installer/satellite.log to see what changes would be applied if the --noop option was omitted. Look for the +++ and --- symbols indicating changes to configurations files. Because the above "no operation" option does not actually create the files, and some Puppet resources in the module expect them to be there, some failure messages are to be expected.

    5. Stop the foreman-maintain services:

      # foreman-maintain service stop
  15. Because of the lengthy upgrade time, use a utility such as screen to suspend and reattach a communication session. You can then check the upgrade progress without staying connected to the command shell continuously. For more information about using the screen command, see How do I use the screen command? article in the Red Hat Knowledge Base.

    If you lose connection to the command shell where the upgrade command is running you can see the logs in /var/log/foreman-installer/satellite.log to check if the process completed successfully.

  16. Enter the following command to install foreman-maintain or to update it to the latest version:

    # yum install rubygem-foreman_maintain
  17. Check the available versions to confirm the version you want is listed:

    # foreman-maintain upgrade list-versions
  18. Use the health check option to determine if the system is ready for upgrade. When prompted, enter the hammer admin user credentials to configure foreman-maintain with hammer credentials. These changes are applied to the /etc/foreman-maintain/foreman-maintain-hammer.yml file.

    # foreman-maintain upgrade check --target-version 6.5 \
    --whitelist="repositories-validate,repositories-setup"

    Review the results and address any highlighted error conditions before performing the upgrade.

  19. Perform the upgrade:

    # foreman-maintain upgrade run --target-version 6.5 \
    --whitelist="repositories-validate,repositories-setup"
    Warning

    If you run the command from a directory containing a config subdirectory, you will encounter the following error:

    ERROR: Scenario (config/satellite.yaml) was not found, can not continue.

    In such a case, change directory, for example to the root user’s home directory, and run the command again.

    If the script fails due to missing or outdated packages, you must download and install these separately. For more information, see the Downloading Packages Manually section in the Installing Satellite Server from a Disconnected Network guide.

  20. If using a BASH shell, after a successful or failed upgrade, enter:

    # hash -d foreman-maintain service 2> /dev/null
  21. Check when the kernel packages were last updated:

    # rpm -qa --last | grep kernel
  22. If a kernel update occurred since the last reboot, reboot the system:

    # reboot
  23. Check and restore any changes required to the DNS and DHCP configuration files using the backups that you make.
  24. If you make changes in the previous step, restart the foreman-maintain services.

    # foreman-maintain service restart
  25. If you have the OpenSCAP plug-in installed, but do not have the default OpenSCAP content available, enter the following command.

    # foreman-rake foreman_openscap:bulk_upload:default
  26. In the Satellite web UI, go to Configure > Discovery Rules and associate selected organizations and locations with discovery rules.

3.2. Synchronizing the New Repositories

You must enable and synchronize the new 6.5 repositories before you can upgrade Capsule Servers and Satellite clients.

Procedure

  1. In the Satellite web UI, navigate to Content > Red Hat Repositories.
  2. In the Search field, enter Satellite 6.5.
  3. From the list of results, expand the following repositories and click the Enable icon to enable the repositories:

    • To upgrade Satellite clients, enable the Red Hat Satellite Tools 6.5 repositories for all Red Hat Enterprise Linux versions that clients use.
    • If you have Capsule Servers, to upgrade them, enable the following repositories too:

      Red Hat Satellite Capsule 6.5 (for RHEL 7 Server) (RPMs)

      Red Hat Satellite Maintenance 6 (for RHEL 7 Server) (RPMs)

      Red Hat Ansible Engine 2.6 RPMs for Red Hat Enterprise Linux 7 Server

      Red Hat Software Collections RPMs for Red Hat Enterprise Linux 7 Server

    Note

    If the 6.5 repositories are not available, refresh the Subscription Manifest. Navigate to Content > Subscriptions, click Manage Manifest, then click Refresh.

  4. Navigate to Content > Sync Status.
  5. Click the arrow next to the product to view the available repositories.
  6. Select the repositories for 6.5.
  7. Click Synchronize Now.

    Important

    If an error occurs when you try to synchronize a repository, refresh the manifest. If the problem persists, raise a support request. Do not delete the manifest from the Customer Portal or in the Satellite web UI; this removes all the entitlements of your content hosts.

  8. If you use Content Views to control updates to the base operating system of Capsule Server, update those Content Views with new repositories, publish, and promote their updated versions. For more information, see Managing Content Views in the Content Management Guide.

3.3. Upgrading Capsule Servers

This section describes how to upgrade Capsule Servers from 6.4 to 6.5.

Before You Begin

  • You must upgrade Satellite Server before you can upgrade any Capsule Servers.
  • Ensure the Red Hat Satellite Capsule 6.5 repository is enabled in Satellite Server and synchronized.
  • Ensure that you synchronize the required repositories on Satellite Server. For more information, see Section 3.2, “Synchronizing the New Repositories”.
  • If you use Content Views to control updates to the base operating system of Capsule Server, update those Content Views with new repositories and publish their updated versions. For more information, see Managing Content Views in the Content Management Guide.
  • Ensure the Capsule’s base system is registered to the newly upgraded Satellite Server.
  • Ensure the Capsule has the correct organization and location settings in the newly upgraded Satellite Server.
  • Review and update your firewall configuration prior to upgrading your Capsule Server. For more information, see Ports and Firewalls Requirements in Installing Capsule Server.
Warning

If you implemented custom certificates, you must retain the content of both the /root/ssl-build directory and the directory in which you created any source files associated with your custom certificates.

Failure to retain these files during an upgrade causes the upgrade to fail. If these files have been deleted, they must be restored from a backup in order for the upgrade to proceed.

Upgrading Capsule Servers

  1. Create a backup.

  2. Back up the DNS and DHCP configuration files /etc/zones.conf and /etc/dhcp/dhcpd.conf as the installer only supports one domain or subnet, and therefore restoring changes from these backups might be required.
  3. If you have made manual edits to DNS or DHCP configuration files and do not want to overwrite the changes, enter the following command:

    # satellite-installer --foreman-proxy-dns-managed=false \
    --foreman-proxy-dhcp-managed=false
  4. Disable all repositories:

    # subscription-manager repos --disable "*"
  5. Enter the following command to enable new repositories:

    The Red Hat Software Collections repository provides a later version of Ruby required by some Red Hat Satellite features, including the Remote Execution feature. The Satellite tools repository provides gofer and katello-agent which provide communication services for managing Errata.

    # subscription-manager repos \
    --enable rhel-7-server-rpms \
    --enable rhel-7-server-satellite-capsule-6.5-rpms \
    --enable rhel-server-rhscl-7-rpms \
    --enable rhel-7-server-satellite-tools-6.5-rpms \
    --enable rhel-7-server-satellite-maintenance-6-rpms \
    --enable rhel-7-server-ansible-2.6-rpms
  6. In the Satellite web UI, go to Hosts > Discovered hosts. If there are discovered hosts available, power off the hosts and then delete all entries under the Discovered hosts page. Select all other organizations in turn using the organization setting menu and repeat this action as required. Reboot these hosts after the upgrade has completed.
  7. Clear the repository cache.

    # yum clean all
  8. Stop the foreman-maintain services.

    # foreman-maintain service stop
  9. Until BZ#1649764 is resolved, update the gofer package:

    # yum update gofer
  10. Restart goferd.

    # systemctl restart goferd
  11. Update all packages.

    # yum update
  12. If you plan to use Capsule Server as a proxy for discovered hosts, install the Discovery plug-in.

    # yum install rubygem-smart_proxy_discovery.noarch
  13. On the Capsule Server, verify that the foreman_url setting is correct.

    # grep foreman_url /etc/foreman-proxy/settings.yml

    The fully qualified domain name of the Satellite Server should display.

  14. Perform the upgrade by running the installer script with the --upgrade option:

    # satellite-installer --scenario capsule --upgrade
    Warning

    If you run the command from a directory containing a config subdirectory, you will encounter the following error:

    ERROR: Scenario (config/capsule.yaml) was not found, can not continue.

    In such a case, change directory, for example to the root user’s home directory, and run the command again.

  15. Check when the kernel packages were last updated:

    # rpm -qa --last | grep kernel
  16. If a kernel update occurred since the last reboot, reboot the system:

    # reboot
  17. Check and restore any changes required to the DNS and DHCP configuration files using the backups made earlier.
  18. Upgrade the foreman-discovery package on Satellite Server and turn on the hosts that were shut down prior to the upgrade.

3.4. Upgrading Satellite Clients

Currently, the Satellite 6.4 version of katello-agent and other client libraries in the Satellite Tools repository are not formally tested or supported against Satellite 6.5.

Upgrade all clients to the new version of katello-agent as soon as possible so that your clients are fully compatible with Satellite Server. This requires changing the Satellite Tools repository from 6.4 to 6.5. In the Satellite web UI, navigate to Content > Red Hat Repositories to change the repository.

Prerequisites

  • You must have upgraded Satellite Server.
  • You must have enabled the new Satellite Tools repositories on the Satellite.
  • You must have synchronized the new repositories in the Satellite.
  • If you have not previously installed katello-agent on your clients, use the manual method.
Warning

If you implemented custom certificates, you must retain the content of both the /root/ssl-build directory and the directory in which you created any source files associated with your custom certificates.

Failure to retain these files during an upgrade causes the upgrade to fail. If these files have been deleted, they must be restored from a backup in order for the upgrade to proceed.

Upgrade Satellite Clients Using the Bulk Repository Set UI:

  1. In the Satellite web UI, navigate to Hosts > Content Hosts and select the Content Hosts that you want to upgrade.
  2. From the Select Action list, select Manage Repository Sets.
  3. From the Repository Sets Management list, select the Red Hat Satellite Tools 6.4 check box.
  4. From the Select Action list, select Override to Disabled, and click Done.
  5. When the process completes, on the same set of hosts from the previous steps, select Manage Repository Sets.
  6. From the Repository Sets Management list, select the Red Hat Satellite Tools 6.5 check box.
  7. From the Select Action list, select Override to Enabled, and click Done.
  8. When the process completes, on the same set of hosts from the previous steps, from the Select Action list, select Manage Packages.
  9. In the Package search field, enter katello-agent.
  10. Until BZ#1649764 is resolved, from the Update list, you must select via remote execution. This is required because if you update the package using the Katello agent, the package update disrupts the communication between the client and Satellite or Capsule Server, which causes the update to fail. For more information, see Running Jobs on Hosts in the Managing Hosts guide.

Upgrade Satellite Clients Manually

  1. Log into the client system.
  2. Disable the repositories for the previous version of Satellite.

    # subscription-manager repos \
    --disable rhel-7-server-satellite-tools-6.4-rpms
  3. Enable the Satellite tools repository for this version of Satellite.

    # subscription-manager repos \
    --enable=rhel-7-server-satellite-tools-6.5-rpms
  4. Upgrade the Katello agent:

    # yum upgrade katello-agent