Chapter 3. Upgrading Red Hat Satellite

Warning

If you have Satellite 6 installed in a high availability configuration, contact Red Hat Support before upgrading to Satellite 6.10.

Use the following procedures to upgrade your existing Red Hat Satellite to Red Hat Satellite 6.10:

Before upgrading, see Section 1.1, “Prerequisites”.

3.1. Upgrading Satellite Server

This section describes how to upgrade Satellite Server from 6.9 to 6.10.

Update your Satellite Server 6.9 to the latest 6.9.z release before proceeding. You can use the following command to update to the latest z-stream.

# satellite-maintain upgrade run --target-version 6.9.z

Ensure that you have updated to the latest z-stream before upgrading to 6.10. For more information about updating Satellite Server, see Section 5.1, “Updating Satellite Server”.

Although the process of upgrading from 6.9 to 6.10 also migrates Pulp content, this can take some considerable time. You can reduce this time by following the procedure in Section 3.1.1, “Preparing to Migrate Content to Pulp 3”

If you have not pre-migrated content from Pulp 2 to Pulp 3, you must upgrade to the latest version of Satellite Server before starting to migrate Pulp content.

Before You Begin

  • Note that you can upgrade Capsules separately from Satellite. For more information, see Section 1.4, “Upgrading Capsules Separately from Satellite”.
  • Review and update your firewall configuration prior to upgrading your Satellite Server. For more information, see Preparing your environment for installation in Installing Satellite Server.
  • Ensure that you do not delete the manifest from the Customer Portal or in the Satellite web UI because this removes all the entitlements of your content hosts.
  • Back up and remove all Foreman hooks before upgrading. Restore any hooks only after Satellite is known to be working after the upgrade is complete. Note that Foreman Hooks functionality is deprecated and will be removed in the next Satellite version.
  • If you have edited any of the default templates, back up the files either by cloning or exporting them. Cloning is the recommended method because that prevents them being overwritten in future updates or upgrades. To confirm if a template has been edited, you can view its History before you upgrade or view the changes in the audit log after an upgrade. In the Satellite web UI, navigate to Monitor > Audits and search for the template to see a record of changes made. If you use the export method, restore your changes by comparing the exported template and the default template, manually applying your changes.
  • Pulp content migration occurs at 6.9, before the upgrade to 6.10.
  • Ensure there is sufficient storage space for /var/lib/pulp/published to double in size before starting the migration process. Check the size of /var/lib/pulp/published with:

    # du -sh /var/lib/pulp/published/
  • Content migration can be run online, but uses processor, disk, and memory resources. Sync and content view publishing operations might take longer as a result.
  • If you have not pre-migrated Pulp content, the PULP_CONTENT_PREMIGRATION_BATCH_SIZE setting defines the number of content units processed at the same time. It affects the amount of RAM used and the I/O load. The lower the value, the longer satellite-maintain content prepare takes to complete. The upgrade downtime is also longer if you have content left to migrate at that point.

    • The default value is 1000.
    • If the system has a hard disk drive, or there are concerns about I/O load, the recommended value is 50.
    • A lower value is not recommended.
  • Use the following method to set PULP_CONTENT_PREMIGRATION_BATCH_SIZE:

    1. Create a directory and file:

      $ mkdir /etc/systemd/system/pulpcore-worker@.service.d/
      $ vi  /etc/systemd/system/pulpcore-worker@.service.d/settings.conf

      with contents:

      [Service]
      User=pulp
      Environment=PULP_CONTENT_PREMIGRATION_BATCH_SIZE=1000
    2. Restart the satellite-maintain services using:

      # systemctl daemon-reload
      # satellite-maintain service restart

      For further information, see Preparing to Migrate Content to Pulp 3.

Capsule Considerations

  • If you use Content Views to control updates to a Capsule Server’s base operating system, or for Capsule Server repository, you must publish updated versions of those Content Views.
  • Note that Satellite Server upgraded from 6.9 to 6.10 can use Capsule Servers still at 6.9.
Warning

If you implemented custom certificates, you must retain the content of both the /root/ssl-build directory and the directory in which you created any source files associated with your custom certificates.

Failure to retain these files during an upgrade causes the upgrade to fail. If these files have been deleted, they must be restored from a backup in order for the upgrade to proceed.

Upgrade Scenarios

You cannot upgrade a self-registered Satellite. You must migrate a self-registered Satellite to the Red Hat Content Delivery Network (CDN) and then perform the upgrade. To migrate a self-registered Satellite to the CDN, see Migrating Self-Registered Satellites in the Satellite 6.10 Upgrading and Updating Red Hat Satellite guide.

FIPS mode

You cannot upgrade Satellite Server from a RHEL base system that is not operating in FIPS mode to a RHEL base system that is operating in FIPS mode.

To run Satellite Server on a Red Hat Enterprise Linux base system operating in FIPS mode, you must install Satellite on a freshly provisioned RHEL base system operating in FIPS mode. For more information, see Preparing your environment for installation in Installing Satellite Server.

3.1.1. Preparing to Migrate Content to Pulp 3

The time to prepare content for Pulp 3 depends on the amount of content and the number of Content Views. For large systems, this can mean several days of downtime. To prevent this, pre-migrate Pulp content while running the latest version of Satellite Server 6.9. This reduces the overall upgrade downtime.

During migration to Pulp 3, the amount of data stored in the /var/lib/pulp/published/ directory can double in size. Ensure that there is enough space on the /var/lib/pulp volume. If the user runs satellite-maintain prep-6.10-upgrade, the content in /var/lib/pulp/content does not need to be duplicated. After Pulp 2 is removed, the extra space can be claimed back. However, shrinking of XFS volumes is not possible. Copying data to a different partition might be necessary in order to bring volume size down to the previous value.

During migration to Pulp 3, the amount of data stored in the PostgreSQL data directory can grow significantly. Ensure that you have enough space for the migration. A rough estimation of the space you require is 2-3 times the size of the MongoDB Pulp 2 database.

Pre-Migration Checks

The sequence should always be if you have any Composite Content Views, take action on those first, and then work on the Content Views.

  • Ensure that all enabled repositories complete the synchronization process.
  • If any repositories are synced with a Warning or Error state, fix the underlying problem and resync the repository.
  • If the Pulp 2 to Pulp 3 migration fails with the error NoMethodError: undefined method `link?' for nil:NilClass, see the Red Hat Knowledgebase solution The Pulp 2 to Pulp 3 migration fails with error "NoMethodError: undefined method `link?' for nil:NilClass" in Red Hat Satellite 6.9 on the customer portal.
  • Ensure that no paused tasks in the system are related to any repositories or Content Views.
  • If some Content View or Composite Content View versions are not required anymore, please consider performing a cleanup of them.
  • For more information about how to perform the cleanup of older Content View or Composite Content View versions using hammer_cli, see How to remove old content view versions in Red Hat Satellite 6 using CLI/hammer?.
  • If you no longer require or use certain repositories, disable those specific repositories.
  • Ensure that all sync plans are disabled till the migration is completed.
  • When the cleanup process finishes, ensure that you clean up the orphan data by following the steps in How to delete orphaned content in /var/lib/pulp on Capsule?.
  • Ensure that you can enter the following command on Satellite without error:

    # foreman-rake katello:correct_repositories COMMIT=true --trace

Use this procedure to begin migrating content from Pulp 2 to Pulp 3.

Procedure

  1. Update the file permissions before upgrading Satellite Server using the following command:

    # satellite-maintain prep-6.10-upgrade

    This might take some time on high latency systems.

  2. View details of the content you are pre-migrating using the following command:

    # satellite-maintain content migration-stats

    Use this command as often as necessary during the migration process to determine how long the process will take. It also identifies corrupted or missing content that might cause the migration to fail. Output is similar to the following:

    Running Retrieve Pulp 2 to Pulp 3 migration statistics
    ============================================
    Retrieve Pulp 2 to Pulp 3 migration statistics:
    ============Migration Summary================
    Migrated/Total RPMs: 0/367633
    Migrated/Total errata: 0/20780140
    Migrated/Total repositories: 0/33924
    Estimated migration time based on yum content: 47 hours, 23 minutes
    Note: ensure there is sufficient storage space for /var/lib/pulp/published to
    double in size before starting the migration process.
    Check the size of /var/lib/pulp/published with 'du -sh /var/lib/pulp/published/'
    Note: ensure there is sufficient storage space for postgresql.
    You will need additional space for your postgresql database.
    The partition holding '/var/opt/rh/rh-postgresql12/lib/pgsql/data/' will need
    additional free space equivalent to the size of your Mongo db database (/var/lib/mongodb/).
    [OK]
  3. Prepare your content for Pulp migration using the following command:

    # satellite-maintain content prepare

    As part of its final step, satellite-maintain content prepare checks whether any content units are unmigrated. If it identifies corrupted or missing content, you might see something similar to the following:

    ============Missing/Corrupted Content Summary================
      WARNING: MISSING OR CORRUPTED CONTENT DETECTED
      Corrupted or Missing Rpm: 1000/104583
      Corrupted or missing content has been detected, you can examine the list of content in /tmp/unmigratable_content-20211025-74422-16cxfae and take action by either:
      1. Performing a 'Verify Checksum' sync under Advanced Sync Options, let it complete, and re-running the migration
      2. Deleting/disabling the affected repositories and running orphan cleanup (foreman-rake katello:delete_orphaned_content) and re-running the migration.
      3. Manually correcting files on the filesystem in /var/lib/pulp/content/ and re-running the migration
      4. Mark currently corrupted or missing content as skipped (foreman-rake katello:approve_corrupted_migration_content). This will skip migration of missing or corrupted content.

    There are several causes of corrupted or missing content:

    • A Repository synchronization or Content View publish or promotion occurred during the migration.
    • Files on disk in /var/lib/pulp/content/ became corrupted due to disk rot.
    • Files in /var/lib/pulp/content/ are missing due to past events such as disk loss with a partial restore.
    • There is some mismatch between subsystems in Satellite such as Katello and Pulp. This can happen if a repository or content view is improperly deleted by skipping steps in a paused Red Hat Satellite Task. Running foreman-rake katello:correct_repositories COMMIT=true can correct this.

      You can review the list of corrupt or missing RPMs written to disk as part of the satellite-maintain content migration-stats command, /tmp/unmigratable_content-20211025-74422-16cxfae20211120-2149-1j4pmfae in the example above.

      For more information about determining which repository the unmigratable contents belong to, see How to determine the repository to run Verify Checksum on for reported corrupted RPMs.

      In most cases, where repositories with unmigratable contents are configured on Satellite with the On-demand download policy and the number of missing or corrupt RPMs are low, you can mark these as skipped using the command:

      # foreman-rake katello:approve_corrupted_migration_content

      If you approve corrupted or missing content and proceed with the upgrade, that content will not appear in the repositories or content view versions after upgrade.

      If these packages exist in the upstream repositories, they are added again when you re-synchronize the repositories after upgrade. They are not restored to the Content Views unless you republish those Content Views. As these packages are likely unusable, the upgrade process only detects that fact.

      If you suspect a Repository synchronization or Content View publish or promotion occurred during the migration, re-running satellite-maintain content prepare migrates the remaining content. If this is the first time you have run satellite-maintain content prepare, Red Hat recommends running it as often as necessary to try to reduce the number of corrupt or missing RPMs. Migrating the remaining content then takes less time.

      Note

      You cannot use Ctrl + C to terminate the satellite-maintain content prepare process. If you attempt to halt the process using Ctrl + C or by disconnecting your SSH session, the process does not terminate but continues in the background. You can use the following command to terminate the process gracefully, whenever necessary, so that you can continue later.

      # satellite-maintain content prepare-abort

      Note that satellite-maintain content prepare-abort can take several minutes to terminate the process. You can continue the migration process using satellite-maintain content prepare whenever it is convenient.

  4. The process does not confirm that migration is complete. You can determine how near to completion the process is by using the following command:

    # satellite-maintain content migration-stats

    at intervals until the indicated migration time is at or near zero.

  5. The final steps of Pulp content migration are completed when upgrading Satellite Server from 6.9 to 6.10.

    Note

    If problems occur, you must restart the pre-migration process from the beginning using the following command:

    # satellite-maintain content migration-reset
  6. The final steps of Pulp content migration are completed when upgrading Satellite Server from 6.9 to 6.10.

3.1.2. Upgrading a Connected Satellite Server

Use this procedure for a Satellite Server with access to the public internet

Warning

If you customize configuration files, manually or using a tool such as Hiera, these changes are overwritten when the installation script runs during upgrading or updating. You can use the --noop option with the satellite-installer script to test for changes. For more information, see the Red Hat Knowledgebase solution How to use the noop option to check for changes in Satellite config files during an upgrade.

Upgrade Satellite Server

  1. Create a backup.

  2. Optional: If you made manual edits to DNS or DHCP configuration in the /etc/zones.conf or /etc/dhcp/dhcpd.conf files, back up the configuration files because the installer only supports one domain or subnet, and therefore restoring changes from these backups might be required.
  3. Optional: If you made manual edits to DNS or DHCP configuration files and do not want to overwrite the changes, enter the following command:

    # satellite-installer --foreman-proxy-dns-managed=false \
    --foreman-proxy-dhcp-managed=false
  4. In the Satellite web UI, navigate to Hosts > Discovered hosts. On the Discovered Hosts page, power off and then delete the discovered hosts. From the Select an Organization menu, select each organization in turn and repeat the process to power off and delete the discovered hosts. Make a note to reboot these hosts when the upgrade is complete.
  5. Ensure that the apache::purge_configs: false entry is either not present or commented out in the /etc/foreman-installer/custom-hiera.yaml file of the Satellite 6.9/Capsule 6.9 servers which will be upgraded to 6.10.
  6. Ensure that the Satellite Maintenance repository is enabled:

    # subscription-manager repos --enable \
    rhel-7-server-satellite-maintenance-6-rpms
  7. Check the available versions to confirm the version you want is listed:

    # satellite-maintain upgrade list-versions
  8. Use the health check option to determine if the system is ready for upgrade. When prompted, enter the hammer admin user credentials to configure satellite-maintain with hammer credentials. These changes are applied to the /etc/foreman-maintain/foreman-maintain-hammer.yml file.

    # satellite-maintain upgrade check --target-version 6.10

    Review the results and address any highlighted error conditions before performing the upgrade.

  9. Because of the lengthy upgrade time, use a utility such as screen to suspend and reattach a communication session. You can then check the upgrade progress without staying connected to the command shell continuously. For more information about using the screen command, see How do I use the screen command? article in the Red Hat Knowledge Base.

    If you lose connection to the command shell where the upgrade command is running you can see the logged messages in the /var/log/foreman-installer/satellite.log file to check if the process completed successfully.

  10. Perform the upgrade:

    # satellite-maintain upgrade run --target-version 6.10
  11. Check when the kernel packages were last updated:

    # rpm -qa --last | grep kernel
  12. Optional: If a kernel update occurred since the last reboot, reboot the system:

    # reboot
  13. If using a BASH shell, after a successful or failed upgrade, enter:

    # hash -d satellite-maintain service 2> /dev/null
  14. If you have migrated content from Pulp 2 to Pulp 3, remove all Pulp 2 content.

    # satellite-maintain content remove-pulp2

    This removes Pulp 2 RPMs, content in /var/lib/pulp/content/, the mongo database, and migration content in the Pulp 3 database.

3.1.3. Upgrading a Disconnected Satellite Server

Use this procedure if your Satellite Server is not connected to the Red Hat Content Delivery Network.

Warning
  • If you customized configuration files, either manually or using a tool such as Hiera, these changes are overwritten when you enter the satellite-maintain command during upgrading or updating. You can use the --noop option with the satellite-installer command to review the changes that are applied during upgrading or updating. For more information, see the Red Hat Knowledgebase solution How to use the noop option to check for changes in Satellite config files during an upgrade.
  • The hammer import and export commands have been replaced with hammer content-import and hammer content-export tooling.

    If you have scripts that are using hammer content-view version export, hammer content-view version export-legacy, hammer repository export, or their respective import commands, you have to adjust them to use the hammer content-export command instead, along with its respective import command.

  • If you implemented custom certificates, you must retain the content of both the /root/ssl-build directory and the directory in which you created any source files associated with your custom certificates.

    Failure to retain these files during an upgrade causes the upgrade to fail. If these files have been deleted, they must be restored from a backup in order for the upgrade to proceed.

Before You Begin

  • Review and update your firewall configuration before upgrading your Satellite Server. For more information, see Ports and Firewalls Requirements in Installing Satellite Server from a Disconnected Network.
  • Ensure that you do not delete the manifest from the Customer Portal or in the Satellite Web UI because this removes all the entitlements of your content hosts.
  • Back up and remove all Foreman hooks before upgrading. Reinstate hooks only after Satellite is known to be working after the upgrade is complete.
Warning

If you implemented custom certificates, you must retain the content of both the /root/ssl-build directory and the directory in which you created any source files associated with your custom certificates.

Failure to retain these files during an upgrade causes the upgrade to fail. If these files have been deleted, they must be restored from a backup in order for the upgrade to proceed.

Upgrade Disconnected Satellite Server

  1. Create a backup.

    • On a virtual machine, take a snapshot.
    • On a physical machine, create a backup.
  2. A pre-upgrade script is available to detect conflicts and list hosts which have duplicate entries in Satellite Server that can be unregistered and deleted after upgrade. In addition, it will detect hosts which are not assigned to an organization. If a host is listed under Hosts > All hosts without an organization association and if a content host with same name has an organization already associated with it then the content host will automatically be unregistered. This can be avoided by associating such hosts to an organization before upgrading.

    Run the pre-upgrade check script to get a list of hosts that can be deleted after upgrading. If any unassociated hosts are found, associating them to an organization before upgrading is recommended.

    # foreman-rake katello:upgrade_check
  3. Optional: If you made manual edits to DNS or DHCP configuration in the /etc/zones.conf or /etc/dhcp/dhcpd.conf files, back up the configuration files because the installer only supports one domain or subnet, and therefore restoring changes from these backups might be required.
  4. Optional: If you made manual edits to DNS or DHCP configuration files and do not want to overwrite the changes, enter the following command:

    # satellite-installer --foreman-proxy-dns-managed=false \
    --foreman-proxy-dhcp-managed=false
  5. Optional: If you use PostgreSQL as an external database, on the PostgreSQL server, install the rh-postgresql12-postgresql-evr package, which is available from the rhel-7-server-satellite-6.10-rpms repository:

    # yum install rh-postgresql12-postgresql-evr
  6. In the Satellite web UI, navigate to Hosts > Discovered hosts. If there are discovered hosts available, turn them off and then delete all entries under the Discovered hosts page. Select all other organizations in turn using the organization setting menu and repeat this action as required. Reboot these hosts after the upgrade has completed.
  7. Ensure that the apache::purge_configs: false entry is either not present or commented out in the /etc/foreman-installer/custom-hiera.yaml file of the Satellite 6.9/Capsule 6.9 servers which will be upgraded to 6.10.
  8. Make sure all external Capsule Servers are assigned to an organization, otherwise they might get unregistered due to host-unification changes.
  9. Remove old repositories:

    # rm /etc/yum.repos.d/*
  10. Stop the satellite-maintain services.

    # satellite-maintain service stop

For more information with additional steps, see Section 5.2 Updating Disconnected Satellite Server.

  1. Optional: If you have applied custom Apache server configurations, note that the custom configurations are reverted to the installation defaults when you perform the upgrade.

    To preview the changes that are applied during the upgrade, enter the satellite-installer command with the --noop (no operation) option. These changes are applied when you enter the satellite-maintain upgrade command in a following step.

    1. Add the following line to the /etc/httpd/conf/httpd.conf configuration file.

      Include /etc/httpd/conf.modules.d/*.conf
    2. Restart the httpd service.

      # systemctl restart httpd
    3. Start the postgresql and rh-mongodb34-mongod database services.

      # systemctl start postgresql
      # systemctl start rh-mongodb34-mongod
    4. Enter the satellite-installer command with the --noop option:

      # satellite-installer --scenario satellite --verbose --noop

      Review the /var/log/foreman-installer/satellite.log to preview the changes that are applied during the upgrade. Locate the +++ and --- symbols that indicate the changes to the configurations files. Although entering the satellite-installer command with the --noop option does not apply any changes to your Satellite, some Puppet resources in the module expect changes to be applied and might display failure messages.

    5. Stop the satellite-maintain services:

      # satellite-maintain service stop
  2. Because of the lengthy upgrade time, use a utility such as screen to suspend and reattach a communication session. You can then check the upgrade progress without staying connected to the command shell continuously. For more information about using the screen command, see How do I use the screen command? article in the Red Hat Knowledge Base.

    If you lose connection to the command shell where the upgrade command is running you can see the logs in /var/log/foreman-installer/satellite.log to check if the process completed successfully.

  3. Check the available versions to confirm the version you want is listed:

    # satellite-maintain upgrade list-versions
  4. Use the health check option to determine if the system is ready for upgrade. When prompted, enter the hammer admin user credentials to configure satellite-maintain with hammer credentials. These changes are applied to the /etc/foreman-maintain/foreman-maintain-hammer.yml file.

    # satellite-maintain upgrade check --target-version 6.10 \
    --whitelist="repositories-validate,repositories-setup"

    Review the results and address any highlighted error conditions before performing the upgrade.

  5. Perform the upgrade:

    # satellite-maintain upgrade run --target-version 6.10 \
    --whitelist="repositories-validate,repositories-setup"
    Warning

    If you run the command from a directory containing a config subdirectory, you will encounter the following error:

    ERROR: Scenario (config/satellite.yaml) was not found, can not continue.

    In such a case, change directory, for example to the root user’s home directory, and run the command again.

    If the script fails due to missing or outdated packages, you must download and install these separately. For more information, see the Resolving Package Dependency Errors section in the Installing Satellite Server from a Disconnected Network guide.

  6. If using a BASH shell, after a successful or failed upgrade, enter:

    # hash -d satellite-maintain service 2> /dev/null
  7. If you have migrated content from Pulp 2 to Pulp 3, remove all Pulp 2 content:

    # satellite-maintain content remove-pulp2

    This removes uploaded and synchronized Pulp 2 packages, content in /var/lib/pulp/content/, MongoDB, and migration content in the Pulp 3 database.

  8. Check when the kernel packages were last updated:

    # rpm -qa --last | grep kernel
  9. Optional: If a kernel update occurred since the last reboot, stop the satellite-maintain services and reboot the system:

    # satellite-maintain service stop
    # reboot
  10. Optional: If you made manual edits to DNS or DHCP configuration files, check and restore any changes required to the DNS and DHCP configuration files using the backups that you made.
  11. If you make changes in the previous step, restart the satellite-maintain services.

    # satellite-maintain service restart
  12. If you have the OpenSCAP plug-in installed, but do not have the default OpenSCAP content available, enter the following command.

    # foreman-rake foreman_openscap:bulk_upload:default
  13. In the Satellite web UI, go to Configure > Discovery Rules and associate selected organizations and locations with discovery rules.

3.2. Synchronizing the New Repositories

You must enable and synchronize the new 6.10 repositories before you can upgrade Capsule Servers and Satellite clients.

Procedure

  1. In the Satellite web UI, navigate to Content > Red Hat Repositories.
  2. Toggle the Recommended Repositories switch to the On position.
  3. From the list of results, expand the following repositories and click the Enable icon to enable the repositories:

    • To upgrade Satellite clients, enable the Red Hat Satellite Tools 6.10 repositories for all Red Hat Enterprise Linux versions that clients use.
    • If you have Capsule Servers, to upgrade them, enable the following repositories too:

      Red Hat Satellite Capsule 6.10 (for RHEL 7 Server) (RPMs)

      Red Hat Satellite Maintenance 6 (for RHEL 7 Server) (RPMs)

      Red Hat Ansible Engine 2.9 RPMs for Red Hat Enterprise Linux 7 Server

      Red Hat Software Collections RPMs for Red Hat Enterprise Linux 7 Server

    Note

    If the 6.10 repositories are not available, refresh the Subscription Manifest. Navigate to Content > Subscriptions, click Manage Manifest, then click Refresh.

  4. Navigate to Content > Sync Status.
  5. Click the arrow next to the product to view the available repositories.
  6. Select the repositories for 6.10.
  7. Click Synchronize Now.

    Important

    If an error occurs when you try to synchronize a repository, refresh the manifest. If the problem persists, raise a support request. Do not delete the manifest from the Customer Portal or in the Satellite web UI; this removes all the entitlements of your content hosts.

  8. If you use Content Views to control updates to the base operating system of Capsule Server, update those Content Views with new repositories, publish, and promote their updated versions. For more information, see Managing Content Views in the Content Management Guide.

3.3. Upgrading Capsule Servers

This section describes how to upgrade Capsule Servers from 6.9 to 6.10.

Before You Begin

  • You must upgrade Satellite Server before you can upgrade any Capsule Servers. Note that you can upgrade Capsules separately from Satellite. For more information, see Section 1.4, “Upgrading Capsules Separately from Satellite”.
  • Ensure the Red Hat Satellite Capsule 6.10 repository is enabled in Satellite Server and synchronized.
  • Ensure that you synchronize the required repositories on Satellite Server. For more information, see Section 3.2, “Synchronizing the New Repositories”.
  • If you use Content Views to control updates to the base operating system of Capsule Server, update those Content Views with new repositories and publish their updated versions. For more information, see Managing Content Views in the Content Management Guide.
  • Ensure the Capsule’s base system is registered to the newly upgraded Satellite Server.
  • Ensure the Capsule has the correct organization and location settings in the newly upgraded Satellite Server.
  • Review and update your firewall configuration prior to upgrading your Capsule Server. For more information, see Preparing Your Environment for Capsule Installation in Installing Capsule Server.
  • Ensure you create a backup of the /etc/puppetlabs/code/environments file.
Warning

If you implemented custom certificates, you must retain the content of both the /root/ssl-build directory and the directory in which you created any source files associated with your custom certificates.

Failure to retain these files during an upgrade causes the upgrade to fail. If these files have been deleted, they must be restored from a backup in order for the upgrade to proceed.

Upgrading Capsule Servers

  1. Create a backup.

  2. Clean yum cache:

    # yum clean metadata
  3. Ensure that the rubygem-foreman_maintain package that provides satellite-maintain is installed and up to date:

    # yum install rubygem-foreman_maintain
  4. On Capsule Server, verify that the foreman_url setting points to the Satellite FQDN:

    # grep foreman_url /etc/foreman-proxy/settings.yml
  5. Ensure that the apache::purge_configs: false entry is either not present or commented out in the /etc/foreman-installer/custom-hiera.yaml file of the Satellite 6.9/Capsule 6.9 servers which will be upgraded to 6.10.
  6. Check the available versions to confirm the version you want is listed:

    # satellite-maintain upgrade list-versions
  7. Because of the lengthy upgrade time, use a utility such as screen to suspend and reattach a communication session. You can then check the upgrade progress without staying connected to the command shell continuously. For more information about using the screen command, see How do I use the screen command? article in the Red Hat Knowledge Base.

    If you lose connection to the command shell where the upgrade command is running you can see the logged messages in the /var/log/foreman-installer/satellite.log file to check if the process completed successfully.

  8. Use the health check option to determine if the system is ready for upgrade:

    # satellite-maintain upgrade check --target-version 6.10

    Review the results and address any highlighted error conditions before performing the upgrade.

  9. Perform the upgrade:

    # satellite-maintain upgrade run --target-version 6.10
    Warning

    If you run the command from a directory containing a config subdirectory, you will encounter the following error:

    ERROR: Scenario (config/capsule.yaml) was not found, can not continue.

    In such a case, change directory, for example to the root user’s home directory, and run the command again.

  10. If you have migrated content from Pulp 2 to Pulp 3, remove all Pulp 2 content.

    # satellite-maintain content remove-pulp2

    This removes Pulp 2 RPMs, content in /var/lib/pulp/content/, the mongo database, and migration content in the Pulp 3 database.

    Accessing content through this Capsule will fail until a full synchronization of the upgraded Capsule is performed.

  11. Check when the kernel packages were last updated:

    # rpm -qa --last | grep kernel
  12. Optional: If a kernel update occurred since the last reboot, reboot the system:

    # reboot
  13. Optional: If you made manual edits to DNS or DHCP configuration files, check and restore any changes required to the DNS and DHCP configuration files using the backups made earlier.
  14. Optional: If you use custom repositories, ensure that you enable these custom repositories after the upgrade completes.
  15. On Satellite Server, perform a complete synchronization of the upgraded Capsule as the MongoDB and RPM repositories are not automatically migrated with Satellite.

    # hammer capsule content synchronize --name ${Capsule} --skip-metadata-check true --async
    Note

    If you did not synchronize your repositories before upgrading your Satellite Server, running this command fails to synchronize your content with Capsule Servers. In this case, follow Section 3.4, “Synchronizing Capsule Servers Through Satellite Web UI” to synchronize your Capsule Servers.

Upgrading Capsule Servers Using Remote Execution in the Satellite web UI

  1. Create a backup.

  2. Optional: If you made manual edits to DNS or DHCP configuration in the /etc/zones.conf or /etc/dhcp/dhcpd.conf files, back up the configuration files because the installer only supports one domain or subnet, and therefore restoring changes from these backups might be required.
  3. Optional: If you made manual edits to DNS or DHCP configuration files and do not want to overwrite the changes, enter the following command:

    # {foreman-installer} --foreman-proxy-dns-managed=false \
    --foreman-proxy-dhcp-managed=false
  4. In the Satellite web UI, navigate to Monitor > Jobs.
  5. Click Run Job.
  6. From the Job category list, select Maintenance Operations.
  7. From the Job template list, select Capsule Upgrade Playbook.
  8. In the Search Query field, enter the host name of the Capsule.
  9. Ensure that Resolves to shows 1 host.
  10. In the target_version field, enter the target version of the Capsule.
  11. In the whitelist_options field, enter the whitelist options.
  12. For Type of query, click Static Query or Dynamic Query depending on the type of query.
  13. Select the schedule for the job execution in Schedule.

3.4. Synchronizing Capsule Servers Through Satellite Web UI

Use this procedure if you did not synchronize your repositories before upgrading your Satellite Server.

Procedure

  1. In the Satellite web UI, navigate to Infrastructure > Capsules.
  2. Click on the name of Capsule Server you wish to synchronize.
  3. Click Synchronize.
  4. Select Complete Sync.

3.5. Upgrading Satellite Clients

The Satellite Tools 6.10 repository provides katello-agent and katello-host-tools, which provide communication services for managing Errata.

Note

The Katello agent is deprecated and will be removed in a future Satellite version. Migrate your workloads to use the remote execution feature to update clients remotely. For more information, see Migrating from Katello Agent to Remote Execution in the Managing Hosts Guide.

For deployments using katello-agent and goferd, update all clients to the new version of katello-agent. For deployments not using katello-agent and goferd, update all clients to the new version of katello-host-tools. Complete this action as soon as possible so that your clients are fully compatible with Satellite Server.

Prerequisites

  • You must have upgraded Satellite Server.
  • You must have enabled the new Satellite Tools 6.10 repositories on the Satellite.
  • You must have synchronized the new repositories in the Satellite.
  • If you have not previously installed katello-agent on your clients and you want to install it, use the manual method. For more information, see Upgrade Satellite Clients Manually.
Warning

If you implemented custom certificates, you must retain the content of both the /root/ssl-build directory and the directory in which you created any source files associated with your custom certificates.

Failure to retain these files during an upgrade causes the upgrade to fail. If these files have been deleted, they must be restored from a backup in order for the upgrade to proceed.

Upgrade Satellite Clients Using the Bulk Repository Set UI:

  1. In the Satellite web UI, navigate to Hosts > Content Hosts and select the Content Hosts that you want to upgrade.
  2. From the Select Action list, select Manage Repository Sets.
  3. From the Repository Sets Management list, select the Red Hat Satellite Tools 6.9 check box.
  4. From the Select Action list, select Override to Disabled, and click Done.
  5. When the process completes, on the same set of hosts from the previous steps, from the Select Action list, select Manage Repository Sets.
  6. From the Repository Sets Management list, select the Red Hat Satellite Tools 6.10 check box.
  7. From the Select Action list, select Override to Enabled, and click Done.
  8. When the process completes, on the same set of hosts from the previous steps, from the Select Action list, select Manage Packages.
  9. In the Package search field, enter one of the following options depending on your configuration:

    • If your deployment uses katello-agent and goferd, enter katello-agent.
    • If your deployment does not use katello-agent and goferd, enter katello-host-tools.
  10. From the Update list, you must select the via remote execution option. This is required because if you update the package using the Katello agent, the package update disrupts the communication between the client and Satellite or Capsule Server, which causes the update to fail. For more information, see Configuring and Setting Up Remote Jobs in the Managing Hosts guide.

Upgrade Satellite Clients Manually

  1. Log into the client system.
  2. Disable the repositories for the previous version of Satellite.

    # subscription-manager repos \
    --disable rhel-7-server-satellite-tools-6.9-rpms
  3. Enable the Satellite Tools 6.10 repository for this version of Satellite.

    # subscription-manager repos \
    --enable=rhel-7-server-satellite-tools-6.10-rpms
  4. Depending on your configuration, complete one of the following steps:

    • If your deployment uses katello-agent and goferd, enter the following command to install or upgrade katello-agent:

      # yum install katello-agent
    • If your deployment does not use katello-agent and goferd, enter the following command to install or upgrade katello-host-tools:

      # yum install katello-host-tools