Chapter 5. Importing Red Hat Content

This section describes how to use products and repositories in Satellite, and to create synchronization plans to ensure your Satellite content remains up to date with the content on the Red Hat Content delivery network (CDN).

5.1. Products in Red Hat Satellite

In Satellite, a Product is an organizational unit to group multiple repositories together. For example, Red Hat Enterprise Linux Server is a Product in Satellite, the repositories for that product consist of different versions, different architectures, and different add-ons. Using Products ensures repositories that depend on each other are synchronized together. For Red Hat repositories, products are created automatically after enabling the repository.

5.2. Content Synchronization Overview

Satellite Server synchronizes its own repositories with the repositories on the Red Hat CDN. This ensures that Satellite Server retains an exact copy of Red Hat’s repositories. Satellite Server fetches this repository information and stores it on Satellite Server’s file system. After an initial synchronization, you can create a synchronization plan that checks to ensure the repositories are up to date with the CDN’s repositories.

You can perform an initial synchronization using ISO images. For more information about using content ISOs, see Appendix C, Importing Content ISOs into a Connected Satellite.

5.3. Disabling the Global HTTP Proxy

If you want to synchronize local repositories without using the configured proxy, you can ignore the global proxy that is set when you run the satellite-installer tool.

Procedure

To ignore the global HTTP proxy, complete the following steps:

  1. In the Satellite web UI, navigate to Content > Products and select the product that you want to change.
  2. Click the Repositories tab, and select the repository that you want to change.
  3. In the Sync Settings area, navigate to Ignore Global HTTP Capsule. Click the edit icon, select the check box, and then click Save.

For CLI users

# hammer repository update --id Repository_ID --ignore-global-proxy yes

5.4. Download Policies Overview

Red Hat Satellite provides multiple download policies for synchronizing RPM content. For example, you might want to download only the content metadata while deferring the actual content download for later.

Satellite Server has the following policies:

Immediate
Satellite Server downloads all metadata and packages during synchronization.
On Demand
Satellite Server downloads only the metadata during synchronization. Satellite Server only fetches and stores packages on the file system when Capsules or directly connected clients request them. This setting has no effect if you set a corresponding repository on a Capsule to Immediate because Satellite Server is forced to download all the packages.
Background
Satellite Server creates a background task to download all packages after the initial synchronization.

The On Demand and Background policies act as a Lazy Synchronization feature because they save time synchronizing content. The lazy synchronization feature must be used only for yum repositories. You can add the packages to Content Views and promote to life cycle environments as normal.

Capsule Server offers the following policies:

Immediate
Capsule Server downloads all metadata and packages during synchronization. Do not use this setting if the corresponding repository on Satellite Server is set to On Demand as Satellite Server is forced to download all the packages.
On Demand
Capsule Server only downloads the metadata during synchronization. Capsule Server fetches and stores packages only on the file system when directly connected clients request them. When you use an On Demand download policy, content is downloaded from Satellite Server if it is not available on Capsule Server.
Background
Capsule Server creates a background task to download all packages after the initial synchronization.
Inherit
Capsule Server inherits the download policy for the repository from the corresponding repository on Satellite Server.

These policies are not available if a Capsule was installed or updated with --enable-foreman-proxy-plugin-pulp set to false.

5.5. Changing the Default Download Policy

You can set the default download policy that Satellite applies to new repositories you create in all organizations. Changing the default value does not change existing settings.

Procedure

To change the default download policy for repositories, complete the following steps:

  1. In the Satellite web UI, navigate to Administer > Settings.
  2. Click the Content tab and locate the Default Repository download policy.
  3. In the Value field, click the edit icon.
  4. Select the required download policy, and then click Save.

For CLI Users

To change the default download policy to one of immediate, on_demand, background, enter the following command:

# hammer settings set \
--name default_download_policy \
--value immediate

5.6. Changing the Download Policy for a Repository

You can also set the download policy for a repository.

Procedure

To set the download policy for a repository, complete the following steps:

  1. In the web UI, navigate to Content > Products, and click the required product name.
  2. On the Repositories tab, click the required repository name, locate the Download Policy field, and click the edit icon.
  3. From the list, select the required download policy and then click Save.

For CLI Users

  1. List the repositories for an organization:

    # hammer repository list \
    --organization-label organization-label
  2. Change the download policy for a repository to one of immediate, on_demand, background:

    # hammer repository update \
    --organization-label organization-label  \
    --product "Red Hat Enterprise Linux Server" \
    --name "Red Hat Enterprise Linux 7 Server Kickstart x86_64 7.5"  \
    --download-policy immediate

5.7. Enabling Red Hat Repositories

To select the repositories to synchronize, you must first identify the product that contains the repository, and then enable that repository based on the relevant release version and base architecture. For Red Hat Enterprise Linux 8, you must enable both AppStream and BaseOS repositories.

Disconnected Satellite

If you use Disconnected Satellite Server, you must import the Content ISOs for Red Hat Satellite and change the CDN URL on Satellite Server before synchronizing content. For more information, see Appendix B, Importing Content ISO Images into a Disconnected Satellite.

Repository Versioning

The difference between associating Red Hat Enterprise Linux operating system with either 7 Server repositories or 7.X repositories is that 7 Server repositories contain all the latest updates while Red Hat Enterprise Linux 7.X repositories stop getting updates after the next minor version release. Note that Kickstart repositories only have minor versions.

For Red Hat Enterprise Linux 8 Clients

To provision Red Hat Enterprise Linux 8 clients, you require the Red Hat Enterprise Linux 8 for x86_64 - AppStream (RPMS) and Red Hat Enterprise Linux 8 for x86_64 - BaseOS (RPMs) repositories.

For Red Hat Enterprise Linux 7 Clients

To provision Red Hat Enterprise Linux 8 clients, you require the Red Hat Enterprise Linux 7 Server (RPMs) repository.

Procedure

  1. In the Satellite web UI, navigate to Content > Red Hat Repositories.
  2. To find repositories, either enter the repository name, or toggle the Recommended Repositories button to the on position to view a list of repositories that you require.
  3. In the Available Repositories pane, click a repository to expand the repository set.
  4. Click the Enable icon next to the base architecture and release version that you want.

For CLI Users

  1. To search for your product, enter the following command:

    # hammer product list --organization "My_Organization"
  2. List the repository set for the product:

    # hammer repository-set list \
    --product "Red Hat Enterprise Linux Server" \
    --organization "My_Organization"
  3. Enable the repository using either the name or ID number. Include the release version, for example,7Server and base architecture, for example, x86_64. For example:

    # hammer repository-set enable \
    --name "Red Hat Enterprise Linux 7 Server (RPMs)" \
    --releasever "7Server" \
    --basearch "x86_64" \
    --product "Red Hat Enterprise Linux Server" \
    --organization "My_Organization"

5.8. Synchronizing Red Hat Repositories

Synchronize the repositories with the Red Hat CDN’s repositories.

For Web UI Users

  1. In the Satellite web UI, navigate to Content > Products and select the product that contains the repositories that you want to synchronize.
  2. Select the repositories that you want to synchronize and click Sync Now.

To view the progress of the synchronization in the web UI, navigate to Content > Sync Status and expand the corresponding product or repository tree.

For CLI Users

Synchronize the enabled repositories in the Red Hat Enterprise Linux Server product:

# hammer product synchronize \
--name "Red Hat Enterprise Linux Server" \
--organization "My_Organization"

You can also synchronize each repository individually. List all repositories in the product, then synchronize using the ID number for the corresponding repositories. For example:

# hammer repository list \
--product "Red Hat Enterprise Linux Server" \
--organization "My_Organization"
# hammer repository synchronize \
--name "Red Hat Enterprise Linux 7 Server RPMs x86_64 7Server" \
--product "Red Hat Enterprise Linux Server" \
--organization "My_Organization"

The synchronization duration depends on the size of each repository and the speed of your network connection. The following table provides estimates of how long it would take to synchronize content, depending on the available Internet bandwidth:

 Single Package (10Mb)Minor Release (750Mb)Major Release (6Gb)

256 Kbps

5 Mins 27 Secs

6 Hrs 49 Mins 36 Secs

2 Days 7 Hrs 55 Mins

512 Kbps

2 Mins 43.84 Secs

3 Hrs 24 Mins 48 Secs

1 Day 3 Hrs 57 Mins

T1 (1.5 Mbps)

54.33 Secs

1 Hr 7 Mins 54.78 Secs

9 Hrs 16 Mins 20.57 Secs

10 Mbps

8.39 Secs

10 Mins 29.15 Secs

1 Hr 25 Mins 53.96 Secs

100 Mbps

0.84 Secs

1 Min 2.91 Secs

8 Mins 35.4 Secs

1000 Mbps

0.08 Secs

6.29 Secs

51.54 Secs

Create a synchronization plan to ensure updates on a regular basis.

5.9. Synchronizing All Repositories in an Organization

Use this procedure to synchronize all repositories within an organization.

Procedure

To synchronize all repositories within an organization, run the following Bash script on your Satellite Server:

ORG="Your_Organization"

for i in $(hammer --no-headers --csv repository list --organization $ORG | awk -F, {'print $1'})
do
  hammer repository synchronize --id ${i} --organization $ORG --async
done

5.10. Recovering a Repository

In the case of repository corruption, you can recover it by using an advanced synchronization, which has three options:

Optimized Sync
Synchronizes the repository bypassing RPMs that have no detected differences from the upstream RPMs.
Complete Sync
Synchronizes all RPMs regardless of detected changes. Use this option if specific RPMs could not be downloaded to the local repository even though they exist in the upstream repository.
Validate Content Sync

Synchronizes all RPMs and then verifies the checksum of all RPMs locally. If the checksum of an RPM differs from the upstream, it re-downloads the RPM. This option is relevant only for yum repositories. Use this option if you have one of the following errors:

  • Specific RPMs cause a 404 error while synchronizing with yum.
  • Package does not match intended download error, which means that specific RPMs are corrupted.

Procedure

To synchronize a specific repository with an advanced option, complete the following steps:

  1. In the Satellite web UI, navigate to Content > Products.
  2. Select the product containing the corrupted repository.
  3. Select the name of a repository you want to synchronize.
  4. From the Select Action menu, select Advanced Sync.
  5. Select the option and click Sync.

For CLI users

  1. Obtain a list of repository IDs:

    # hammer repository list --organization "My_Organization"
  2. Synchronize a corrupted repository using the necessary option:

    • For the optimized synchronization:

      # hammer repository synchronize --incremental true --id 1
    • For the complete synchronization:

      # hammer repository synchronize --skip-metadata-check true --id 1
    • For the validate content synchronization:

      # hammer repository synchronize --validate-contents true --id 1

5.11. Limiting Synchronization Speed

You can control the speed of synchronization to avoid exhaustion of available bandwidth and to prevent other performance issues. This is done by configuring PULP_CONCURRENCY and max_speed parameters. Note that these settings are overwritten on an upgrade. Back up any changed files prior to an upgrade to be able to restore the configuration.

  1. To control the number of synchronization jobs that run in parallel, configure the PULP_CONCURRENCY parameter in the /etc/default/pulp_workers file. For example, to set the number of jobs that run in parallel to 1, change PULP_CONCURRENCY to 1:

    PULP_CONCURRENCY=1

    By default, on a system with less than 8 CPUs, PULP_CONCURRENCY is set to the number of CPUs. On a system with more than 8 CPUs, it is set to 8.

  2. To set the maximum network speed for synchronizing in bytes per second, configure the max_speed parameter. This parameter must be configured separately for each importer in the /etc/pulp/server/plugins.conf.d/ directory. For example, to set the maximum speed for synchronizing RPM content to 10 bytes per second, set the "max_speed" parameter in the /etc/pulp/server/plugins.conf.d/yum_importer.json file to 10:

    # cat /etc/pulp/server/plugins.conf.d/yum_importer.json
    {
        "proxy_host": null,
        "proxy_port": null,
        "proxy_username": null,
        "proxy_password": null,
        "max_speed": 10
    }
  3. Verify the syntax of the file after editing:

    # json_verify < /etc/pulp/server/plugins.conf.d/yum_importer.json
    JSON is valid
  4. Restart the satellite-maintain services to apply the changes:

    # satellite-maintain service restart

5.12. Creating a Synchronization Plan

A synchronization plan checks and updates the content at a scheduled date and time. In Red Hat Satellite 6, you can create a synchronization plan and assign products to the plan.

Procedure

To create a synchronization plan, complete the following steps:

  1. In the Satellite web UI, navigate to Content > Sync Plans and click New Sync Plan.
  2. In the Name field, enter a name for the plan.
  3. In the Description field, enter a description of the plan.
  4. From the Interval list, select the interval at which you want the plan to run.
  5. From the Start Date and Start Time lists, select when to start running the synchronization plan.
  6. Click Save.
  7. Click the Products tab, then click Add. Select the Red Hat Enterprise Linux Server product and click Add Selected.

For CLI Users

  1. To create the synchronization plan, enter the following command:

    # hammer sync-plan create \
    --name "Red Hat Products 2" \
    --description "Example Plan for Red Hat Products" \
    --interval daily \
    --sync-date "2016-02-01 01:00:00" \
    --enabled true \
    --organization "My_Organization"
  2. Assign the Red Hat Enterprise Linux Server product to it:

    # hammer product set-sync-plan \
    --name "Red Hat Enterprise Linux Server" \
    --sync-plan "Red Hat Products" \
    --organization "My_Organization"