Chapter 4. Importing Red Hat Content

Our Satellite Server now has the necessary subscription information imported. We are ready to add content to our system. For this chapter, we explore the concept of a Definitive Media Library (DML) and how to synchronize content to create our DML.

4.1. Creating a Definitive Media Library

A DML is a repository that stores and protects the definitive, authorized versions of software and configurations. In other words, the DML acts as a master version of any content imported into the Satellite. This includes Red Hat content, such as RPM files, kickstart trees, and ISO images. As mentioned in Section 1.1, “Overview of Red Hat Satellite 6 Content Management”, Red Hat Satellite 6 stores and manages content in a DML.

4.2. Using Products and Repositories in Satellite

In Satellite, we use the concept of a Product as an organizational unit to group multiple repositories together. Such repository collections are analogous to the concept of real life products. For example, if we view Red Hat Enterprise Linux Server as a Product in Satellite, the repositories for that product might consist of different versions (6.0, 6.1, 7.0), different architectures (i386, x86_64, s390x, arm), and different add-ons (Optional repositories, Supplementary repositories, Virt V2V tools). This unifies all related repositories within the DML. Using Products ensures repositories that depend on each other are synchronized together. For Red Hat repositories, products are created automatically after enabling the repository.

In this chapter, our aim is to start creating our DML using Red Hat content. To do this, we synchronize our DML with Red Hat’s Products and their repositories.

4.3. Synchronizing Content

Once we choose which repositories form our DML, the Satellite Server synchronizes its own repositories with the repositories on the Red Hat CDN. This ensures that the Satellite Server retains an exact copy of Red Hat’s repositories as a part of its DML. The Satellite Server fetches this repository information and stores it on the Satellite Server’s file system. After an initial synchronization, you can create a synchronization plan that checks to make sure the repositories in the DML is up to date with the CDN’s repositories.

It is possible to perform an initial synchronization using ISO images. See Appendix C, Importing Content ISOs into a Connected Satellite for more information on using Content ISOs. For locations with bandwidth limitations, using an On Demand or Background download policy as described below might be quicker than downloading and importing Content ISOs.

4.4. Using Download Policies

The Satellite Server provides multiple download policies for synchronizing RPM content. For example, you might aim to save time and only download the content metadata while deferring the actual content download for later. The Satellite Server offers the following policies:

  • Immediate - Satellite Server downloads all metadata and packages during synchronization.
  • On Demand - Satellite Server only downloads the metadata during synchronization. The server only fetches and stores packages on the file system when clients request them.
  • Background - Satellite Server creates a background task to download all packages after the initial synchronization.

The latter two policies act as a Lazy Synchronization feature because they save time synchronizing content. The lazy sync feature must only be used for yum repositories.

All repositories stored in the Satellite Server use a download policy and you can change the download policy to suit your needs.

4.5. Selecting Red Hat Repositories to Synchronize

The first step in selecting the repositories to synchronize is to identify the product that contains the repository, then enable that repository based on release version and base architecture.

Important

If using a Disconnected Satellite, you need to import the Content ISOs for Red Hat Satellite and change the CDN URL on the Satellite Server before synchronizing content. For more information, see Appendix B, Importing Content ISOs into a Disconnected Satellite.

For Web UI Users

Navigate to Content > Red Hat Repositories. This displays a set of tabs for different content types. Each tab contains a list of products. The RPMs should be the default tab upon loading this page. This tab contains list of all subscribed products that provide RPM content.

The relationship between products and specific repositories is connected through a cascading hierarchy. Select a product and this opens a list of repository sets for that product. Select a repository set and this opens a list of repositories that you can enable. For our scenario, select Red Hat Enterprise Linux Server, then Red Hat Enterprise Linux 7 Server (RPMs), then enable Red Hat Enterprise Linux 7 Server RPMs x86_64 7Server. This enables the latest RPM files for Red Hat Enterprise Linux 7.

Note

The difference between associating Red Hat Enterprise Linux Operating System with either 7 Server repositories or 7.X repositories is that 7 Server repositories contain all the latest updates while Red Hat Enterprise Linux 7.X repositories stop getting updates after the next minor version release. Note that Kickstart repositories only have minor versions.

For CLI Users

The relationship between products and repositories is the same. Search for your product with the following command:

# hammer product list --organization "ACME"

List the repository set for the product:

# hammer repository-set list \
--product "Red Hat Enterprise Linux Server" \
--organization "ACME"

This displays the repositories in the product’s repository set, including their name and ID number. Enable the repository using either the name or ID number. Also include the release version (7Server) and base architecture (x86_64). For example:

# hammer repository-set enable \
--name "Red Hat Enterprise Linux 7 Server (RPMs)" \
--releasever "7Server" \
--basearch "x86_64" \
--product "Red Hat Enterprise Linux Server" \
--organization "ACME"

For this scenario, use either the Web UI or the CLI to enable the following repositories for ACME:

RepositoryTypeDescription

Red Hat Enterprise Linux 7 Server RPMs x86_64 7Server

RPM

Repositories for the latest version of Red Hat Enterprise Linux 7. Use the 7 Server repository instead of the 7.2 repository so that you receive continuous package updates.

Red Hat Satellite Tools 6.2 for RHEL 7 Server RPMs x86_64

RPM

The Satellite Tools repository, which contains system management agents and tools for client systems. After provisioning a new system, Satellite installs tools such as katello-agent and Puppet to the client. Use the 7 Server repository instead of the 7.2 repository so that you receive continuous package updates.

Red Hat Enterprise Linux 7.2 Kickstart x86_64 7Server

Kickstart

The kickstart tree for Red Hat Enterprise Linux 7.2. Use this as installation media for provisioning new systems over PXE.

These repositories will provide some initial content for our scenario’s DML. You can select more repositories based upon your needs.

Note

For this scenario, all repositories use x86_64 as the base architecture.

4.6. Synchronizing Red Hat Repositories

We have enabled specific repositories to form our initial DML. Now we synchronize the repositories with the Red Hat CDN’s repositories.

For Web UI Users

Navigate to Content > Products and select Red Hat Enterprise Linux Server. This displays all enabled repositories in our product. Select all repositories and click Sync Now. You can also view the progress of the synchronization in the Web UI. Navigate to Content > Sync Status and expanding the Product/Repository tree (or click Expand All).

For CLI Users

Synchronize the enabled repositories in the Red Hat Enterprise Linux Server product:

# hammer product synchronize \
--name "Red Hat Enterprise Linux Server" \
--organization "ACME"

You can also synchronize each repository individually. List all repositories in the product, then synchronize using the ID number for the corresponding repositories. For example:

# hammer repository list \
--product "Red Hat Enterprise Linux Server" \
--organization "ACME"
# hammer repository synchronize \
--name "Red Hat Enterprise Linux 7 Server RPMs x86_64 7Server" \
--product "Red Hat Enterprise Linux Server" \
--organization "ACME"

The synchronization duration depends on the size of each repository and the speed of your network connection. The following table provides estimates of how long it would take to synchronize content, depending on the available Internet bandwidth:

 Single Package (10Mb)Minor Release (750Mb)Major Release (6Gb)

256 Kbps

5 Mins 27 Secs

6 Hrs 49 Mins 36 Secs

2 Days 7 Hrs 55 Mins

512 Kbps

2 Mins 43.84 Secs

3 Hrs 24 Mins 48 Secs

1 Day 3 Hrs 57 Mins

T1 (1.5 Mbps)

54.33 Secs

1 Hr 7 Mins 54.78 Secs

9 Hrs 16 Mins 20.57 Secs

10 Mbps

8.39 Secs

10 Mins 29.15 Secs

1 Hr 25 Mins 53.96 Secs

100 Mbps

0.84 Secs

1 Min 2.91 Secs

8 Mins 35.4 Secs

1000 Mbps

0.08 Secs

6.29 Secs

51.54 Secs

A manual synchronization is often required for the initial content import into the DML. However, it is recommended create a synchronization plan to ensure that our DML is kept up to date on a regular basis.

Note

Once the synchronization completes, the kickstart repository appears in Red Hat Satellite 6’s list of installation media and is assigned to your organization. To view this, navigate to Hosts > Installation Media.

Note

You can change the download policy for a Red Hat repository. Select a repository in the Red Hat Enterprise Linux Server product and scroll to the Download Policy field. If using the CLI, use the hammer repository update command with the --download-policy option.

4.6.1. Limiting Synchronization Speed

You can control the speed of synchronization to avoid exhaustion of available bandwidth and prevent other performance issues. This is done by configuring PULP_CONCURRENCY and max_speed parameters. Note that these settings are overwritten on an upgrade. It is recommended to backup changed files prior to an upgrade to be able to restore the configuration.

  1. To control the number of synchronization jobs that run in parallel, configure the PULP_CONCURRENCY parameter in the /etc/default/pulp_workers file. For example, to set the number of jobs that run in parallel to 1, change PULP_CONCURRENCY to 1:

    PULP_CONCURRENCY=1

    Default value for the PULP_CONCURRENCY parameter is 2.

  2. To set the maximum network speed for synchronizing in bytes per second, configure the max_speed parameter. This parameter must be configured separately for each importer in the /etc/pulp/server/plugins.conf.d/ directory.

    1. For example, to set the maximum speed for synchronizing RPM content to 10 bytes per second, set the "max_speed" parameter in the /etc/pulp/server/plugins.conf.d/yum_importer.json file to 10:

      # cat /etc/pulp/server/plugins.conf.d/yum_importer.json
      {
          "proxy_host": null,
          "proxy_port": null,
          "proxy_username": null,
          "proxy_password": null,
          "max_speed": 10
      }
    2. Verify the syntax of the file after editing:

      # json_verify < /etc/pulp/server/plugins.conf.d/yum_importer.json
      JSON is valid
  3. Restart Satellite services to apply the changes:

    # katello-service restart

4.7. Creating a Synchronization Plan

A synchronization plan checks and updates the content in your DML at a regularly scheduled date and time. Red Hat Satellite 6 provides users with the ability to create a synchronization plan and assign products to it.

For Web UI Users

Navigate to Content > Sync Plans and click New Sync Plan. The UI provides a set of fields where you can input details about your synchronization plan:

  • Name - A plain text name for the plan. Enter Example Plan.
  • Description - A plain text description of the plan. Enter Example Plan for ACME's repositories.
  • Interval - Defines when to run the synchronization. Select daily.
  • Start Date and Start Time - Defines when to run the synchronization. We already completed a synchronization today, so set the synchronization for tomorrow at 1:00 (1AM).

Click Save to create a plan. The plan details page displays along with two tabs for Details and Products.

Now add your products. Click the Products tab, then click Add. Select the Red Hat Enterprise Linux Server product and click Add Selected.

For CLI Users

Create the synchronization plan with the following command:

# hammer sync-plan create \
--name "Red Hat Products 2" \
--description "Example Plan for ACME's Red Hat Products" \
--interval daily \
--sync-date "2016-02-01 01:00:00" \
--enabled true \
--organization "ACME"

Then assign the Red Hat Enterprise Linux Server product to it:

# hammer product set-sync-plan \
--name "Red Hat Enterprise Linux Server" \
--sync-plan "Red Hat Products" \
--organization "ACME"

Now the Satellite Server checks its DML content against the Red Hat CDN on a daily basis and keeps its Red Hat repositories up to date.

4.8. Chapter Summary

This chapter showed how to import Red Hat content into ACME’s Satellite Server and keep it up to date through synchronization plans.

The next chapter explores a importing custom content into the Satellite Server’s DML. This process is a little similar to importing Red Hat content except we create and manage custom products.