Red Hat Training

A Red Hat training course is available for Red Hat Satellite

Chapter 5. Importing Custom Content

The previous chapter examined how to import Red Hat content into the Definitive Media Library (DML). This chapter focuses on custom content, which differs slightly from Red Hat content. We create our own product beforehand, customize it, and add our own repositories. In addition, you can add Puppet modules to a custom repository.

5.1. Using Custom Products in Satellite

In Section 4.2, “Using Products and Repositories in Satellite”, we explored the concept of a Product in Red Hat Satellite 6 and how it is used to group repositories together. Red Hat Satellite 6 also provides the ability to create custom Products so you can add multiple related repositories. Both Red Hat content and custom content in Red Hat Satellite 6 share some similarities:

  • The relationship between a product and its repositories is the same and the repositories still require synchronization.
  • Custom Products require a subscription for clients to access, similar to subscriptions to Red Hat Products. Red Hat Satellite 6 creates a new subscription for each custom Product you create.

In this chapter, we aim to create a product that contains two related repositories: an RPM repository containing RPM content and a Puppet repository for modules to configure the RPM content.

For more information about creating and packaging RPMs, see the RPM Packaging Guide in the Red Hat Enterprise Linux documentation.

5.2. Creating a Custom Product

In our scenario, ACME is aiming to develop a product called Exampleware, which is a web-based application that requires a PostgreSQL database. ACME might aim to use production-level Exampleware in the field using PostgreSQL from the Red Hat Enterprise Linux repositories but test Exampleware with newer versions of PostgreSQL. For this situation, let’s create a custom product for PostgreSQL so that we can synchronize the newer versions.

For Web UI Users

Navigate to Content > Products, click New Product and enter the following details:

  • Name - The plain text name for the product. Enter PostgreSQL.
  • Label - An internal ID for the product. Red Hat Satellite 6 automatically completes this field based on what you have entered for Name.
  • GPG Key - The GPG Key for the entire product. Leave this blank as we import a GPG for a specific version of PostgreSQL and attach it to the repository instead of the product.
  • Sync Plan - A synchronization plan for the product. We can attach this to the Example Plan we created in the previous chapter.
  • Description - A plain text description of the product. Enter Content from PostgreSQL repositories.

When you have entered this information, click Save.

For CLI Users

To create the product, enter the following command: 

# hammer product create \
--name "PostgreSQL" \
--sync-plan "Example Plan" \
--description "Content from PostgreSQL repositories" \
--organization "ACME"

This creates a new product where we can create our own custom repositories and synchronize their content.

5.3. Importing a Custom GPG Key

Before we create a custom product, we might need to create a custom GPG key. This is used to provide a certain level of security for RPM transactions with the PostgreSQL repository.

First download a copy of the version specific repository package to your client system. In our case, we download pgdg-redhat95: 

[user@client ~]$ wget http://yum.postgresql.org/9.5/redhat/rhel-7-x86_64/pgdg-redhat95-9.5-2.noarch.rpm

Extract the RPM file without installing it: 

[user@client ~]$ rpm2cpio pgdg-redhat95-9.5-2.noarch.rpm | cpio -idmv

The GPG key is located relative to the extraction at etc/pki/rpm-gpg/RPM-GPG-KEY-PGDG-95.

For Web UI Users

Navigate to Content > GPG keys. Click New Gpg Key. Provide the GPG key with a name (PostgreSQL 9.5) and select Upload GPG Key. Click Browse and select the GPG key you extracted. Click Save.

For CLI Users

Copy the GPG key to your Satellite Server: 

[user@client ~]$ scp ~/etc/pki/rpm-gpg/RPM-GPG-KEY-PGDG-95 root@satellite.example.com:~/.

Upload the GPG key to Satellite: 

[root@satellite ~]# hammer gpg create \
--key ~/RPM-GPG-KEY-PGDG-95 \
--name "PostgreSQL 9.5" \
--organization "ACME"

Now we have a GPG key we can associate with our repository.

5.4. Creating a Custom RPM Repository

Normally, a production-level server would use the version of PostgreSQL included with Red Hat Enterprise Linux for stability purposes. However, ACME’s developers might aim to test Exampleware with a more recent version of PostgreSQL. In this instance, they can create a custom repository for a newer version of PostgreSQL.

For Web UI Users

We follow on from creating our custom PostgreSQL product. After creating our custom product, the repositories screen appears. Click Create Repository, which displays a form for a new repository. Enter the following details:

  • Name - A plain text name for the repository. Enter PostgreSQL 9.5.
  • Label - An internal ID for the repository. Red Hat Satellite 6 automatically completes this field based on what you have entered for Name.
  • Type - The type of repository. You can select either a repository for RPM files (yum), Puppet modules (puppet), or Docker images (docker). For our scenario, select yum. A new set of fields appear.
  • URL - The URL of the external repository to use as a source. Enter http://yum.postgresql.org/9.5/redhat/rhel-7-x86_64/.
  • Download Policy - Determines the type of synchronization Satellite Server performs. Select Immediate. See Section 4.4, “Using Download Policies” for more information.
  • Mirror on Sync - Ensures the content that is no longer part of the upstream repository is removed during synchronization. Leave this checked, which is the default.
  • Checksum - The checksum for the repository. For this example, leave this as the Default, which defaults to SHA256. This is the checksum required for Red Hat Enterprise Linux 7. For Red Hat Enterprise Linux 5 or previous versions, select SHA1 for the checksum.
  • Publish via HTTP - Enables this repository for publication through HTTP. This option is automatically selected.
  • GPG Key - The GPG Key for this repository. Select the PostgreSQL 9.5 GPG key we created previously.

Click Save to save this repository entry.

The repository uses the synchronization plan to periodically keep it up to date. However, let’s perform an initial synchronization. Select the PostgreSQL 9.5 repository and click Sync Now. This starts a task to synchronize our repository with the external PostgreSQL repository.

Note

Monitor the progress of this synchronization on the Content > Sync Status page.

For CLI Users

Enter the following command to create the repository: 

# hammer repository create \
--name "PostgreSQL 9.5" \
--content-type "yum" \
--publish-via-http true \
--url http://yum.postgresql.org/9.5/redhat/rhel-7-x86_64/ \
--gpg-key "PostgreSQL 9.5" \
--product "PostgreSQL" \
--organization "ACME"

Then synchronize the repository: 

# hammer repository synchronize \
--name "PostgreSQL 9.5" \
--product "PostgreSQL" \
--organization "ACME"

Now we have a synchronized copy of PostgreSQL 9.5. Let’s take this a step further and include a Puppet module to configure a PostgreSQL server.

Note

The Products page in the Web UI also provides a Repo Discovery function that finds all repositories from a URL and allows you to select which ones to add to your custom Product. For example, you can use the Repo Discovery to search http://yum.postgresql.org/9.5/redhat/ and list all PostgreSQL 9.5 repositories for different Red Hat Enterprise Linux versions and architectures. This helps users save time importing multiple repositories from a single source.

Important

Red Hat does not support the upstream RPMs directly from the PostgreSQL site. These RPMs are used to demonstrate the synchronization process. For any issues with these RPMs, contact the developers for PostgreSQL.

5.5. Creating a Custom Puppet Repository

Custom products can also include repositories for Puppet modules. This provides a method to incorporate state configuration of hosts.

First, let’s create a repository for our PostgreSQL product.

For Web UI Users

Ensure you are on the Product page (Content > Products). Click on the PostgreSQL product, which displays our repository listing. Click Create Repository, which displays a form for a new repository. Enter the following details:

  • Name - A plain text name for the repository. Enter PostgreSQL Puppet Modules.
  • Label - An internal ID for the repository. Red Hat Satellite 6 automatically completes this field based on what you have entered for Name.
  • Type - The type of repository. Select puppet and a URL field appears.
  • URL - The URL of the external repository to use as a source. We manually import our Puppet module, but you can use a repository source to synchronize your own Puppet modules.

Click Save to save this repository entry.

For CLI Users

Enter the following command to create our Puppet module repository: 

# hammer repository create \
--name "PostgreSQL Puppet Modules" \
--content-type "puppet" \
--product "PostgreSQL" \
--organization "ACME"

Now we have a custom repository for our Puppet modules. Let’s add one now.

5.6. Managing Individual Puppet Modules

For this scenario, we manually import a Puppet module for configuring PostgreSQL.

Download this module from the Puppet Forge site (https://forge.puppetlabs.com/puppetlabs/postgresql). Open the page in your browser and click download latest tar.gz to save to your local file system.

For Web UI Users

Ensure you are on the repository listing page for the PostgreSQL product. Click on the PostgreSQL Puppet Modules repository, which displays the details page for that repository.

Navigate to the Upload Puppet Module section, click Browse, select the PostgreSQL Puppet Module we downloaded previously, and click Upload. After a few seconds, Satellite Server reports Content successfully uploaded.

Note

Click on the Manage Puppet Modules page to manage and remove Puppet modules from a product.

For CLI Users

Copy the Puppet module to your Satellite Server’s file system: 

[user@client ~]$ scp ~/puppet_module.tar.gz root@satellite.example.com:~/.

Import the Puppet module to the PostgreSQL Puppet Modules repository: 

[root@satellite ~]# hammer repository upload-content \
--path ~/puppet_module.tar.gz \
--name "PostgreSQL Puppet Modules" \
--product "PostgreSQL" \
--organization "ACME"

Now we have a custom repository that contains both RPM content and a Puppet module to install and configure a server using the RPM content.

Important

Red Hat does not support the modules from Puppet Forge. The PostgreSQL module is used to demonstrate the module management process. For any issues with these modules, contact the module developer.

5.7. Synchronizing Puppet Repositories

In addition to creating a repository of uploaded Puppet modules, Satellite Server can synchronize a complete Puppet module repository. In this example, Satellite Server synchronizes the entire Puppet Forge repository.

For Web UI Users

Navigate to Content > Products and click New Product. A form for a new Product appears. Enter the following details:

  • Name - The plain text name for the product. Enter Puppet Forge.
  • Label - An internal ID for the product. Red Hat Satellite 6 automatically completes this field based on what you have entered for Name.
  • GPG Key - The GPG Key for the entire product. Leave this blank.
  • Sync Plan - A synchronization plan for the product. We can attach this to the Example Plan we created in the previous chapter.
  • Description - A plain text description of the product. Enter All modules from Puppet Forge.

When you have entered this information, click Save.

After creating our custom product, the repositories screen appears. Click Create Repository, which displays a form for a new repository. Enter the following details:

  • Name - A plain text name for the repository. Enter Puppet Forge Modules.
  • Label - An internal ID for the repository. Red Hat Satellite 6 automatically completes this field based on what you have entered for Name.
  • Type - The type of repository. Select puppet and a URL field appears.
  • URL - The URL of the external repository to use as a source. Enter http://forge.puppetlabs.com/.

Click Save to save this repository entry.

Select the Puppet Forge Modules repository and click Sync Now. This imports all modules from Puppet Forge into Satellite Server.

For CLI Users

Create the product: 

# hammer product create \
--name "Puppet Forge" \
--sync-plan "Example Plan" \
--description "All modules from Puppet Forge" \
--organization "ACME"

Create the Puppet Forge repository: 

# hammer repository create \
--name "Puppet Forge Modules" \
--content-type "puppet" \
--product "Puppet Forge" \
--organization "ACME" \
--url http://forge.puppetlabs.com/

Synchronize the repository: 

# hammer repository synchronize \
--name "Puppet Forge Modules" \
--product "Puppet Forge" \
--organization "ACME"
Note

The Puppet Forge repository contains several thousand modules and can take a long time to synchronize.

Important

Red Hat does not support the modules from Puppet Forge. The modules are used to demonstrate the synchronization process. For any issues with these modules, contact the module developer.

5.8. Synchronizing Puppet Modules from a Git Repository

Red Hat Satellite 6 includes a utility called pulp-puppet-module-builder, which you can install on other systems from the pulp-puppet-tools RPM. This tool checks out a Git repository, builds all the modules, and publishes them in a structure that Satellite 6 can synchronize. One common method is to run the utility on Satellite Server itself, publish to a local directory, and synchronize against that directory. For example: 

# mkdir /modules
# chmod 755 /modules
# pulp-puppet-module-builder \
--output-dir=/modules \
--url=git@mygitserver.com:mymodules.git \
--branch=develop

This checks out the develop branch of the Git repository from git@mygitserver.com:mymodules.git and publishes it to /modules. Add this directory as the URL (file:///modules) for a new repository on Satellite Server. For example:

For Web UI Users

Open a custom product (in this case, we use MyProduct as an example) and click Create Repository. Enter the following details:

  • Name - A plain text name for the repository. Enter Modules from Git.
  • Label - An internal ID for the repository. Red Hat Satellite 6 automatically completes this field based on what you have entered for Name.
  • Type - The type of repository. Select puppet and a URL field appears.
  • URL - The URL of the external repository to use as a source. Enter file:///modules.

For CLI Users

Create the Puppet Forge repository: 

# hammer repository create \
--name "Modules from Git" \
--content-type "puppet" \
--product "MyProduct" \
--organization "ACME" \
--url file:///modules
Note

The same process also applies to publishing modules on a remote HTTP server. For example, if you use webserver.example.com as a standard web host to publish the Puppet modules, log into the host and enter the following commands: 

# mkdir /var/www/html/modules/
# chmod 755 /var/www/html/modules/
# pulp-puppet-module-builder \
--output-dir=/var/www/html/modules/ \
--url=git@mygitserver.com:mymodules.git \
--branch=develop

On Satellite Server, set the repository’s URL to http://webserver.example.com/modules/.

5.9. Creating a Custom File Type Repository in Red Hat Satellite

Custom products in Red Hat Satellite can also include repositories for custom file types. This provides a generic method to incorporate arbitrary files in a Product. Applications range from distributing SSH keys and source code files to larger files such as virtual machine images and ISO files.

You can upload files to the repository or synchronize it from an upstream Satellite Server. When you add files to a custom file type repository, you can use the normal Satellite management functions such as adding a specific version to a Content View to provide version control and making the repository of files available on various Capsule Servers. Clients must download the files over HTTP or HTTPS using curl -O.

A file type repository in Satellite Server can be created only in a custom product, but you can create an independent file type repository in a directory on the system where Satellite is installed, or on a remote HTTP server, and then synchronize the contents of that directory into Satellite. This method is useful when you have multiple files to add to a Satellite repository.

This example creates a Product My File Product, and then creates a file type repository My Files within that product. For more information about file type repositories outside of Satellite, see Appendix E, Creating a Remote File Type Repository.

For Web UI Users

  1. Create a Custom Product

    Navigate to Content > Products, click Create Product and enter the following details:

    • Name - The plain text name for the product. Enter My File Product.
    • Label - An internal ID for the product. Red Hat Satellite 6 automatically completes this field based on what you have entered for Name.
    • GPG Key - The GPG Key for the entire product. You can leave this blank.
    • Sync Plan - A synchronization plan for the product. For example, the Example Plan created in the previous chapter.
    • Description - A plain text description of the product. Enter My files.

  2. Create a File Type Repository

    Navigate to Content > Products. Click the name of a product, in this example My File Product. On the Repositories tab, click New Repository and enter the following details:

    • Name - A plain text name for the repository. Enter My Files.
    • Label - An internal ID for the repository. Red Hat Satellite 6 automatically completes this field based on what you have entered for Name.
    • Type - The content type of the repository. Select file.
    • Upstream URL - The URL of the upstream repository to use as a source. You can leave this blank if you plan to only upload files manually.
    • Verify SSL - Leave this option selected if you want to verify that the upstream repository’s SSL certificates are signed by a trusted CA. Alternatively, clear this check box.
    • Upstream Username - If required for authentication, enter the user name for the upstream repository. Clear this field if the repository does not require authentication.
    • Upstream Password - Enter the password for the upstream repository. Clear this field if the repository does not require authentication.
  3. Click Save to save this repository entry.

For CLI Users

  1. Create a Custom Product 

    # hammer product create \
--name "My File Product" \
--sync-plan "Example Plan" \
--description "My files" \
--organization "ACME"

    Optional Parameters

    gpg-key gpg_key_name

    Key name to search by

    gpg-key-id gpg_key_id

    GPG key numeric identifier

    sync-plan sync_plan_name

    Sync plan name to search by

    sync-plan-id sync_plan_id

    Sync plan numeric identifier

  2. Create a File Type Repository 

    # hammer repository create \
--name "My Files" \
--content-type "file" \
--product "My File Product" \
--organization "ACME"

    Optional Parameters

    checksum-type sha_version

    Repository checksum, currently 'sha1' & 'sha256' are supported

    download-policy policy_name

    Download policy for yum repos (either 'immediate', 'on_demand', or 'background').

    gpg-key gpg_key_name

    Key name to search by

    gpg-key-id gpg_key_id

    GPG key numeric identifier

    mirror-on-sync boolean

    Must this repo be mirrored from the source, and stale RPMs removed, when synced? Set to true or false, yes or no, 1 or 0.

    publish-via-http boolean

    Must this also be published using HTTP? Set to true or false, yes or no, 1 or 0.

    upstream-username repository_username

    Upstream repository user, if required for authentication

    upstream-password repository_password

    Password for the upstream repository user

    url source_repo_url

    URL of the Source repository

    verify-ssl-on-sync boolean

    Must Katello verify that the upstream URL’s SSL certificates are signed by a trusted CA? Set to true or false, yes or no, 1 or 0.

5.10. Uploading Files To a Custom File Type Repository in Red Hat Satellite

For Web UI Users

  1. Navigate to Content > Products.
  2. Select a custom product by name. For example, My File Product.
  3. Select a file type repository by name. For example, My Files.
  4. Click Browse to search and select the file you want to upload.
  5. Click Upload to upload the selected file to Satellite Server.
  6. Visit the URL where the repository is published to see the file.

For CLI Users

# hammer repository upload-content \
--product "My File Product" \
--name "My Files" \
--organization "ACME" \
--path example_file

The --path option can indicate a file, a directory of files, or a glob expression of files. Globs must be escaped by single or double quotes.

5.11. Downloading Files to a Host From a Custom File Type Repository in Red Hat Satellite

You can download files to a client over HTTPS using curl -O, and optionally over HTTP if the Publish via HTTP repository option is selected.

Prerequisites

For Web UI Users

  1. Navigate to Content > Products.
  2. Select a custom product by name. For example, My File Product.
  3. Select a file type repository by name. For example, My Files.
  4. Check to see if Publish via HTTP is enabled. If it is not, you need the certificates to use HTTPS.
  5. Copy the URL where the repository is published.

For CLI Users

  1. List the file type repositories. 

    # hammer repository list --content-type file
---|----------|-----------------|--------------|----
ID | NAME     | PRODUCT         | CONTENT TYPE | URL
---|----------|-----------------|--------------|----
7  | My Files | My File Product | file         |
---|----------|-----------------|--------------|----

  2. Display the repository information. 

    # hammer repository info --name "My Files" --product "My File Product" --organization-id 1
    • If HTTP is enabled, the output looks something like this: 
    Publish Via HTTP:   yes
Published At:       http://satellite.example.com/pulp/isos/uuid/
    • If HTTP is not enabled, the output looks something like this: 
    Publish Via HTTP:   no
Published At:       https://satellite.example.com/pulp/isos/uuid/

On the client, enter a command in the appropriate format for HTTP or HTTPS:

  • for HTTP: 

    # curl -O satellite.example.com/pulp/isos/uuid/my_file

  • for HTTPS: 

    # curl -O --cert ./Default\ Organization-key-cert.pem  --cacert katello-server-ca.crt satellite.example.com/pulp/isos/uuid/my_file

5.12. Creating a Custom File Type Repository in a Local Directory

You can create a custom file type repository, from a directory of files, external to Satellite Server using the pulp-manifest command. You can then synchronize the files into Satellite Server. When you add files to a file type repository, you can work with the files as with any other repository.

This procedure describes configuring a repository in a directory on the base system where Satellite is installed. To create a file type repository in a directory on a remote server, see Appendix E, Creating a Remote File Type Repository.

To Create a File Type Repository in a Local Directory:

  1. Ensure the Server and Satellite Tools repositories are enabled: 

    # subscription-manager repos --enable=rhel-7-server-rpms \
--enable=rhel-7-server-satellite-tools-6.3-rpms

  2. Install the Pulp Manifest package: 

    # yum install python-pulp-manifest

  3. Create a directory that you want to use as the file type repository in the HTTP server’s public folder: 

    # mkdir my_file_repo

  4. Add files to the directory or create a test file: 

    # touch my_file_repo/test.txt

  5. Enter the Pulp Manifest command to create the manifest: 

    # pulp-manifest my_file_repo

  6. Verify the manifest was created: 

    # ls my_file_repo
PULP_MANIFEST  test.txt

To Import Files From a File Type Repo in a Local Directory

  1. Ensure a custom product exists in Satellite Server. This example uses My File Product.

  2. Create a File Type Repository:

    1. In the Satellite web UI, navigate to Content > Products.
    2. Select the name of a product, in this example My File Product.

    3. On the Repositories tab, select New Repository and enter the following details:

      • Name - A plain text name for the repository. Enter My Files.
      • Label - An internal ID for the repository. Red Hat Satellite 6 automatically completes this field based on what you enter for Name.
      • Type - The content type of the repository. Select file.
      • Upstream URL - The path to the local directory with the repository to use as the source, in the form file:///my_file_repo.
      • Verify SSL - Ensure this field is clear.
      • Upstream Username - Ensure this field is clear.
      • Upstream Password - Ensure this field is clear.
  3. Select Save to save this repository entry.

  4. Update the file type repository:

    1. Navigate to Content > Products.
    2. Select the name of a product, in this example My File Product.
    3. Select the name of the repository you want to update, in this example My Files.
    4. From the Select Action menu, select Sync Now.
    5. Visit the URL where the repository is published to see the files.

5.13. Chapter Summary

This chapter demonstrated how to create custom repositories for non-Red Hat content. This includes RPM files, Puppet modules, and custom file types.

At this point we have all our base content imported into our Satellite Server’s DML. This means we can start using it in our application life cycle.

The next chapter looks at developing an application life cycle to match the development and production process of ACME.