Composing, installing, and managing RHEL for Edge images

Red Hat Enterprise Linux 8

Creating, deploying, and managing Edge systems with Red Hat Enterprise Linux 8

Red Hat Customer Content Services

Abstract

Use the image builder tool to compose customized RHEL (rpm-ostree) images optimized for Edge. Then, remotely install, and securely manage and scale deployments of the images on Edge servers.

Making open source more inclusive

Red Hat is committed to replacing problematic language in our code, documentation, and web properties. We are beginning with these four terms: master, slave, blacklist, and whitelist. Because of the enormity of this endeavor, these changes will be implemented gradually over several upcoming releases. For more details, see our CTO Chris Wright’s message.

Providing feedback on Red Hat documentation

We appreciate your feedback on our documentation. Let us know how we can improve it.

Submitting comments on specific passages

  1. View the documentation in the Multi-page HTML format and ensure that you see the Feedback button in the upper right corner after the page fully loads.
  2. Use your cursor to highlight the part of the text that you want to comment on.
  3. Click the Add Feedback button that appears near the highlighted text.
  4. Add your feedback and click Submit.

Submitting feedback through Jira (account required)

  1. Log in to the Jira website.
  2. Click Create in the top navigation bar
  3. Enter a descriptive title in the Summary field.
  4. Enter your suggestion for improvement in the Description field. Include links to the relevant parts of the documentation.
  5. Click Create at the bottom of the dialogue.

Chapter 1. Introducing RHEL for Edge images

A RHEL for Edge image is an rpm-ostree image that includes system packages to remotely install RHEL on Edge servers.

The system packages include:

  • Base OS package
  • Podman as the container engine
  • Additional RPM content

Differently from RHEL images, RHEL for Edge is an immutable operating system, that is, it contains a read-only root directory with the following characteristics:

  • The packages are isolated from root directory
  • Package installs create layers that make it easy to rollback to previous versions
  • Efficient updates to disconnected environments
  • Supports multiple operating system branches and repositories
  • Has a hybrid rpm-ostree package system

You can deploy a RHEL for Edge image on Bare Metal, Appliance, and Edge servers.

You can compose customized RHEL for Edge images using the image builder tool. You can also create RHEL for Edge images by accessing the edge management application in the Red Hat Hybrid Cloud Console platform and configure automated management.

The edge management application simplifies the way you can provision and register your images. To learn more about the edge management, see the Create RHEL for Edge images and configure automated management documentation.

Warning

Using RHEL for Edge customized images that were created using the image builder on-prem version artifacts is not supported in the edge management application. See Edge management supportability.

With a RHEL for Edge image, you can achieve the following:

Key features

1.1. RHEL for Edge—supported architecture

Currently, you can deploy RHEL for Edge images on AMD and Intel 64-bit systems.

Note

Currently, RHEL for Edge does not support ARM systems.

1.2. How to compose and deploy a RHEL for Edge image

Composing and deploying a RHEL for Edge image involves two phases:

  1. Composing a RHEL rpm-ostree image using the image builder tool. You can access image builder through a command-line interface in the composer-cli tool, or use a graphical user interface in the RHEL web console.
  2. Deploying the image using RHEL installer.

While composing a RHEL for Edge image, you can select any of the following image types. Composing the different RHEL for Edge images might or might not require network access. See the table:

Table 1.1. RHEL for Edge images type

Image typeDescriptionSuitable for network-based deploymentsSuitable for non-network-based deployments

RHEL for Edge Commit (.tar)

Commit image is not directly bootable, even though it contains a full operating system. To boot the Commit image type, you must deploy it.

Yes

No

RHEL for Edge Container (.tar)

The Container creates an OSTree commit and embeds it into an OCI container with a web server. When the Container is started, the web server serves the commit as an OSTree repository.

No

Yes

RHEL for Edge Installer (.iso)

The RHEL for Edge Installer image type pulls the commit from the running container and creates an installable boot ISO with a Kickstart file configured to use the embedded OSTree commit.

No

Yes

RHEL for Edge Raw image (.raw.xz)

The compressed raw images consist of a file that contains a partition layout with an existing deployed OSTree commit in it. You can flash the RHEL Raw Images on a hard disk or boot on a virtual machine.

Yes

Yes

RHEL for Edge Simplified Installer (.iso)

The RHEL for Edge Simplified Installer image uses the Ignition tool to inject the user configuration into the images at an early stage of the boot process.

Yes

Yes

The image types vary in terms of their contents, and are therefore suitable for different types of deployment environments.

1.3. Non-network-based deployments

Use image builder to create flexible RHEL rpm-ostree images to suit your requirements, and then use Anaconda to deploy them in your environment.

You can access image builder through a command-line interface in the composer-cli tool, or use a graphical user interface in the RHEL web console.

Composing and deploying a RHEL for Edge image in non-network-based deployments involves the following high-level steps:

  1. Install and register a RHEL system
  2. Install image builder
  3. Using image builder, create a blueprint with customizations for RHEL for Edge Container image
  4. Import the RHEL for Edge blueprint in image builder
  5. Create a RHEL for Edge image embed in an OCI container with a webserver ready to deploy the commit as an OSTree repository
  6. Download the RHEL for Edge Container image file
  7. Deploy the container serving a repository with the RHEL for Edge Container commit
  8. Using image builder, create another blueprint for RHEL for Edge Installer image
  9. Create a RHEL for Edge Installer image configured to pull the commit from the running container embedded with RHEL for Edge Container image
  10. Download the RHEL for Edge Installer image
  11. Run the installation

The following diagram represents the RHEL for Edge image non-network deployment workflow:

Figure 1.1. Deploying RHEL for Edge in non-network environment

RHEL for Edge non-network deployment workflow

1.4. Network-based deployments

Use image builder to create flexible RHEL rpm-ostree images to suit your requirements, and then use Anaconda to deploy them in your environment. image builder automatically identifies the details of your deployment setup and generates the image output as an edge-commit as a .tar file.

You can access image builder through a command-line interface in the composer-cli tool, or use a graphical user interface in the RHEL web console.

You can compose and deploy the RHEL for Edge image by performing the following high-level steps:

  1. Install and register a RHEL system
  2. Install image builder
  3. Using image builder, create a blueprint for RHEL for Edge image
  4. Import the RHEL for Edge blueprint in image builder
  5. Create a RHEL for Edge Commit (.tar) image
  6. Download the RHEL for Edge image file
  7. Set up a web server
  8. Create a new blueprint for a RHEL for Edge Installer (.iso)
  9. Create the RHEL for Edge Installer (.iso) image, pointing at the OSTree content from the RHEL for Edge Commit (.tar) artifact
  10. Download the RHEL for Edge installer ISO image you created
  11. Boot the edge device using the RHEL for Edge Installer ISO image

The following diagram represents the RHEL for Edge network image deployment workflow:

Figure 1.2. Deploying RHEL for Edge in network-base environment

RHEL for Edge network deployment workflow

1.5. Difference between RHEL RPM images and RHEL for Edge images

You can create RHEL system images in traditional package-based RPM format and also as RHEL for Edge (rpm-ostree) images.

You can use the traditional package-based RPMs to deploy RHEL on traditional data centers. However, with RHEL for Edge images you can deploy RHEL on servers other than traditional data centers. These servers include systems where processing of large amounts of data is done closest to the source where data is generated—Edge servers.

The RHEL for Edge (rpm-ostree) images are not a package manager. They only support complete bootable file system trees, not individual files. These images do not have information regarding the individual files such as how these files were generated or anything related to their origin.

The rpm-ostree images need a separate mechanism, the package manager, to install additional applications in the /var directory. With that, the rpm-ostree image keeps the operating system unchanged, while maintaining the state of the /var and /etc directories. The atomic updates enable rollbacks and background staging of updates.

Refer to the following table to know how RHEL for Edge images differ from the package-based RHEL RPM images.

Table 1.2. Difference between RHEL RPM images and RHEL for Edge images

Key attributes

RHEL RPM image

RHEL for Edge image

OS assembly

You can assemble the packages locally to form an image.

The packages are assembled in an ostree which you can install on a system.

OS updates

You can use yum update to apply the available updates from the enabled repositories.

You can use rpm-ostree upgrade to stage an update if any new commit is available in the ostree remote at /etc/ostree/remotes.d/. The update takes effect on system reboot.

Repository

The package contains YUM repositories

The package contains Ostree remote repository

User access permissions

Read write

Read-only (/usr)

Data persistence

You can mount the image to any non tmpfs mount point

/etc & /var are read-write enabled and include persisting data.

Chapter 2. Setting up image builder

Use image builder to create your customized RHEL for Edge images. After you install image builder on a RHEL system, image builder is available as an application in RHEL web console. You can also access image builder through a command line interface in the composer-cli tool.

Note

It is recommended to install image builder on a virtual machine.

In the environment where you want to install image builder, ensure that you first meet the system requirements and then install it.

2.1. Image builder system requirements

The environment where RHEL image builder runs, for example a virtual machine, must meet the requirements that are listed in the following table.

Note

Running RHEL image builder inside a container is not supported.

Table 2.1. Image builder system requirements

Parameter

Minimal Required Value

System type

A dedicated virtual machine

Processor

2 cores

Memory

4 GiB

Disk space

20 GiB

Access privileges

Administrator level (root)

Network

Connectivity to Internet

Note

The 20 GiB disk space requirement is enough to install and run image builder in the host. To build and deploy image builds, you must allocate additional dedicated disk space.

2.2. Installing image builder

To install image builder on a dedicated virtual machine, follow these steps:

Prerequisites

  • The virtual machine is created and is powered on.
  • You have installed RHEL and you have subscribed to RHSM or Red Hat Satellite.
  • You have enabled the BaseOS and AppStream repositories to be able to install the RHEL image builder packages.

Procedure

  1. Install the following packages on the virtual machine.

    • osbuild-composer
    • composer-cli
    • cockpit-composer
    • bash-completion
    • firewalld
    # yum install osbuild-composer composer-cli cockpit-composer bash-completion firewalld

    Image builder is installed as an application in RHEL web console.

  2. Reboot the virtual machine
  3. Configure the system firewall to allow access to the web console:

    # firewall-cmd --add-service=cockpit && firewall-cmd --add-service=cockpit --permanent
  4. Enable image builder.

    # systemctl enable osbuild-composer.socket cockpit.socket --now

    The osbuild-composer and cockpit services start automatically on first access.

  5. Load the shell configuration script so that the autocomplete feature for the composer-cli command starts working immediately without reboot:

    $ source /etc/bash_completion.d/composer-cli

Additional resources

Chapter 3. Managing image builder repositories

You can use the following types of repositories in image builder:

Custom third-party repositories
Use these to include packages that are not available in the official RHEL repositories.
Official repository overrides
Use these if you want to download base system RPMs from elsewhere than the official repositories, for example, a custom mirror in your network. Because using official repository overrides disables the default repositories, your custom mirror must contain all the necessary packages.

3.1. Adding custom third-party repositories

You can add custom third-party sources to your repositories and manage these repositories by using the composer-cli.

Prerequisites

  • You have the URL of the custom third-party repository.

Procedure

  1. Create a repository source file:

    id = "repository_id"
    name = "repository_name"
    type = "repository_type"
    url = "repository-url"
    check_gpg = false
    check_ssl = false

    For example:

    id = "k8s"
    name = "Kubernetes"
    type = "yum-baseurl"
    url = "https://packages.cloud.google.com/yum/repos/kubernetes-el7-x86_64"
    check_gpg = false
    check_ssl = false
    system = false
  2. Save the file in the TOML format.
  3. Add the new third-party source with the following command:

    $ composer-cli sources add <file-name>.toml

Verification

  • Check if the new source was successfully added:

    $ composer-cli sources list
  • Check the new source content:

    $ composer-cli sources info <source_id>

3.2. Adding third-party repositories with specific distributions

You can specify a list of distributions in the custom third-party source file by using the optional field distro. The repository file uses the distribution string list while resolving dependencies during the image building.

Any request that specifies rhel-8 will use this source. For example, if you list packages and specify rhel-8, it will include this source. However, listing packages for the host distribution will not include this source.

Procedure

  1. Create a repository source file:

    check_gpg = true
    check_ssl = true
    distros = ["list_of_distributions"]
    id = "repository_id"
    name = "repository-name"
    system = false
    type = ""repository_type"
    url = "repository-url"

    For example, to specify the distribution:

    check_gpg = true
    check_ssl = true
    distros = ["rhel-8"]
    id = "rh9-local"
    name = "packages for RHEL"
    system = false
    type = "yum-baseurl"
    url = "http://local/repos/rhel8/projectrepo/"
  2. Save the file in the TOML format.

3.3. Checking repositories metadata with GPG

To detect and avoid corrupted packages, you can use the DNF package manager to check the GNU Privacy Guard (GPG) signature on RPM packages, and also to check if the repository metadata has been signed with a GPG key.

For security reasons, you can distribute the key in a separate channel from the RPMs, by making your GPG key available over https. You can indicate which GPG key to use to do the check, by setting check_repogpg = true in the source. If the key is available over https, set the gpgkeys entry to the URL for the key. Optionally, you can also embed the whole key into the source gpgkeys entry to import it directly instead of fetching it from the URL.

Procedure

  1. Access the folder where you want to create a repository:

    $ cd repo/
  2. Run the createrepo_c to create a repository from RPM packages:

    $ createrepo_c .
  3. Access the directory where the repodata is:

    $ cd repodata/
  4. Set up a repository by signing your repomd.xml file:

    $ gpg -u YOUR-GPG-KEY-EMAIL --yes --detach-sign --armor repomd.xml
  5. Check the GPG signature.

    1. Set check_repogpg = true in the repository source.
    2. If your key is available over https, set the gpgkeys field with the key URL for the key. You can add as many URL keys as you need The following is an example:

      check_gpg = true
      check_ssl = true
      id = "repository_id"
      name = "repository_name"
      system = false
      type = "repository_type"
      url = "repository_URL"
      check_repogpg = true
      gpgkeys=["_GPG_key_URL"]
    3. Optional: You can embed the whole key into the gpgkeys field. You can add as many keys as you need. For example, add the GPG key directly in the gpgkeys field:

      check_gpg = true
      check_ssl = true
      check_repogpg
      id = "repository_id"
      name = "repository_name"
      system = false
      type = "repository_type"
      url = "repository_URL"
      gpgkeys=["GPG_key"]

Verification

  • Test the signature of the repository manually:

    $ gpg --verify repomd.xml.asc
  • If the test does not find the signature, you will be prompt with an error similar to the following one:

    $ GPG verification is enabled, but GPG signature is not available.
    This may be an error or the repository does not support GPG verification:
    Status code: 404 for http://repo-server/rhel/repodata/repomd.xml.asc (IP: 192.168.1.3)
  • If the signature is invalid, you will be prompt with an error similar to the following one:

    repomd.xml GPG signature verification error: Bad GPG signature

3.4. Image builder default system repositories

The osbuild-composer back end does not inherit the system repositories located in the /etc/yum.repos.d/ directory. Instead, it has its own set of official repositories defined in the /usr/share/osbuild-composer/repositories directory. This includes the Red Hat official repository, which contains the base system RPMs to install additional software or update already installed programs to newer versions. If you want to override the official repositories, you must define overrides in /etc/osbuild-composer/repositories. This directory is for user defined overrides and the files located there take precedence over those in the /usr directory.

The configuration files are not in the usual YUM repository format known from the files in /etc/yum.repos.d/. Instead, they are simple JSON files.

3.5. Overriding a system repository

You can configure a repository override for image builder in the /etc/osbuild-composer/repositories directory with the following steps.

Note

Prior to RHEL 8.5 release, the name of the repository overrides is rhel-8.json. Starting from RHEL 8.5, the names also respect the minor version: rhel-84.json, rhel-85.json, and so on.

Prerequisites

  • You have a custom repository that is accessible from the host system

Procedure

  1. Create a directory where you want to store your repository overrides:

    $ sudo mkdir -p /etc/osbuild-composer/repositories
  2. You can create your own JSON file structure.
  3. Create a JSON file, using a name corresponding to your RHEL version. Alternatively, you can copy the file for your distribution from /usr/share/osbuild-composer/ and modify its content.

    For RHEL 8, use /etc/osbuild-composer/repositories/rhel-88.json.

  4. Add the following structure to your JSON file, for example:

    {
        "<ARCH>": [
            {
                "name": "baseos",
                "baseurl": "http://mirror.example.com/composes/released/RHEL-8/8.0/BaseOS/x86_64/os/",
                "gpgkey": "-----BEGIN PGP PUBLIC KEY BLOCK-----\n\n (…​)",
                "check_gpg": true,
                "metadata_expire": ""
            }
        ]
    }

    Specify only one of the following attributes:

    • baseurl - string: a base URL of the repository.
    • metalink - string: a URL of a metalink file that contains a list of valid mirror repositories.
    • mirrorlist - string: a URL of a mirrorlist file that contains a list of valid mirror repositories

      The remaining fields are optional.

      1. Alternatively, you can copy the JSON file for your distribution.

        1. Copy the repository file to the directory you created. In the following command, replace rhel-version.json with your RHEL version, for example: rhel-8.json.

          $  cp /usr/share/osbuild-composer/repositories/rhel-version.json /etc/osbuild-composer/repositories/
  5. Using a text editor, edit the baseurl paths in the rhel-8.json file and save it. For example:

    $ vi /etc/osbuild-composer/repositories/rhel-version.json
  6. Restart the osbuild-composer.service:

    $ sudo systemctl restart osbuild-composer.service

Verification

  • Check if the repository points to the correct URLs:

    $ cat /etc/yum.repos.d/redhat.repo

    You can see that the repository points to the correct URLs which are copied from the /etc/yum.repos.d/redhat.repo file.

3.6. Overriding a system repository with support for subscriptions

The osbuild-composer service can use system subscriptions that are defined in the /etc/yum.repos.d/redhat.repo file. To use a system subscription in osbuild-composer, define a repository override that has:

  • The same baseurl as the repository defined in /etc/yum.repos.d/redhat.repo.
  • The value of ”rhsm”: true defined in the JSON object.

Prerequisites

Procedure

  1. Obtain the baseurl from the /etc/yum.repos.d/redhat.repo file:

    # cat /etc/yum.repos.d/redhat.repo
    [AppStream]
    name = AppStream mirror example
    baseurl = https://mirror.example.com/RHEL-8/8.0/AppStream/x86_64/os/
    enabled = 1
    gpgcheck = 0
    sslverify = 1
    sslcacert = /etc/pki/ca1/ca.crt
    sslclientkey = /etc/pki/ca1/client.key
    sslclientcert = /etc/pki/ca1/client.crt
    metadata_expire = 86400
    enabled_metadata = 0
  2. Configure the repository override to use the same baseurl and set rhsm to true:

    {
        "x86_64": [
            {
                "name": "AppStream mirror example",
                "baseurl": "https://mirror.example.com/RHEL-8/8.0/AppStream/x86_64/os/",
                "gpgkey": "-----BEGIN PGP PUBLIC KEY BLOCK-----\n\n (…​)",
                "check_gpg": true,
                "rhsm": true
            }
        ]
    }
    Note

    osbuild-composer does not automatically use repositories defined in /etc/yum.repos.d/. You need to manually specify them either as a system repository override or as an additional source using composer-cli. System repository overrides are usually used for “BaseOS” and “AppStream” repositories, whereas composer-cli sources are used for all the other repositories.

As a result, image builder reads the /etc/yum.repos.d/redhat.repo file from the host system and uses it as a source of subscriptions.

Chapter 4. Composing a RHEL for Edge image using image builder in RHEL web console

Use image builder to create a custom RHEL for Edge image (OSTree commit).

To access image builder and to create your custom RHEL for Edge image, you can either use the RHEL web console interface or the command-line interface.

You can compose RHEL for Edge images using image builder in RHEL web console by performing the following high-level steps:

  1. Access image builder in RHEL web console
  2. Create a blueprint for RHEL for Edge image.
  3. Create a RHEL for Edge image. You can create the following images:

    • RHEL for Edge Commit image.
    • RHEL for Edge Container image.
    • RHEL for Edge Installer image.
  4. Download the RHEL for Edge image

4.1. Accessing image builder in the RHEL web console

To access image builder in RHEL web console, ensure that you have met the following prerequisites and then follow the procedure.

Prerequisites

  • You have installed a RHEL system.
  • You have administrative rights on the system.
  • You have subscribed the RHEL system to Red Hat Subscription Manager (RHSM) or to Red Hat Satellite Server.
  • The system is powered on and accessible over network.
  • You have installed image builder on the system.

Procedure

  1. On your RHEL system, access https://localhost:9090/ in a web browser.
  2. For more information about how to remotely access image builder, see Managing systems using the RHEL 8 web console document.
  3. Log in to the web console using an administrative user account.
  4. On the web console, in the left hand menu, click Apps.
  5. Click Image Builder.

    The image builder dashboard opens in the right pane. You can now proceed to create a blueprint for the RHEL for Edge images.

4.2. Creating a blueprint for a RHEL for Edge image using image builder in RHEL web console

To create a blueprint for a RHEL for Edge image using image builder in RHEL web console, ensure that you have met the following prerequisites and then follow the procedure.

Prerequisites

  • On a RHEL system, you have opened the image builder dashboard.

Procedure

  1. On the image builder dashboard, click Create Blueprint.

    The Create Blueprint dialogue box opens.

  2. On the Details page:

    1. Enter the name of the blueprint and, optionally, its description. Click Next.
  3. Optional: In the Packages page:

    1. On the Available packages search, enter the package name and click the > button to move it to the Chosen packages field. Search and include as many packages as you want. Click Next.

      Note

      These customizations are all optional unless otherwise specified.

  4. On the Kernel page, enter a kernel name and the command-line arguments.
  5. On the File system page, select Use automatic partitioning. The filesystem customization is not supported for OSTree systems, because OSTree images have their own mount rule, such as read-only. Click Next.
  6. On the Services page, you can enable or disable services:

    1. Enter the service names you want to enable or disable, separating them by a comma, by space, or by pressing the Enter key. Click Next.
  7. On the Firewall page, set up your firewall setting:

    1. Enter the Ports, and the firewall services you want to enable or disable.
    2. Click the Add zone button to manage your firewall rules for each zone independently. Click Next.
  8. On the Users page, add a users by following the steps:

    1. Click Add user.
    2. Enter a Username, a password, and a SSH key. You can also mark the user as a privileged user, by clicking the Server administrator checkbox. Click Next.
  9. On the Groups page, add groups by completing the following steps:

    1. Click the Add groups button:

      1. Enter a Group name and a Group ID. You can add more groups. Click Next.
  10. On the SSH keys page, add a key:

    1. Click the Add key button.

      1. Enter the SSH key.
      2. Enter a User. Click Next.
  11. On the Timezone page, set your timezone settings:

    1. On the Timezone field, enter the timezone you want to add to your system image. For example, add the following timezone format: "US/Eastern".

      If you do not set a timezone, the system uses Universal Time, Coordinated (UTC) as default.

    2. Enter the NTP servers. Click Next.
  12. On the Locale page, complete the following steps:

    1. On the Keyboard search field, enter the package name you want to add to your system image. For example: ["en_US.UTF-8"].
    2. On the Languages search field, enter the package name you want to add to your system image. For example: "us". Click Next.
  13. On the Others page, complete the following steps:

    1. On the Hostname field, enter the hostname you want to add to your system image. If you do not add a hostname, the operating system determines the hostname.
    2. Mandatory only for the Simplifier Installer image: On the Installation Devices field, enter a valid node for your system image. For example: dev/sda. Click Next.
  14. Mandatory only when building FIDO images: On the FIDO device onboarding page, complete the following steps:

    1. On the Manufacturing server URL field, enter the following information:

      1. On the DIUN public key insecure field, enter the insecure public key.
      2. On the DIUN public key hash field, enter the public key hash.
      3. On the DIUN public key root certs field, enter the public key root certs. Click Next.
  15. On the OpenSCAP page, complete the following steps:

    1. On the Datastream field, enter the datastream remediation instructions you want to add to your system image.
    2. On the Profile ID field, enter the profile_id security profile you want to add to your system image. Click Next.
  16. Mandatory only when building Ignition images: On the Ignition page, complete the following steps:

    1. On the Firstboot URL field, enter the package name you want to add to your system image.
    2. On the Embedded Data field, drag or upload your file. Click Next.
  17. . On the Review page, review the details about the blueprint. Click Create.

The image builder view opens, listing existing blueprints.

4.3. Creating a RHEL for Edge Commit image using image builder in RHEL web console

The “RHEL for Edge Commit (.tar)” image type contains a full operating system, but it is not directly bootable. To boot the Commit image type, you must deploy it in a running container.

To create a RHEL for Edge Commit image using image builder in RHEL web console, follow the steps:

Prerequisites

  • On a RHEL system, you have accessed the image builder dashboard.

Procedure

  1. On the image builder dashboard click Create Image.
  2. On the Image output page, perform the following steps:

    1. From the Select a blueprint dropdown menu, select the blueprint you want to use.
    2. From the Image output type dropdown list, select “RHEL for Edge Commit (.tar)” for network-based deployment.
    3. Click Next.
    4. On the OSTree settings page, enter:

      1. Repository URL: specify the URL to the OSTree repository of the commit to embed in the image. For example, http://10.0.2.2:8080/repo/.
      2. Parent commit: specify a previous commit, or leave it empty if you do not have a commit at this time.
      3. In the Ref textbox, specify a reference path for where your commit is going to be created. By default, the web console specifies rhel/8/$ARCH/edge. The "$ARCH" value is determined by the host machine. Click Next.
    5. On the Review page, check the customizations and click Create.

      Image builder starts to create a RHEL for Edge Commit image for the blueprint that you created.

      Note

      The image creation process takes up to 20 minutes to complete.

Verification

  1. To check the RHEL for Edge Commit image creation progress:

    1. Click the Images tab.

After the image creation process is complete, you can download the resulting “RHEL for Edge Commit (.tar)” image.

Additional resources

4.4. Creating a RHEL for Edge Container image using image builder in RHEL web console

You can create RHEL for Edge images by selecting “RHEL for Edge Container (.tar)”. The RHEL for Edge Container (.tar) image type creates an OSTree commit and embeds it into an OCI container with a web server. When the container is started, the web server serves the commit as an OSTree repository.

Follow the steps in this procedure to create a RHEL for Edge Container image using image builder in RHEL web console.

Prerequisites

  • On a RHEL system, you have accessed the image builder dashboard.
  • You have created a blueprint.

Procedure

  1. On the image builder dashboard click Create Image.
  2. On the Image output page, perform the following steps:
  3. From the Select a blueprint dropdown menu, select the blueprint you want to use.

    1. From the Image output type dropdown list, select “RHEL for Edge Container (.tar)” for network-based deployment.
    2. Click Next.
    3. On the OSTree page, enter:

      1. Repository URL: specify the URL to the OSTree repository of the commit to embed in the image. For example, http://10.0.2.2:8080/repo/. By default, the repository folder for a RHEL for Edge Container image is "/repo".

        To find the correct URL to use, access the running container and check the nginx.conf file. To find which URL to use, access the running container and check the nginx.conf file. Inside the nginx.conf file, find the root directory entry to search for the /repo/ folder information. Note that, if you do not specify a repository URL when creating a RHEL for Edge Container image (.tar) using image builder, the default /repo/ entry is created in the nginx.conf file.

      2. Parent commit: specify a previous commit, or leave it empty if you do not have a commit at this time.
      3. In the Ref textbox, specify a reference path for where your commit is going to be created. By default, the web console specifies rhel/8/$ARCH/edge. The "$ARCH" value is determined by the host machine. Click Next.
    4. On the Review page, check the customizations. Click Save blueprint.
  4. Click Create.

    Image builder starts to create a RHEL for Edge Container image for the blueprint that you created.

    Note

    The image creation process takes up to 20 minutes to complete.

Verification

  1. To check the RHEL for Edge Container image creation progress:

    1. Click the Images tab.

After the image creation process is complete, you can download the resulting “RHEL for Edge Container (.tar)” image.

Additional resources

4.5. Creating a RHEL for Edge Installer image using image builder in RHEL web console

You can create RHEL for Edge Installer images for non-network-based deployment by selecting RHEL for Edge Installer (.iso). The RHEL for Edge Installer (.iso) image type pulls the OSTree commit repository from the running container served by the RHEL for Edge Container (.tar) and creates an installable boot ISO image with a Kickstart file that is configured to use the embedded OSTree commit.

Follow the steps in this procedure to create a RHEL for Edge image using image builder in RHEL web console.

Prerequisites

Procedure

  1. On the image builder dashboard click Create Image.
  2. On the Image output page, perform the following steps:

    1. From the Select a blueprint dropdown menu, select the blueprint you want to use.
    2. From the Image output type dropdown list, select RHEL for Edge Installer (.iso) image.
    3. Click Next.
    4. On the OSTree settings page, enter:

      1. Repository URL: specify the URL to the OSTree repository of the commit to embed in the image. For example, http://10.0.2.2:8080/repo/.
      2. In the Ref textbox, specify a reference path for where your commit is going to be created. By default, the web console specifies rhel/8/$ARCH/edge. The "$ARCH" value is determined by the host machine. Click Next.
    5. On the Review page, check the customizations. Click Save blueprint.
  3. Click Create.

    Image builder starts to create a RHEL for Edge Installer image for the blueprint that you created.

    Note

    The image creation process takes up to 20 minutes to complete.

Verification

After the image creation process is complete, you can download the resulting RHEL for Edge Installer (.iso) image.

  1. To check the RHEL for Edge Installer image creation progress:

    1. Click the Images tab.

After the image creation process is complete, you can download the resulting RHEL for Edge Installer (.iso) image and boot the ISO image into a device.

Additional resources

4.6. Downloading a RHEL for Edge image

After you successfully create the RHEL for Edge image by using image builder, download the image on the local host.

Procedure

To download an image:

  1. From the More Options menu, click Download.

    The image builder tool downloads the file at your default download location.

The downloaded file consists of a .tar file with an OSTree repository for RHEL for Edge Commit and RHEL for Edge Container images, or a .iso file for RHEL for Edge Installer images, with an OSTree repository. This repository contains the commit and a json file which contains information metadata about the repository content.

4.7. Additional resources

Chapter 5. Composing a RHEL for Edge image using image builder command-line

You can use image builder to create a customized RHEL for Edge image (OSTree commit).

To access image builder and to create your custom RHEL for Edge image, you can either use the RHEL web console interface or the command-line interface.

For Network-based deployments, the workflow to compose RHEL for Edge images using the CLI, involves the following high-level steps:

  1. Create a blueprint for RHEL for Edge image
  2. Create a RHEL for Edge Commit image
  3. Download the RHEL for Edge Commit image

For Non-Network-based deployments, the workflow to compose RHEL for Edge images using the CLI, involves the following high-level steps:

  1. Create a blueprint for RHEL for Edge image
  2. Create a blueprint for the RHEL for Edge Installer image
  3. Create a RHEL for Edge Container image
  4. Create a RHEL for Edge Installer image
  5. Download the RHEL for Edge image

To perform the steps, use the composer-cli package.

Note

To run the composer-cli commands as non-root, you must be part of the weldr group or you must have administrator access to the system.

5.1. Network-based deployments workflow

This provides steps on how to build OSTree commits. These OSTree commits contain a full operating system, but are not directly bootable. To boot them, you need to deploy them using a Kickstart file.

5.1.1. Creating a RHEL for Edge Commit image blueprint using image builder command-line interface

Create a blueprint for RHEL for Edge Commit image using the CLI.

Prerequisite

  • You do not have an existing blueprint. To verify that, list the existing blueprints:

    $ sudo composer-cli blueprints list

Procedure

  1. Create a plain text file in the TOML format, with the following content:

    name = "blueprint-name"
    description = "blueprint-text-description"
    version = "0.0.1"
    modules = [ ]
    groups = [ ]

    Where,

    • blueprint-name is the name and blueprint-text-description is the description for your blueprint.
    • 0.0.1 is the version number according to the Semantic Versioning scheme.
    • Modules describe the package name and matching version glob to be installed into the image, for example, the package name = "tmux" and the matching version glob is version = "2.9a".

      Notice that currently there are no differences between packages and modules.

    • Groups are packages groups to be installed into the image, for example the group package anaconda-tools.

      At this time, if you do not know the modules and groups, leave them empty.

  2. Include the required packages and customize the other details in the blueprint to suit your requirements.

    For every package that you want to include in the blueprint, add the following lines to the file:

    [[packages]]
    name = "package-name"
    version = "package-version"

    Where,

    • package-name is the name of the package, such as httpd, gdb-doc, or coreutils.
    • package-version is the version number of the package that you want to use.

      The package-version supports the following dnf version specifications:

    • For a specific version, use the exact version number such as 8.0.
    • For the latest available version, use the asterisk *.
    • For the latest minor version, use formats such as 8.*.
  3. Push (import) the blueprint to the image builder server:

    # composer-cli blueprints push blueprint-name.toml
  4. List the existing blueprints to check whether the created blueprint is successfully pushed and exists.

    # composer-cli blueprints show BLUEPRINT-NAME
  5. Check whether the components and versions listed in the blueprint and their dependencies are valid:

    # composer-cli blueprints depsolve blueprint-name

Additional resources

5.1.2. Creating a RHEL for Edge Commit image using image builder command-line interface

To create a RHEL for Edge Commit image using image builder command-line interface, ensure that you have met the following prerequisites and follow the procedure.

Prerequisites

  • You have created a blueprint for RHEL for Edge Commit image.

Procedure

  1. Create the RHEL for Edge Commit image.

    # composer-cli compose start blueprint-name image-type

    Where,

    • blueprint-name is the RHEL for Edge blueprint name.
    • image-type is edge-commit for network-based deployment.

      A confirmation that the composer process has been added to the queue appears. It also shows a Universally Unique Identifier (UUID) number for the image created. Use the UUID number to track your build. Also keep the UUID number handy for further tasks.

  2. Check the image compose status.

    # composer-cli compose status

    The output displays the status in the following format:

    <UUID> RUNNING date blueprint-name blueprint-version image-type
    Note

    The image creation process takes up to 20 minutes to complete.

    To interrupt the image creation process, run:

    # composer-cli compose cancel <UUID>

    To delete an existing image, run:

    # composer-cli compose delete <UUID>

    After the image is ready, you can download it and use the image on your network deployments.

5.1.3. Creating a RHEL for Edge image update with a ref commit using image builder command-line interface

If you performed a change in an existing blueprint, for example, you added a new package, and you want to update an existing RHEL for Edge image with this new package, you can use the --parent argument to generate an updated RHEL for Edge Commit (.tar) image. The --parent argument must be a ref that exists in the repository specified by the URL argument. The --ref argument enables you to specify an existing ref that retrieves a parent for the new commit that you are building. You must specify the parent commit as a ref value to be resolved and pulled, not as a commit ID, for example rhel/8/x86_64/edge. When you specify the parent commit as a ref value, image builder can read information from the parent commit that will affect parts of the new commit that you are building. As a result, image builder reads the parent commit’s user database and preserves UIDs and GIDs for the package-created system users and groups.

To create a RHEL for Edge image with a parent argument using image builder CLI, ensure that you have met the following prerequisites and follow the procedure.

Prerequisites

  • You have updated an existing blueprint for RHEL for Edge image.
  • You have an existing RHEL for Edge image (OSTree commit). See Extracting RHEL for Edge image commit.
  • The ref being built is available at the OSTree repository specified by the URL.

Procedure

  1. Create the RHEL for Edge commit image:

    # composer-cli compose start-ostree --ref rhel/8/x86_64/edge --parent parent-OSTree-REF --url URL blueprint-name image-type

    For example:

    • To create a new RHEL for Edge commit based on a parent and with a new ref, run the following command:

      # composer-cli compose start-ostree --ref rhel/8/x86_64/edge --parent rhel/8/x86_64/edge --url http://10.0.2.2:8080/repo rhel_update edge-commit
    • To create a new RHEL for Edge commit based on the same ref, run the following command:

      # composer-cli compose start-ostree --ref rhel/8/x86_64/edge --url http://10.0.2.2:8080/repo rhel_update edge-commit

      Where:

      • The --ref argument specifies the same path value that you used to build an OSTree repository.
      • The --parent argument specifies the parent commit as a ref to be resolved and pulled, not as a commit ID, for example rhel/8/x86_64/edge.
      • blueprint-name is the RHEL for Edge blueprint name.
      • The --url argument specifies the URL to the OSTree repository of the commit to embed in the image, for example, http://10.0.2.2:8080/repo.
      • image-type is edge-commit for network-based deployment.

        Note
        • The --parent argument can only be used for the RHEL for Edge Commit (.tar) image type. Using the --url and --parent arguments together results in errors with the RHEL for Edge Container (.tar) image type.
        • If you omit the parent ref argument, the system falls back to the ref specified by the --ref argument.

        A confirmation that the composer process has been added to the queue appears. It also shows a Universally Unique Identifier (UUID) number for the image created. Use the UUID number to track your build. Also keep the UUID number handy for further tasks.

  2. Check the image compose status.

    # composer-cli compose status

    The output displays the status in the following format:

    <UUID> RUNNING date blueprint-name blueprint-version image-type
    Note

    The image creation process takes a few minutes to complete.

    (Optional) To interrupt the image creation process, run:

    # composer-cli compose cancel <UUID>

    (Optional) To delete an existing image, run:

    # composer-cli compose delete <UUID>

After the image creation is complete, to upgrade an existing OSTree deployment, you need:

5.1.4. Downloading a RHEL for Edge image using the image builder command-line interface

To download a RHEL for Edge image using image builder command line interface, ensure that you have met the following prerequisites and then follow the procedure.

Prerequisites

  • You have created a RHEL for Edge image.

Procedure

  1. Review the RHEL for Edge image status.

    # composer-cli compose status

    The output must display the following:

    $ <UUID> FINISHED date blueprint-name blueprint-version image-type
  2. Download the image.

    # composer-cli compose image <UUID>

    Image builder downloads the image as a tar file to the current directory.

    The UUID number and the image size is displayed alongside.

    $ <UUID>-commit.tar: size MB

The image contains a commit and a json file with information metadata about the repository content.

5.2. Non-network-based deployments workflow

To build a boot ISO image that installs an OSTree-based system using the "RHEL for Edge Container" and the "RHEL for Edge Installer" images and that can be later deployed to a device in disconnected environments, follow the steps.

5.2.1. Creating a RHEL for Edge Container blueprint by using image builder CLI

To create a blueprint for RHEL for Edge Container image, perform the following steps:

Procedure

  1. Create a plain text file in the TOML format, with the following content:

    name = "blueprint-name"
    description = "blueprint-text-description"
    version = "0.0.1"
    modules = [ ]
    groups = [ ]

    Where,

    • blueprint-name is the name and blueprint-text-description is the description for your blueprint.
    • 0.0.1 is the version number according to the Semantic Versioning scheme.
    • Modules describe the package name and matching version glob to be installed into the image, for example, the package name = "tmux" and the matching version glob is version = "2.9a".

      Notice that currently there are no differences between packages and modules.

    • Groups are packages groups to be installed into the image, for example the group package anaconda-tools.

      At this time, if you do not know the modules and groups, leave them empty.

  2. Include the required packages and customize the other details in the blueprint to suit your requirements.

    For every package that you want to include in the blueprint, add the following lines to the file:

    [[packages]]
    name = "package-name"
    version = "package-version"

    Where,

    • package-name is the name of the package, such as httpd, gdb-doc, or coreutils.
    • package-version is the version number of the package that you want to use.

      The package-version supports the following dnf version specifications:

    • For a specific version, use the exact version number such as 8.0.
    • For the latest available version, use the asterisk *.
    • For the latest minor version, use formats such as 8.*.
  3. Push (import) the blueprint to the image builder server:

    # composer-cli blueprints push blueprint-name.toml
  4. List the existing blueprints to check whether the created blueprint is successfully pushed and exists.

    # composer-cli blueprints show BLUEPRINT-NAME
  5. Check whether the components and versions listed in the blueprint and their dependencies are valid:

    # composer-cli blueprints depsolve blueprint-name

Additional resources

5.2.2. Creating a RHEL for Edge Installer blueprint using image builder CLI

You can create a blueprint to build a RHEL for Edge Installer (.iso) image, and specify user accounts to automatically create one or more users on the system at installation time. See Creating an administrative user account for a RHEL for Edge image blueprint. It creates a user on the system at installation time.

Warning

When you create a user in the blueprint with the customizations.user customization, the blueprint creates the user under the /usr/lib/passwd directory and the password, under the /usr/etc/shadow directory. Note that you cannot change the password in further versions of the image in a running system using OSTree updates. The users you create with blueprints must be used only to gain access to the created system. After you access the system, you need to create users, for example, using the useradd command.

To create a blueprint for RHEL for Edge Installer image, perform the following steps:

Procedure

  1. Create a plain text file in the TOML format, with the following content:

    name = "blueprint-installer"
    description = "blueprint-for-installer-image"
    version = "0.0.1"
    
    [[customizations.user]]
    name = "user"
    description = "account"
    password = "user-password"
    key = "user-ssh-key "
    home = "path"
    groups = ["user-groups"]

    Where,

    • blueprint-name is the name and blueprint-text-description is the description for your blueprint.
    • 0.0.1 is the version number according to the Semantic Versioning scheme.
  2. Push (import) the blueprint to the image builder server:

    # composer-cli blueprints push blueprint-name.toml
  3. List the existing blueprints to check whether the created blueprint is successfully pushed and exists.

    # composer-cli blueprints show blueprint-name
  4. Check whether the components and versions listed in the blueprint and their dependencies are valid:

    # composer-cli blueprints depsolve blueprint-name

Additional resources

5.2.3. Creating a RHEL for Edge Container image by using image builder CLI

To create a RHEL for Edge Container image using image builder command-line interface, ensure that you have met the following prerequisites and follow the procedure.

Prerequisites

  • You have created a blueprint for RHEL for Edge Container image.

Procedure

  1. Create the RHEL for Edge Container image.

    # composer-cli compose start-ostree --ref rhel/8/x86_64/edge --url URL-OSTree-repository blueprint-name image-type

    Where,

    • --ref is the same value that customer used to build ostree repository
    • --url is the URL to the OSTree repository of the commit to embed in the image. For example, http://10.0.2.2:8080/repo/. By default, the repository folder for a RHEL for Edge Container image is "/repo". See Setting up a web server to install RHEL for Edge image.

      To find the correct URL to use, access the running container and check the nginx.conf file. To find which URL to use, access the running container and check the nginx.conf file. Inside the nginx.conf file, find the root directory entry to search for the /repo/ folder information. Note that, if you do not specify a repository URL when creating a RHEL for Edge Container image (.tar) using image builder, the default /repo/ entry is created in the nginx.conf file.

    • blueprint-name is the RHEL for Edge blueprint name.
    • image-type is edge-container for non-network-based deployment.

      A confirmation that the composer process has been added to the queue appears. It also shows a Universally Unique Identifier (UUID) number for the image created. Use the UUID number to track your build. Also keep the UUID number handy for further tasks.

  2. Check the image compose status.

    # composer-cli compose status

    The output displays the status in the following format:

    <UUID> RUNNING date blueprint-name blueprint-version image-type
    Note

    The image creation process takes up to 20 minutes to complete.

    To interrupt the image creation process, run:

    # composer-cli compose cancel <UUID>

    To delete an existing image, run:

    # composer-cli compose delete <UUID>

    After the image is ready, it can be used for non-network deployments. See Creating a RHEL for Edge Container image for non-network-based deployments.

5.2.4. Creating a RHEL for Edge Installer image using command-line interface for non-network-based deployments

To create a RHEL for Edge Installer image that embeds the OSTree commit, using image builder command-line interface, ensure that you have met the following prerequisites and then follow the procedure.

Prerequisites

  • You have created a blueprint for RHEL for Edge Installer image.
  • You have created a RHEL for Edge Edge Container image and deployed it using a web server.

Procedure

  1. Begin to create the RHEL for Edge Installer image.

    # composer-cli compose start-ostree --ref rhel/8/x86_64/edge --url URL-OSTree-repository blueprint-name image-type

    Where,

    • ref is the same value that customer used to build ostree repository
    • URL-OSTree-repository is the URL to the OSTree repository of the commit to embed in the image. For example, http://10.0.2.2:8080/repo. See Creating a RHEL for Edge Container image for non-network-based deployments.
    • blueprint-name is the RHEL for Edge Installer blueprint name.
    • image-type is edge-installer.

      A confirmation that the composer process has been added to the queue appears. It also shows a Universally Unique Identifier (UUID) number for the image created. Use the UUID number to track your build. Also keep the UUID number handy for further tasks.

  2. Check the image compose status.

    # composer-cli compose status

    The command output displays the status in the following format:

    <UUID> RUNNING date blueprint-name blueprint-version image-type
    Note

    The image creation process takes a few minutes to complete.

    To interrupt the image creation process, run:

    # composer-cli compose cancel <UUID>

    To delete an existing image, run:

    # composer-cli compose delete <UUID>

    After the image is ready, you can use it for non-network deployments. See Installing the RHEL for Edge image for non-network-based deployments.

5.2.5. Downloading a RHEL for Edge Installer image using the image builder CLI

To download a RHEL for Edge Installer image using image builder command line interface, ensure that you have met the following prerequisites and then follow the procedure.

Prerequisites

  • You have created a RHEL for Edge Installer image.

Procedure

  1. Review the RHEL for Edge image status.

    # composer-cli compose status

    The output must display the following:

    $ <UUID> FINISHED date blueprint-name blueprint-version image-type
  2. Download the image.

    # composer-cli compose image <UUID>

    Image builder downloads the image as an .iso file to the current directory.

    The UUID number and the image size is displayed alongside.

    $ <UUID>-boot.iso: size MB

The resulting image is a bootable ISO image.

5.3. Supported image customizations

You can customize your image by adding to your blueprint an additional RPM package, by enabling a service, or by customizing a kernel command line parameter. You can use several image customizations within blueprints. To make use of these options, you must configure the customizations in the blueprint and import (push) it to image builder.

Note

These customizations are not supported when using image builder in the web console.

Select a distribution
name = "blueprint_name"
description = "blueprint_version"
version = "0.1"
distro = "different_minor_version"

Replace "different_minor_version" to build a different minor version, for example, if you want to build a RHEL 8.6, use distro = "rhel-86". On a RHEL 8.7, you can build minor versions such as RHEL 8.6 and earlier releases. If you do not specify a distribution, the blueprint uses the host distribution. In case you upgrade the host operating system, the blueprints with no distribution set will build images using the new OS version.

Select a package group
[[packages]]
name = "package_group_name"

Replace "package_group_name" with the name of the group. For example, "@server with gui".

Set the image hostname
[customizations]
hostname = "baseimage"
User specifications for the resulting system image
[[customizations.user]]
name = "USER-NAME"
description = "USER-DESCRIPTION"
password = "PASSWORD-HASH"
key = "PUBLIC-SSH-KEY"
home = "/home/USER-NAME/"
shell = "/usr/bin/bash"
groups = ["users", "wheel"]
uid = NUMBER
gid = NUMBER

The GID is optional and must already exist in the image. Optionally, a package creates it, or the blueprint creates the GID by using the [[customizations.group]] entry.

Important

To generate the password hash, you must install python3 on your system.

# yum install python3

Replace PASSWORD-HASH with the actual password hash. To generate the password hash, use a command such as:

$ python3 -c 'import crypt,getpass;pw=getpass.getpass();print(crypt.crypt(pw) if (pw==getpass.getpass("Confirm: ")) else exit())'

Replace PUBLIC-SSH-KEY with the actual public key.

Replace the other placeholders with suitable values.

You must enter the name. You can omit any of the lines that you do not need.

Repeat this block for every user to include.

Group specifications for the resulting system image
[[customizations.group]]
name = "GROUP-NAME"
gid = NUMBER

Repeat this block for every group to include.

Set an existing users SSH key
[[customizations.sshkey]]
user = "root"
key = "PUBLIC-SSH-KEY"
Note

The "Set an existing user SSH key" customization is only applicable for existing users. To create a user and set an SSH key, see the User specifications for the resulting system image customization.

Append a kernel boot parameter option to the defaults
[customizations.kernel]
append = "KERNEL-OPTION"
By default, image builder builds a default kernel into the image. But, you can customize the kernel with the following configuration in blueprint
[customizations.kernel]
name = "KERNEL-rt"
Define a kernel name to use in an image
[customizations.kernel.name]
name = "KERNEL-NAME"
Set the timezone and the Network Time Protocol (NTP) servers for the resulting system image
[customizations.timezone]
timezone = "TIMEZONE"
ntpservers = "NTP_SERVER"

If you do not set a timezone, the system uses Universal Time, Coordinated (UTC) as default. Setting NTP servers is optional.

Set the locale settings for the resulting system image
[customizations.locale]
languages = ["LANGUAGE"]
keyboard = "KEYBOARD"

Setting both the language and the keyboard options is mandatory. You can add many other languages. The first language you add will be the primary language and the other languages will be secondary. For example:

[customizations.locale]
languages = ["en_US.UTF-8"]
keyboard = "us"

To list the values supported by the languages, run the following command:

$ localectl list-locales

To list the values supported by the keyboard, run the following command:

$ localectl list-keymaps
Firewall customization

Set the firewall for the resulting system image. By default, the firewall blocks all access, except for services that enable their ports explicitly, such as sshd. The following blueprint can be used to open other ports or services.

If you do not want to use the [customizations.firewall] or the [customizations.firewall.services], either remove the attributes, or set them to an empty list []. If you only want to use the default firewall setup, you can omit the customization from the blueprint.

Note

The Google and OpenStack templates explicitly disable the firewall for their environment. This cannot be overridden by the blueprint.

[customizations.firewall]
ports = ["PORTS"]

Where ports is an optional list of strings that contain ports or a range of ports and protocols to open. You can configure ports by using the following format: port:protocol format

You can configure the port ranges by using the portA-portB:protocol format. For example:

[customizations.firewall]
ports = ["22:tcp", "80:tcp", "imap:tcp", "53:tcp", "53:udp", "30000-32767:tcp", "30000-32767:udp"]

You can use numeric ports, or their names from the /etc/services to enable or disable port lists.

Customize the firewall services

services is an optional object with the following attributes containing services to enable or disable for firewalld:

  • enabled - An optional list of strings for services to enable.
  • disabled - An optional list of strings for services to disable.

    Check the available firewall services.

    $ firewall-cmd --get-services

    In the blueprint, under section customizations.firewall.service, specify the firewall services that you want to customize.

    [customizations.firewall.services]
    enabled = ["SERVICES"]
    disabled = ["SERVICES"]

    For example:

    [customizations.firewall.services]
    enabled = ["ftp", "ntp", "dhcp"]
    disabled = ["telnet"]

    The services listed in firewall.services are different from the service-names available in the /etc/services file.

    Note

    If you do not want to customize the firewall services, omit the [customizations.firewall] and [customizations.firewall.services] sections from the blueprint.

Set which services to enable during the boot time
[customizations.services]
enabled = ["SERVICES"]
disabled = ["SERVICES"]

You can control which services to enable during the boot time. Some image types already have services enabled or disabled to ensure that the image works correctly and this setup cannot be overridden. The [customizations.services] customization in the blueprint do not replace these services, but add them to the list of services already present in the image templates.

Note

Each time a build starts, it clones the repository of the host system. If you refer to a repository with a large amount of history, it might take some time to clone and it uses a significant amount of disk space. Also, the clone is temporary and the build removes it after it creates the RPM package.

Specify a custom filesystem configuration

You can specify a custom filesystem configuration in your blueprints and therefore create images with a specific disk layout, instead of the default layout configuration. By using the non-default layout configuration in your blueprints, you can benefit from:

  • security benchmark compliance
  • protection against out-of-disk errors
  • improved performance
  • consistency with existing setups

    To customize the filesystem configuration in your blueprint:

    Note

    The filesystem customization is not supported for OSTree systems, because OSTree images have their own mount rule, such as read-only.

    [[customizations.filesystem]]
    mountpoint = "MOUNTPOINT"
    size = MINIMUM-PARTITION-SIZE

    The blueprint supports the following mountpoints and their sub-directories:

    • / - the root mount point
    • /var
    • /home
    • /opt
    • /srv
    • /usr
    • /app
    • /data
    • /boot - Supported from RHEL 8.7 and RHEL 9.1 onward.

      Note

      Customizing mount points is only supported from RHEL 8.5 and RHEL 9.0 distributions onward, by using the CLI. In earlier distributions, you can only specify the root partition as a mount point and specify the size argument as an alias for the image size.

      If you have more than one partition in the customized image, you can create images with a customized file system partition on LVM and resize those partitions at runtime. To do this, you can specify a customized filesystem configuration in your blueprint and therefore create images with the desired disk layout. The default filesystem layout remains unchanged - if you use plain images without file system customization, and cloud-init resizes the root partition.

      Note

      From 8.6 onward, for the osbuild-composer-46.1-1.el8 RPM and later version, the physical partitions are no longer available and filesystem customizations create logical volumes.

      The blueprint automatically converts the file system customization to a LVM partition.

      The MINIMUM-PARTITION-SIZE value has no default size format. The blueprint customization supports the following values and units: kB to TB and KiB to TiB. For example, you can define the mount point size in bytes:

      [[customizations.filesystem]]
      mountpoint = "/var"
      size = 1073741824

      You can also define the mount point size by using units.

      Note

      You can only define the mount point size by using units for the package version provided for RHEL 8.6 and RHEL 9.0 distributions onward.

      For example:

      [[customizations.filesystem]]
      mountpoint = "/opt"
      size = "20 GiB"
      
      or
      
      [[customizations.filesystem]]
      mountpoint = "/boot"
      size = "1 GiB"
Create customized directories and files for your image under the /etc directory

To create customized files and directories in your image, use the [[customizations.files]] and the [[customizations.directories]] blueprint customizations. Currently, you can use these customizations only in the /etc directory.

Note

These blueprint customizations are supported by all image types, except the image types that deploy OSTree commits, such as edge-raw-image, edge-installer, and edge-simplified-installer.

Create a custom directory blueprint customization

With the [[customizations.directories]] blueprint customization, you can create customized directories in the /etc directory of your image.

Warning

If you use the customizations.directories with a directory path which already exists in the image with mode, user or group already set, the image build fails to prevent changing the ownership or permissions of the existing directory.

With the [[customizations.directories]] blueprint customization you can:

  • Create new directories.
  • Set user and group ownership for the directory you are creating.
  • Set the directory mode permission in the octal format.
  • Ensure that parent directories are created as needed.

To customize a directory configuration in your blueprint, create a file with the following content, for example:

[[customizations.directories]]
path = "/etc/directory_name"
mode = "octal_access_permission"
user = "user_string_or_integer"
group = "group_string_or_integer"
ensure_parents = boolean

The blueprint entries are described as following:

  • path - Mandatory - enter the path to the directory that you want to create. It must be an absolute path under the /etc directory.
  • mode - Optional - set the access permission on the directory, in the octal format. If you do not specify a permission, it defaults to 0755. The leading zero is optional.
  • user - Optional - set a user as the owner of the directory. If you do not specify a user, it defaults to root. You can specify the user as a string or as an integer.
  • group - Optional - set a group as the owner of the directory. If you do not specify a group, it defaults to root. You can specify the group as a string or as an integer.
  • ensure_parents - Optional - Specify whether you want to create parent directories as needed. If you do not specify a value, it defaults to false.

Create a custom file blueprint customization

You can use the custom file blueprint customization to create new files or to replace existing files. The parent directory of the file you specify must exist, otherwise, the image build fails. Ensure that the parent directory exists by specifying it in the [[customizations.directories]] customization.

Warning

If you combine the files customizations with other blueprint customizations, it might affect the functioning of the other customizations, or it might override the current files customizations. If you are not sure about the customizations, use the appropriate blueprint customization.

With the [[customizations.files]] blueprint customization you can:

  • Create new text files.
  • Modifying existing files. WARNING: this can override the existing content.
  • Set user and group ownership for the file you are creating.
  • Set the mode permission in the octal format.

    Note

    You cannot create or replace the following files:

  • /etc/fstab
  • /etc/shadow
  • /etc/passwd
  • /etc/group

To customize a file in your blueprint, create a file with the following content, for example:

[[customizations.files]]
path = "/etc/directory_name"
mode = "octal_access_permission"
user = "user_string_or_integer"
group = "group_string_or_integer"
data = "Hello world!"

The blueprint entries are described as following:

  • path - Mandatory - enter the path to the file that you want to create. It must be an absolute path under the /etc directory.
  • mode Optional - set the access permission on the file, in the octal format. If you do not specify a permission, it defaults to 0644. The leading zero is optional.
  • user - Optional - set a user as the owner of the file. If you do not specify a user, it defaults to root. You can specify the user as a string or as an integer.
  • group - Optional - set a group as the owner of the file. If you do not specify a group, it defaults to root. You can specify the group as a string or as an integer.
  • data - Optional - Specify the content of a plain text file. If you do not specify a content, it creates an empty file.

5.4. Additional resources

Chapter 6. Building simplified installer images to provision a RHEL for Edge image

You can build a RHEL for Edge Simplified Installer image, which is optimized for unattended installation to a device, and provision the image to a RHEL for Edge image.

6.1. Simplified installer image build and deployment

Build a RHEL for Edge Simplified Installer image by using the edge-simplified-installer image type,.

To build a RHEL for Edge Simplified Installer image, provide an existing OSTree commit. The resulting simplified image contains a raw image that has the OSTree commit deployed. After you boot the Simplified installer ISO image, it provisions a RHEL for Edge system that you can use on a hard disk or as a boot image in a virtual machine. You can log in to the deployed system with the user name and password that you specified in the blueprint that you used to create the Simplified Installer image.

The RHEL for Edge Simplified Installer image is optimized for unattended installation to a device and supports both network-base deployment and non-network-based deployments. However, for network-based deployment, it supports only UEFI HTTP boot.

Composing and deploying a simplified RHEL for Edge image involves the following high-level steps:

  1. Install and register a RHEL system
  2. Install image builder
  3. Using image builder, create a blueprint with customizations for RHEL for Edge Container image
  4. Import the RHEL for Edge blueprint in image builder
  5. Create a RHEL for Edge image embed in an OCI container with a web server ready to deploy the commit as an OSTree repository
  6. Create a blueprint for the edge-simplified-installer image
  7. Build a simplified RHEL for Edge image
  8. Download the RHEL for Edge simplified image
  9. Install the raw image with the edge-simplified-installer virt-install

The following diagram represents the RHEL for Edge Simplified building and provisioning workflow:

Figure 6.1. Building and provisioning RHEL for Edge in network-base environment

RHEL for Edge Simplified workflow

6.2. Creating a blueprint for a Simplified image using image builder CLI

To create a blueprint for a simplified RHEL for Edge image, you must customize it with a device file location to enable an unattended installation to a device and a URL to perform the initial device credential exchange. You also must specify users and user groups in the blueprint. For that,follow the steps:

Procedure

  1. Create a plain text file in the Tom’s Obvious, Minimal Language (TOML) format, with the following content:

    name = "simplified-installer-blueprint"
    description = "blueprint for the simplified installer image"
    version = "0.0.1"
    packages = []
    modules = []
    groups = []
    distro = ""
    
    [customizations]
    installation_device = "/dev/vda"
    
    [[customizations.user]]
    name = "admin"
    password = "admin"
    groups = ["users", "wheel"]
    
    [customizations.fdo]
    manufacturing_server_url = "http://10.0.0.2:8080"
    diun_pub_key_insecure = "true"
    Note

    The FDO customization in the blueprints is optional, and you can build your RHEL for Edge Simplified Installer image with no errors.

    • name is the name and description is the description for your blueprint.
    • 0.0.1 is the version number according to the Semantic Versioning scheme.
    • Modules describe the package name and matching version glob to be installed into the image, for example, the package name = "tmux" and the matching version glob is version = "2.9a". Notice that currently there are no differences between packages and modules.
    • Groups are packages groups to be installed into the image, for example the anaconda-tools group package. If you do not know the modules and groups, leave them empty.
    • installation-device is the customization to enable an unattended installation to your device.
    • manufacturing_server_url is the URL to perform the initial device credential exchange.
    • name is the user name to login to the image.
    • password is a password of your choice.
    • groups are any user groups, such as "widget".
  2. Push (import) the blueprint to the image builder server:

    # composer-cli blueprints push blueprint-name.toml
  3. List the existing blueprints to check whether the created blueprint is successfully pushed and exists.

    # composer-cli blueprints show blueprint-name
  4. Check whether the components and versions listed in the blueprint and their dependencies are valid:

    # composer-cli blueprints depsolve blueprint-name

6.3. Creating a RHEL for Edge Simplified Installer image using image builder CLI

To create a RHEL for Edge Simplified image using image builder command-line interface, ensure that you have met the following prerequisites and then follow the procedure.

Prerequisites

Procedure

  1. Create the bootable ISO image.

    # composer-cli compose start-ostree \
    blueprint-name \
    edge-simplified-installer \
    --ref rhel/8/x86_64/edge \
    --url URL-OSTree-repository \

    Where,

    • blueprint-name is the RHEL for Edge blueprint name.
    • edge-simplified-installer is the image type .
    • --ref is the reference for where your commit is going to be created.
    • --url is the URL to the OSTree repository of the commit to embed in the image. For example, http://10.0.2.2:8080/repo/. You can either start a RHEL for Edge Container or set up a web server. See Creating a RHEL for Edge Container image for non-network-based deployments and Setting up a web server to install RHEL for Edge image.

      A confirmation that the composer process has been added to the queue appears. It also shows a Universally Unique Identifier (UUID) number for the image created. Use the UUID number to track your build. Also keep the UUID number handy for further tasks.

  2. Check the image compose status.

    # composer-cli compose status

    The output displays the status in the following format:

    <UUID> RUNNING date blueprint-name blueprint-version image-type
    Note

    The image creation processes can take up to ten minutes to complete.

    To interrupt the image creation process, run:

    # composer-cli compose cancel <UUID>

    To delete an existing image, run:

    # composer-cli compose delete <UUID>

6.4. Downloading a simplified RHEL for Edge image using the image builder command-line interface

To download a RHEL for Edge image using image builder command line interface, ensure that you have met the following prerequisites and then follow the procedure.

Prerequisites

  • You have created a RHEL for Edge image.

Procedure

  1. Review the RHEL for Edge image status.

    # composer-cli compose status

    The output must display the following:

    $ <UUID> FINISHED date blueprint-name blueprint-version image-type
  2. Download the image.

    # composer-cli compose image <UUID>

    Image builder downloads the image as an .iso file at the current directory path where you run the command.

    The UUID number and the image size is displayed alongside.

    $ <UUID>-simplified-installer.iso: size MB

As a result, you downloaded a RHEL for Edge Simplified Installer ISO image. You can use it directly as a boot ISO to install a RHEL for Edge system.

6.5. Creating a blueprint for a Simplified image using image builder GUI

To create a RHEL for Edge Simplified Installer image, you must create a blueprint and ensure that you customize it with:

  • A device node location to enable an unattended installation to your device.
  • A URL to perform the initial device credential exchange.
  • A user or user group.
Note

You can also add any other customizations that your image requires.

To create a blueprint for a simplified RHEL for Edge image in the image builder GUI, complete the following steps:

Prerequisites

Procedure

  1. Click Create Blueprint in the upper-right corner of the image builder app.

    A dialog wizard with fields for the blueprint name and description opens.

  2. On the Details page:

    1. Enter the name of the blueprint and, optionally, its description. Click Next.
  3. Optional: On the Packages page, complete the following steps:

    1. In the Available packages search, enter the package name and click the > button to move it to the Chosen packages field. Search and include as many packages as you want. Click Next.

      Note

      The customizations are all optional unless otherwise specified.

  4. Optional: On the Kernel page, enter a kernel name and the command-line arguments.
  5. Optional: On the File system page, select Use automatic partitioning.The filesystem customization is not supported for OSTree systems, because OSTree images have their own mount rule, such as read-only. Click Next.
  6. Optional: On the Services page, you can enable or disable services:

    1. Enter the service names you want to enable or disable, separating them by a comma, by space, or by pressing the Enter key. Click Next.
  7. Optional: On the Firewall page, set up your firewall setting:

    1. Enter the Ports, and the firewall services you want to enable or disable.
    2. Click the Add zone button to manage your firewall rules for each zone independently. Click Next.
  8. On the Users page, add a users by following the steps:

    1. Click Add user.
    2. Enter a Username, a password, and a SSH key. You can also mark the user as a privileged user, by clicking the Server administrator checkbox.

      Note

      When you specify the user in the blueprint customization and then create an image from that blueprint, the blueprint creates the user under the /usr/lib/passwd directory and the password under the /usr/etc/shadow during installation time. You can log in to the device with the username and password you created for the blueprint. After you access the system, you must create users, for example, using the useradd command.

      Click Next.

  9. Optional: On the Groups page, add groups by completing the following steps:

    1. Click the Add groups button:

      1. Enter a Group name and a Group ID. You can add more groups. Click Next.
  10. Optional: On the SSH keys page, add a key:

    1. Click the Add key button.

      1. Enter the SSH key.
      2. Enter a User. Click Next.
  11. Optional: On the Timezone page, set your timezone settings:

    1. On the Timezone field, enter the timezone you want to add to your system image. For example, add the following timezone format: "US/Eastern".

      If you do not set a timezone, the system uses Universal Time, Coordinated (UTC) as default.

    2. Enter the NTP servers. Click Next.
  12. Optional: On the Locale page, complete the following steps:

    1. On the Keyboard search field, enter the package name you want to add to your system image. For example: ["en_US.UTF-8"].
    2. On the Languages search field, enter the package name you want to add to your system image. For example: "us". Click Next.
  13. Mandatory: On the Others page, complete the following steps:

    1. In the Hostname field, enter the hostname you want to add to your system image. If you do not add a hostname, the operating system determines the hostname.
    2. Mandatory: In the Installation Devices field, enter a valid node for your system image to enable an unattended installation to your device. For example: dev/sda1. Click Next.
  14. Optional: On the FIDO device onboarding page, complete the following steps:

    1. On the Manufacturing server URL field, enter the manufacturing server URL to perform the initial device credential exchange, for example: "http://10.0.0.2:8080". The FDO customization in the blueprints is optional, and you can build your RHEL for Edge Simplified Installer image with no errors.
    2. On the DIUN public key insecure field, enter the certification public key hash to perform the initial device credential exchange. This field accepts "true" as value, which means this is an insecure connection to the manufacturing server. For example: manufacturing_server_url="http://${FDO_SERVER}:8080" diun_pub_key_insecure="true". You must use only one of these three options: "key insecure", "key hash" and "key root certs".
    3. On the DIUN public key hash field, enter the hashed version of your public key. For example: 17BD05952222C421D6F1BB1256E0C925310CED4CE1C4FFD6E5CB968F4B73BF73. You can get the key hash by generating it based on the certificate of the manufacturing server. To generate the key hash, run the command:

      # openssl x509 -fingerprint -sha256 -noout -in /etc/fdo/aio/keys/diun_cert.pem | cut -d"=" -f2 | sed 's/://g'

      The /etc/fdo/aio/keys/diun_cert.pem is the certificate that is stored in the manufacturing server.

    4. On the DIUN public key root certs field, enter the public key root certs. This field accepts the content of the certification file that is stored in the manufacturing server. To get the content of certificate file, run the command:

      $ cat /etc/fdo/aio/keys/diun_cert.pem.
  15. Click Next.
  16. On the Review page, review the details about the blueprint. Click Create.

The image builder view opens, listing existing blueprints.

6.6. Creating a RHEL for Edge Simplified Installer image using image builder GUI

To create a RHEL for Edge Simplified image using image builder GUI, ensure that you have met the following prerequisites and then follow the procedure.

Prerequisites

  • You opened the image builder app from the web console in a browser.
  • You created a blueprint for the RHEL for Edge Simplified image.
  • You served an OSTree repository of the commit to embed in the image, for example, http://10.0.2.2:8080/repo. See Setting up a web server to install RHEL for Edge image.
  • The FDO manufacturing server is up and running.

Procedure

  1. Access mage builder dashboard.
  2. On the blueprint table, find the blueprint you want to build an image for.
  3. Navigate to the Images tab and click Create Image. The Create image wizard opens.
  4. On the Image output page, complete the following steps:

    1. From the Select a blueprint list, select the blueprint you created for the RHEL for Edge Simplified image.
    2. From the Image output type list, select RHEL for Edge Simplified Installer (.iso).
    3. In the Image Size field, enter the image size. Minimum image size required for Simplified Installer image is:
  5. Click Next.
  6. In the OSTree settings page, complete the following steps:

    1. In the Repository URL field, enter the repository URL to where the parent OSTree commit will be pulled from.
    2. In the Ref field, enter the ref branch name path. If you do not enter a ref, the default ref for the distro is used.
  7. On the Review page, review the image customization and click Create.

The image build starts and takes up to 20 minutes to complete. To stop the building, click Stop build.

6.7. Downloading a simplified RHEL for Edge image using the image builder GUI

To download a RHEL for Edge image using image builder GUI, ensure that you have met the following prerequisites and then follow the procedure.

Prerequisites

  • You have successfully created a RHEL for Edge image. See link.

Procedure

  1. Access the image builder dashboard. The blueprint list dashboard opens.
  2. In the blueprint table, find the blueprint you built your RHEL for Edge Simplified Installer image for.
  3. Navigate to the Images tab.
  4. Choose one of the options:

    • Download the image.
    • Download the logs of the image to inspect the elements and verify if any issue is found.
Note

You can use the RHEL for Edge Simplified Installer ISO image that you downloaded directly as a boot ISO to install a RHEL for Edge system.

6.8. Setting up an UEFI HTTP Boot server

To set up an UEFI HTTP Boot server, so that you can start to provision a RHEL for Edge Virtual Machine over network by connecting to this UEFI HTTP Boot server, follow the steps:

Prerequisites

  • You have created the ISO simplified installer image.
  • An http server that serves the ISO content.

Procedure

  1. Mount the ISO image to the directory of your choice:

    # mkdir /mnt/rhel8-install/
    # mount -o loop,ro -t iso9660 /path_directory/installer.iso /mnt/rhel8-install/

    Replace /path_directory/installer.iso with the path to the RHEL for Edge bootable ISO image.

  2. Copy the files from the mounted image to the HTTP server root. This command creates the /var/www/html/rhel8-install/ directory with the contents of the image.

    # mkdir /var/www/html/httpboot/
    # cp -R /mnt/rhel8-install/* /var/www/html/httpboot/
    # chmod -R +r /var/www/html/httpboot/*
    Note

    Some copying methods can skip the .treeinfo file which is required for a valid installation source. Running the cp command for whole directories as shown in this procedure will copy .treeinfo correctly.

  3. Update the /var/www/html/EFI/BOOT/grub.cfg file, by replacing:

    1. coreos.inst.install_dev=/dev/sda with coreos.inst.install_dev=/dev/vda
    2. linux /images/pxeboot/vmlinuz with linuxefi /images/pxeboot/vmlinuz
    3. initrd /images/pxeboot/initrd.img with initrdefi /images/pxeboot/initrd.img
    4. coreos.inst.image_file=/run/media/iso/disk.img.xz with coreos.inst.image_url=http://{IP-ADDRESS}/disk.img.xz

      The IP-ADDRESS is the ip address of this machine, which will serve as a http boot server.

  4. Start the httpd service:

    # systemctl start httpd.service

    As a result, after you set up an UEFI HTTP Boot server, you can install your RHEL for Edge devices by using UEFI HTTP boot.

6.9. Deploying the Simplified ISO image in a Virtual Machine

Deploy the RHEL for Edge ISO image you generated by creating a RHEL for Edge Simplified image by using any the following installation sources:

  • UEFI HTTP Boot
  • virt-install

This example shows how to create a virt-install installation source from your ISO image for a network-based installation .

Prerequisites

  • You have created an ISO image.
  • You set up a network configuration to support UEFI HTTP boot.

Procedure

  1. Set up a network configuration to support UEFI HTTP boot. See Setting up UEFI HTTP boot with libvirt.
  2. Use the virt-install command to create a RHEL for Edge Virtual Machine from the UEFI HTTP Boot.

    # virt-install \
        --name edge-install-image \
        --disk path=”  “, ,format=qcow2
        --ram 3072 \
        --memory 4096 \
        --vcpus 2 \
        --network network=integration,mac=mac_address \
        --os-type linux
        --os-variant rhel8 \
        --cdrom "/var/lib/libvirt/images/”ISO_FILENAME"
        --boot uefi,loader_ro=yes,loader_type=pflash,nvram_template=/usr/share/edk2/ovmf/OVMF_VARS.fd,loader_secure=no
        --virt-type kvm \
        --graphics none \
         --wait=-1
         --noreboot

After you run the command, the Virtual Machine installation starts.

Verification

  • Log in to the created Virtual Machine.

6.10. Deploying the Simplified ISO image from a USB flash drive

Deploy the RHEL for Edge ISO image you generated by creating a RHEL for Edge Simplified image by using an USB installation.

This example shows how to create a USB installation source from your ISO image.

Prerequisites

  • You have created a simplified installer image, which is an ISO image.
  • You have a 8 GB USB flash drive.

Procedure

  1. Copy the ISO image file to a USB flash drive.
  2. Connect the USB flash drive to the port of the computer you want to boot.
  3. Boot the ISO image from the USB flash drive.The boot menu shows you the following options:

    Install Red Hat Enterprise Linux 8
    Test this media & install Red Hat Enterprise Linux 8
  4. Choose Install Red Hat Enterprise Linux 8. This starts the system installation.

Additional resources

Chapter 7. Automatically provisioning and onboarding RHEL for Edge devices with FDO

You can build a RHEL for Edge Simplified Installer image, and provision it to a RHEL for Edge image. The FIDO Device Onboarding (FDO) process automatically provisions and onboards your Edge devices, and exchanges data with other devices and systems connected on the networks.

Important

Red Hat provides the FDO process as a Technology Preview feature and should run on secure networks. Technology Preview features are not supported with Red Hat production service level agreements (SLAs) and might not be functionally complete. These features provide early access to upcoming product features, enabling customers to test functionality and provide feedback during the development process. See Technology Preview Features Support Scope on the Red Hat Customer Portal for information about the support scope for Technology Preview features.

7.1. The FIDO Device Onboarding (FDO) process

The FIDO Device Onboarding (FDO) is the process that:

  • Provisions and onboards a device.
  • Automatically configures credentials for this device. The FDO process is an automatic onboarding mechanism that is triggered by the installation of a new device.
  • Enables this device to securely connect and interact on the network.

With FIDO Device Onboarding (FDO), you can perform a secure device onboarding by adding new devices into your IoT architecture. This includes the specified device configuration that needs to be trusted and integrated with the rest of the running systems. The FDO process is an automatic onboarding mechanism that is triggered by the installation of a new device.

The FDO protocol performs the following tasks:

  • Solves the trust and chain of ownership along with the automation needed to securely onboard a device at scale.
  • Performs device initialization at the manufacturing stage and late device binding for its actual use. This means that actual binding of the device to a management system happens on the first boot of the device without requiring manual configuration on the device.
  • Supports automated secure devices onboarding, that is, zero touch installation and onboarding that does not need any specialized person at the edge location. After the device is onboarded, the management platform can connect to it and apply patches, updates, and rollbacks.

With FDO, you can benefit from the following:

  • FDO is a secure and simple way to enroll a device to a management platform. Instead of embedding a Kickstart configuration to the image, FDO applies the device credentials during the device first boot directly to the ISO image.
  • FDO solves the issue of late binding to a device, enabling any sensitive data to be shared over a secure FDO channel.
  • FDO cryptographically identifies the system identity and ownership before enrolling and passing the configuration and other secrets to the system. That enables non-technical users to power-on the system.

To build a RHEL for Edge Simplified Installer image and automatically onboard it, provide an existing OSTree commit. The resulting simplified image contains a raw image that has the OSTree commit deployed. After you boot the Simplified installer ISO image, it provisions a RHEL for Edge system that you can use on a hard disk or as a boot image in a virtual machine.

The RHEL for Edge Simplified Installer image is optimized for unattended installation to a device and supports both network-base deployment and non-network-based deployments. However, for network-based deployment, it supports only UEFI HTTP boot.

The FDO protocol is based on the following servers:

Manufacturing server
  1. Generates the device credentials.
  2. Creates an Ownership voucher that is used to set the ownership of the device, later in the process.
  3. Binds the device to a specific management platform.
Owner management system
  1. Receives the Ownership voucher from the Manufacturing server and becomes the owner of the associated device.
  2. Later in the process, it creates a secure channel between the device and the Owner onboarding server after the device authentication.
  3. Uses the secure channel to send the required information, such as files and scripts for the onboarding automation to the device.
Service-info API server
Based on Service-info API server’s configuration and modules available on the client, it performs the final steps of onboarding on target client devices, such as copying SSH keys and files, executing commands, creating users, encrypting disks and so on
Rendezvous server
  1. Gets the Ownership voucher from the Owner management system and makes a mapping of the device UUID to the Owner server IP. Then, the Rendezvous server matches the device UUID with a target platform and informs the device about which Owner onboarding server endpoint this device must use.
  2. During the first boot, the Rendezvous server will be the contact point for the device and it will direct the device to the owner, so that the device and the owner can establish a secure channel.
Device client

This is installed on the device. The Device client performs the following actions:

  1. Starts the queries to the multiple servers where the onboarding automation will be executed.
  2. Uses TCP/IP protocols to communicate with the servers.

The following diagram represents the FIDO device onboarding workflow:

Figure 7.1. Deploying RHEL for Edge in non-network environment

FDO device onboarding

At the Device Initialization, the device contacts the Manufacturing server to get the FDO credentials, a set of certificates and keys to be installed on the operating system with the Rendezvous server endpoint (URL). It also gets the Ownership Voucher, that is maintained separately in case you need to change the owner assignment.

  1. The Device contacts the Manufacturing server
  2. The Manufacturing server generates an Ownership Voucher and the Device Credentials for the Device.
  3. The Ownership Voucher is transferred to the Owner onboarding server.

At the On-site onboarding, the Device gets the Rendezvous server endpoint (URL) from its device credentials and contacts Rendezvous server endpoint to start the onboarding process, which will redirect it to the Owner management system, that is formed by the Owner onboarding server and the Service Info API server.

  1. The Owner onboarding server transfers the Ownership Voucher to the Rendezvous server, which makes a mapping of the Ownership Voucher to the Owner.
  2. The device client reads device credentials.
  3. The device client connects to the network.
  4. After connecting to the network, the Device client contacts the Rendezvous server.
  5. The Rendezvous server sends the owner endpoint URL to the Device Client, and registers the device.
  6. The Device client connects to the Owner onboarding server shared by the Rendezvous server.
  7. The Device proves that it is the correct device by signing a statement with a device key.
  8. The Owner onboarding server proves itself correct by signing a statement with the last key of the Owner Voucher.
  9. The Owner onboarding server transfers the information of the Device to the Service Info API server.
  10. The Service info API server sends the configuration for the Device.
  11. The Device is onboarded.

7.2. Automatically provisioning and onboarding RHEL for Edge devices

To build a RHEL for Edge Simplified Installer image and automatically onboard it, provide an existing OSTree commit. The resulting simplified image contains a raw image that has the OSTree commit deployed. After you boot the Simplified installer ISO image, it provisions a RHEL for Edge system that you can use on a hard disk or as a boot image in a virtual machine.

The RHEL for Edge Simplified Installer image is optimized for unattended installation to a device and supports both network-base deployment and non-network-based deployments. However, for network-based deployment, it supports only UEFI HTTP boot.

Automatically provisioning and onboarding a RHEL for Edge device involves the following high-level steps:

  1. Install and register a RHEL system
  2. Install image builder
  3. Using image builder, create a blueprint with customizations for RHEL for a rhel-edge-container image type.

    name = "rhel-edge-container"
    description = "Minimal RHEL for Edge Container blueprint"
    version = "0.0.1"
  4. Import the RHEL for Edge Container blueprint in image builder
  5. Create a RHEL for Edge Container image
  6. Use the RHEL for Edge Container image to serve the OSTree commit, which will be later used when building the RHEL for Edge Simplified Installer image type
  7. Create a blueprint for and edge-simplified-installer image type with customizations for storage device path and FDO customizations

    name = "rhel-edge-simplified-installer-with-fdo"
    description = "Minimal RHEL for Edge Simplified Installer with FDO blueprint"
    version = "0.0.1"
    packages = []
    modules = []
    groups = []
    distro = ""
    
    [customizations]
    installation_device = "/dev/vda"
    
    [customizations.fdo]
    manufacturing_server_url = "http://10.0.0.2:8080"
    diun_pub_key_insecure = "true"
  8. Build a simplified installer RHEL for Edge image
  9. Download the RHEL for Edge simplified installer image
  10. At this point, the FDO server infrastructure should be up and running, and the specific onboarding details handled by the service-info API server, that is part of the owner’s infrastructure, are configured
  11. Install the simplified installer ISO image to a device. The FDO client runs on the Simplified Installer ISO and the UEFI directory structure makes the image bootable.
  12. The network configuration enables the device to reach out to the manufacturing server to perform the initial device credential exchange.
  13. After the system reaches the endpoint, the device credentials are created for the device.
  14. The device uses the device credentials to reach the Rendezvous server, where it checks the cryptographic credentials based on the vouchers that the Rendezvous server has, and then the Rendezvous server redirects the device to the Owner server.
  15. The device contacts the Owner server. They establish a mutual trust and the final steps of onboarding happen based on the configuration of the Service-info API server. For example, it installs the SSH keys in the device, transfer the files, create the users, run the commands, encrypt the filesystem, and so on.

7.3. Generating key and certificates

To run the FIDO Device Onboarding (FDO) infrastructure, you need to generate keys and certificates. FDO generates these keys and certificates to configure the manufacturing server. FDO automatically generates the certificates and .yaml configuration files when you install the services, and re-creating them is optional. After you install and start the services, it runs with the default settings.

Important

Red Hat provides the fdo-admin-tool tool as a Technology Preview feature and should run on secure networks. Technology Preview features are not supported with Red Hat production service level agreements (SLAs) and might not be functionally complete. These features provide early access to upcoming product features, enabling customers to test functionality and provide feedback during the development process. See Technology Preview Features Support Scope on the Red Hat Customer Portal for information about the support scope for Technology Preview features.

Prerequisites

  • You installed the fdo-admin-cli RPM package

Procedure

  1. Create a directory for the keys and certificates:

    $ mkdir /etc/fdo/keys
  2. Generate the keys and certificates in the /etc/fdo directory:

    $ for i in "diun" "manufacturer" "device-ca" "owner"; do fdo-admin-tool generate-key-and-cert $i; done
    $ ls keys
    device_ca_cert.pem device_ca_key.der diun_cert.pem diun_key.der manufacturer_cert.pem manufacturer_key.der owner_cert.pem owner_key.der
  3. Check the key and certificates that were created in the /etc/fdo/keys directory:

    $ tree keys

    You can see the following output:

    – device_ca_cert.pem
    – device_ca_key.der
    – diun_cert.pem
    – diun_key.dre
    – manufacturer_cert.pem
    – manufacturer_key.der
    – owner_cert.pem
    – owner_key.pem

Additional resources

  • See the fdo-admin-tool generate-key-and-cert –-help manual page

7.4. Installing and running the manufacturing server

The fdo-manufacturing-server RPM package enables you to run the Manufacturing Server component of the FDO protocol. It also stores other components, such as the owner vouchers, the manufacturer keys, and information about the manufacturing sessions. During the device installation, the Manufacturing server generates the device credentials for the specific device, including its GUID, rendezvous information and other metadata. Later on in the process, the device uses this rendezvous information to contact the Rendezvous server.

Important

Red Hat provides the fdo-manufacturing-server tool as a Technology Preview feature and should run on secure networks because Technology Preview features are not supported with Red Hat production service level agreements (SLAs) and might not be functionally complete. These features provide early access to upcoming product features, enabling customers to test functionality and provide feedback during the development process. See Technology Preview Features Support Scope on the Red Hat Customer Portal for information about the support scope for Technology Preview features.

To install the manufacturing server RPM package, complete the following steps:

Procedure

  1. Install the fdo-admin-cli package:

    # yum install -y fdo-admin-cli
  2. Check if the fdo-manufacturing-server RPM package is installed:

    $ rpm -qa | grep fdo-manufacturing-server --refresh
  3. Check if the files were correctly installed:

    $ ls /usr/share/doc/fdo

    You can see the following output:

    Output:
    manufacturing-server.yml
    owner-onboarding-server.yml
    rendezvous-info.yml
    rendezvous-server.yml
    serviceinfo-api-server.yml
  4. Optional: Check the content of each file, for example:

    $ cat /usr/share/doc/fdo/manufacturing-server.yml
  5. Create a directory for each of the components: owner vouchers, manufacturer keys, and manufacturing sessions.

    # mkdir -p /etc/fdo/stores/manufacturer_keys
    # mkdir -p /etc/fdo/stores/manufacturing_sessions
    # mkdir -p /etc/fdo/stores/owner_vouchers
  6. Configure the Manufacturing server. You must provide the following information:

    • The Manufacturing server URL
    • The IP address or DNS name for the Rendezvous server
    • The path to the keys and certificates you generated. See Generating key and certificates.

      You can find an example of a Manufacturing server configuration file in the /usr/share/doc/fdo/manufacturing-server.yml directory. The following is a manufacturing server.yml example that is created and saved in the /etc/fdo directory. It contains paths to the directories, certificates, keys that you created, the Rendezvous server IP address and the default port.

      session_store_driver:
        Directory:
          path: /etc/fdo/stores/manufacturing_sessions/
      ownership_voucher_store_driver:
        Directory:
          path: /etc/fdo/stores/owner_vouchers
      public_key_store_driver:
        Directory:
          path: /etc/fdo/stores/manufacturer_keys
      bind: "0.0.0.0:8080"
      protocols:
        plain_di: false
        diun:
          mfg_string_type: SerialNumber
          key_type: SECP384R1
          allowed_key_storage_types:
            - Tpm
            - FileSystem
          key_path: /etc/fdo/keys/diun_key.der
          cert_path: /etc/fdo/keys/diun_cert.pem
      rendezvous_info:
        - deviceport: 8082
          ip_address: 192.168.122.99
          ownerport: 8082
          protocol: http
      manufacturing:
        manufacturer_cert_path: /etc/fdo/keys/manufacturer_cert.pem
        device_cert_ca_private_key: /etc/fdo/keys/device_ca_key.der
        device_cert_ca_chain: /etc/fdo/keys/device_ca_cert.pem
        owner_cert_path: /etc/fdo/keys/owner_cert.pem
        manufacturer_private_key: /etc/fdo/keys/manufacturer_key.der
  7. Start the Manufacturing server.

    1. Check if the systemd unit file are in the server:

      # systemctl list-unit-files | grep fdo | grep manufacturing
      fdo-manufacturing-server.service disabled disabled
    2. Enable and start the manufacturing server.

      # systemctl enable --now fdo-manufacturing-server.service
    3. Open the default ports in your firewall:

      # firewall-cmd --add-port=8080/tcp --permanent
      # systemctl restart firewalld
    4. Ensure that the service is listening on the port 8080:

      # ss -ltn
  8. Install RHEL for Edge onto your system using the simplified installer. See Building simplified installer images to provision a RHEL for Edge image.

7.5. Installing, configuring, and running the Rendezvous server

Install the fdo-rendezvous-server RPM package to enable the systems to receive the voucher generated by the Manufacturing server during the first device boot. The Rendezvous server then matches the device UUID with the target platform or cloud and informs the device about which Owner server endpoint the device must use.

Prerequisites

  • You created a manufacturer_cert.pem certificate. See Generating key and certificates.
  • You copied the manufacturer_cert.pem certificate to the /etc/fdo/keys directory in the Rendezvous server.

Procedure

  1. Install the fdo-rendezvous-server RPM packages:

    # yum install -y fdo-rendezvous-server
  2. Create the directories to store the following directories: the registered directory, where the registered device owner vouchers will be hosted and the sessions directory:

    # mkdir -p /etc/fdo/stores/rendezvous_registered
    # mkdir -p /etc/fdo/stores/rendezvous_sessions
  3. Create the rendezvous-server.yml configuration file, including the path to the manufacturer certificate. You can find an example in /usr/share/doc/fdo/rendezvous-server.yml. The following example shows a configuration file that is saved in /etc/fdo/rendezvous-server.yml.

    storage_driver:
      Directory:
        path: /etc/fdo/stores/rendezvous_registered
    session_store_driver:
      Directory:
        path: /etc/fdo/stores/rendezvous_sessions
    trusted_manufacturer_keys_path: /etc/fdo/keys/manufacturer_cert.pem
    max_wait_seconds: ~
    bind: "0.0.0.0:8082"
  4. Check the Rendezvous server service status:

    # systemctl list-unit-files | grep fdo | grep rende
    fdo-rendezvous-server.service disabled disabled
    1. If the service is stopped and disabled, enable and start it:

      # systemctl enable --now fdo-rendezvous-server.service
  5. Check that the server is listening on the default configured port 8082:

    # ss -ltn
  6. Open the port if you have a firewall configured on this server:

    # firewall-cmd --add-port=8082/tcp --permanent
    # systemctl restart firewalld

7.6. Installing, configuring, and running the Owner server

Install the fdo-owner-cli and fdo-owner-onboarding-server RPM package to enable the systems to receive the voucher generated by the Manufacturing server during the first device boot. The Rendezvous server then matches the device UUID with the target platform or cloud and informs the device about which Owner server endpoint the device must use.

Prerequisites

  • The device where the server will be deployed has a Trusted Platform Module (TPM) device to encrypt the disk. If not, you will get an error when booting the RHEL for Edge device.
  • You created the device_ca_cert.pem, owner_key.der, and owner_cert.pem with keys and certificates and copied them into the /etc/fdo/keys directory.

Procedure

  1. Install the required RPMs in this server:

    # dnf install -y fdo-owner-cli fdo-owner-onboarding-server
  2. Create the additional directories that the Owner server service needs:

    # mkdir -p /etc/fdo/stores/owner_vouchers
    # mkdir -p /etc/fdo/stores/owner_onboarding_sessions
  3. Prepare the owner-onboarding-server.yml configuration file and save it to the /etc/fdo/ directory. Include the path to the certificates you already copied and information about where to publish the Owner server service in this file.

    The following is an example available in /usr/share/doc/fdo/owner-onboarding-server.yml. You can find references to the Service Info API, such as the URL or the authentication token.

    ---
    ownership_voucher_store_driver:
      Directory:
        path: /etc/fdo/stores/owner_vouchers
    session_store_driver:
      Directory:
        path: /etc/fdo/stores/owner_onboarding_sessions
    trusted_device_keys_path: /etc/fdo/keys/device_ca_cert.pem
    owner_private_key_path: /etc/fdo/keys/owner_key.der
    owner_public_key_path: /etc/fdo/keys/owner_cert.pem
    bind: "0.0.0.0:8081"
    service_info_api_url: "http://localhost:8083/device_info"
    service_info_api_authentication:
      BearerToken:
        token: Kpt5P/5flBkaiNSvDYS3cEdBQXJn2Zv9n1D50431/lo=
    owner_addresses:
      - transport: http
        addresses:
          - ip_address: 192.168.122.149
  4. Create and configure the Service Info API.

    1. Add the automated information for onboarding, such as user creation, files to be copied or created, commands to be executed, disk to be encrypted, and so on. Use the Service Info API configuration file example in /usr/share/doc/fdo/serviceinfo-api-server.yml as a template to create the configuration file under /etc/fdo/.

      ---
      service_info:
        initial_user:
          username: admin
          sshkeys:
          - "ssh-rsa AAAA...."
        files:
        - path: /root/resolv.conf
          source_path: /etc/resolv.conf
        commands:
        - command: touch
          args:
          - /root/test
          return_stdout: true
          return_stderr: true
        diskencryption_clevis:
        - disk_label: /dev/vda4
          binding:
            pin: tpm2
            config: "{}"
          reencrypt: true
        additional_serviceinfo: ~
      bind: "0.0.0.0:8083"
      device_specific_store_driver:
        Directory:
          path: /etc/fdo/stores/serviceinfo_api_devices
      service_info_auth_token: Kpt5P/5flBkaiNSvDYS3cEdBQXJn2Zv9n1D50431/lo=
      admin_auth_token: zJNoErq7aa0RusJ1w0tkTjdITdMCWYkndzVv7F0V42Q=
  5. Check the status of the systemd units:

    # systemctl list-unit-files | grep fdo
    fdo-owner-onboarding-server.service        disabled        disabled
    fdo-serviceinfo-api-server.service         disabled        disabled
    1. If the service is stopped and disabled, enable and start it:

      # systemctl enable --now fdo-owner-onboarding-server.service
      # systemctl enable --now fdo-serviceinfo-api-server.service
      Note

      You must restart the systemd services every time you change the configuration files.

  6. Check that the server is listening on the default configured port 8083:

    # ss -ltn
  7. Open the port if you have a firewall configured on this server:

    # firewall-cmd --add-port=8081/tcp --permanent
    # firewall-cmd --add-port=8083/tcp --permanent
    # systemctl restart firewalld

7.7. Automatically onboarding a RHEL for Edge device by using FDO authentication

To prepare your device to automatically onboard a RHEL for Edge device and provision it as part of the installation process, complete the following steps:

Prerequisites

  • You built an OSTree commit for RHEL for Edge and used that to generate an edge-simplified-installer artifact.
  • Your device is assembled.
  • You installed the fdo-manufacturing-server RPM package. See Installing the manufacturing server package.

Procedure

  1. Start the installation process by booting the RHEL for Edge simplified installer image on your device. You can install it from a CD-ROM or from a USB flash drive, for example.
  2. Verify through the terminal that the device has reached the manufacturing service to perform the initial device credential exchange and has produced an ownership voucher.

    You can find the ownership voucher at the storage location configured by the ownership_voucher_store_driver: parameter at the manufacturing-sever.yml file.

    The directory should have an ownership_voucher file with a name in the GUID format which indicates that the correct device credentials were added to the device.

    The onboarding server uses the device credential to authenticate against the onboarding server. It then passes the configuration to the device. After the device receives the configuration from the onboarding server, it receives an SSH key and installs the operating system on the device. Finally, the system automatically reboots, encrypts it with a strong key stored at TPM.

Verification

After the device automatically reboots, you can log in to the device with the credentials that you created as part of the FDO process.

  1. Log in to the device by providing the username and password you created in the Service Info API.

Chapter 8. Deploying a RHEL for Edge image in a network-base environment

You can deploy a RHEL for Edge image using the RHEL installer graphical user interface or a Kickstart file. The overall process for deploying a RHEL for Edge image depends on whether your deployment environment is network-based or non-network-based.

Note

To deploy the images on bare metal, use a Kickstart file.

Network-based deployments

Deploying a RHEL for Edge image in a network-based environment involves the following high-level steps:

  1. Extract the image file contents.
  2. Set up a web server
  3. Install the image

8.1. Extracting the RHEL for Edge image commit

After you download the commit, extract the .tar file and note the ref name and the commit ID.

The downloaded commit file consists of a .tar file with an OSTree repository. The OSTree repository has a commit and a compose.json file.

The compose.json file has information metadata about the commit with information such as the "Ref", the reference ID and the commit ID. The commit ID has the RPM packages.

To extract the package contents, perform the following the steps:

Prerequisites

  • Create a Kickstart file or use an existing one.

Procedure

  1. Extract the downloaded image .tar file:

    # tar xvf <UUID>-commit.tar
  2. Go to the directory where you have extracted the .tar file.

    It has a compose.json file and an OSTree directory. The compose.json file has the commit number and the OSTree directory has the RPM packages.

  3. Open the compose.json file and note the commit ID number. You need this number handy when you proceed to set up a web server.

    If you have the jq JSON processor installed, you can also retrieve the commit ID by using the jq tool:

    # jq '.["ostree-commit"]' < compose.json
  4. List the RPM packages in the commit.

    # rpm-ostree db list rhel/8/x86_64/edge --repo=repo
  5. Use a Kickstart file to run the RHEL installer. Optionally, you can use any existing file or can create one by using the Kickstart Generator tool.

    In the Kickstart file, ensure that you include the details about how to provision the file system, create a user, and how to fetch and deploy the RHEL for Edge image. The RHEL installer uses this information during the installation process.

    The following is a Kickstart file example:

    lang en_US.UTF-8
    keyboard us
    timezone Etc/UTC --isUtc
    text
    zerombr
    clearpart --all --initlabel
    autopart
    reboot
    user --name=core --group=wheel
    sshkey --username=core "ssh-rsa AAAA3Nza…​."
    rootpw --lock
    network --bootproto=dhcp
    
    ostreesetup --nogpg --osname=rhel --remote=edge --url=https://mirror.example.com/repo/ --ref=rhel/8/x86_64/edge

    The OStree-based installation uses the ostreesetup command to set up the configuration. It fetches the OSTree commit, by using the following flags:

    • --nogpg - Disable GNU Privacy Guard (GPG) key verification.
    • --osname - Management root for the operating system installation.
    • --remote - Management root for the operating system installation
    • --url - URL of the repository to install from.
    • --ref - Name of the branch from the repository that the installation uses.
    • --url=http://mirror.example.com/repo/ - is the address of the host system where you extracted the edge commit and served it over nginx. You can use the address to reach the host system from the guest computer.

      For example, if you extract the commit image in the /var/www/html directory and serve the commit over nginx on a computer whose hostname is www.example.com, the value of the --url parameter is http://www.example.com/repo.

      Note

      Use the http protocol to start a service to serve the commit, because https is not enabled on the Apache HTTP Server.

8.2. Setting up a web server to install RHEL for Edge images

After you have extracted the RHEL for Edge image contents, set up a web server to provide the image commit details to the RHEL installer by using HTTP.

The following example provides the steps to set up a web server by using a container.

Prerequisites

Procedure

  1. Create the nginx configuration file with the following instructions:

    events {
    
    }
    
    http {
        server{
            listen 8080;
            root /usr/share/nginx/html;
                    }
             }
    
    pid /run/nginx.pid;
    daemon off;
  2. Create a Dockerfile with the following instructions:

    FROM registry.access.redhat.com/ubi8/ubi
    RUN yum -y install nginx && yum clean all
    COPY kickstart.ks /usr/share/nginx/html/
    COPY repo /usr/share/nginx/html/
    COPY nginx /etc/nginx.conf
    EXPOSE 8080
    CMD ["/usr/sbin/nginx", "-c", "/etc/nginx.conf"]
    ARG commit
    ADD ${commit} /usr/share/nginx/html/

    Where,

    • kickstart.ks is the name of the Kickstart file from the RHEL for Edge image. The Kickstart file includes directive information. To help you manage the images later, it is advisable to include the checks and settings for Greenboot checks. For that, you can update the Kickstart file to include the following settings:

      lang en_US.UTF-8
      keyboard us
      timezone Etc/UTC --isUtc
      text
      zerombr
      clearpart --all --initlabel
      autopart
      reboot
      user --name=core --group=wheel
      sshkey --username=core "ssh-rsa AAAA3Nza…​."
      
      ostreesetup --nogpg --osname=rhel --remote=edge
      --url=https://mirror.example.com/repo/
      --ref=rhel/8/x86_64/edge
      
      %post
      cat << EOF > /etc/greenboot/check/required.d/check-dns.sh
      #!/bin/bash
      
      DNS_SERVER=$(grep nameserver /etc/resolv.conf | cut -f2 -d" ")
      COUNT=0
      
      # check DNS server is available
      ping -c1 $DNS_SERVER
      while [ $? != '0' ] && [ $COUNT -lt 10 ]; do
      
      							
      							
      							
      							COUNT++
      echo "Checking for DNS: Attempt $COUNT ."
      sleep 10
      ping -c 1 $DNS_SERVER
      done
      EOF
      %end

      Any HTTP service can host the OSTree repository, and the example, which uses a container, is just an option for how to do this. The Dockerfile performs the following tasks:

      1. Uses the latest Universal Base Image (UBI)
      2. Installs the web server (nginx)
      3. Adds the Kickstart file to the server
      4. Adds the RHEL for Edge image commit to the server
  3. Build a Docker container

    # podman build -t name-of-container-image --build-arg commit=uuid-commit.tar .
  4. Run the container

    # podman run --rm -d -p port:8080 localhost/name-of-container-image

    As a result, the server is set up and ready to start the RHEL Installer by using the commit.tar repository and the Kickstart file.

8.3. Installing the RHEL for Edge image using a Kickstart file

To install the RHEL for Edge image that uses a Kickstart file, use the web server. The web server uses the RHEL for Edge image commit.tar repository and the Kickstart file to start the RHEL Installer.

Prerequisites

Procedure

  1. Run the RHEL Anaconda Installer by using libvirt virt-install:

    virt-install \
    --name rhel-edge-test-1 \
    --memory 2048 \
    --vcpus 2 \
    --disk path=prepared_disk_image.qcow2,format=qcow2,size=8 \
    --os-variant rhel8 \
    --cdrom /home/username/Downloads/rhel-8-x86_64-boot.iso
  2. On the installation screen:

    Figure 8.1. Red Hat Enterprise Linux boot menu

    The Red Hat Enterprise Linux boot menu
    1. Press the e key to add an additional kernel parameter:

      inst.ks=http://edge_device_ip:port/kickstart.ks

      The kernel parameter specifies that you want to install RHEL by using the Kickstart file and not the RHEL image contained in the RHEL Installer.

    2. After adding the kernel parameters, press Ctrl+X to boot the RHEL installation by using the Kickstart file.

      The RHEL Installer starts, fetches the Kickstart file from the server (HTTP) endpoint and executes the commands, including the command to install the RHEL for Edge image commit from the HTTP endpoint. After the installation completes, the RHEL Installer prompts you for login details.

Verification

  1. On the Login screen, enter your user account credentials and click Enter.
  2. Verify whether the RHEL for Edge image is successfully installed.

    $ rpm-ostree status

    The command output provides the image commit ID and shows that the installation is successful.

    Following is a sample output:

    State: idle
    Deployments:
    * ostree://edge:rhel/8/x86_64/edge
    		  Timestamp: 2020-09-18T20:06:54Z
    			Commit: 836e637095554e0b634a0a48ea05c75280519dd6576a392635e6fa7d4d5e96

Chapter 9. Deploying a RHEL for Edge image in a non-network-base environment

The RHEL for Edge Container (.tar) in combination with the RHEL for Edge Installer (.iso) image type result in a ISO image. The ISO image can be used in disconnected environments during the image deployment to a device. However, network access might require network access to build the different artifacts.

Deploying a RHEL for Edge image in a non-network-based environment involves the following high-level steps:

  1. Download the RHEL for Edge Container. See Downloading a RHEL for Edge image for information about how to download the RHEL for Edge image.
  2. Load the RHEL for Edge Container image into Podman
  3. Run the RHEL for Edge Container image into Podman
  4. Load the RHEL for Edge Installer blueprint
  5. Build the RHEL for Edge Installer image
  6. Prepare a .qcow2 disk
  7. Boot the Virtual Machine (VM)
  8. Install the image

9.1. Creating a RHEL for Edge Container image for non-network-based deployments

You can build a running container by loading the downloaded RHEL for Edge Container OSTree commit into Podman. For that, follow the steps:

Prerequisites

Procedure

  1. Navigate to the directory where you have downloaded the RHEL for Edge Container OSTree commit.
  2. Load the RHEL for Edge Container OSTree commit into Podman.

    $ sudo podman load -i UUID-container.tar

    The command output provides the image ID, for example: @8e0d51f061ff1a51d157804362bc875b649b27f2ae1e66566a15e7e6530cec63

  3. Tag the new RHEL for Edge Container image, using the image ID generated by the previous step.

    $ sudo podman tag image-ID localhost/edge-container

    The podman tag command assigns an additional name to the local image.

  4. Run the container named edge-container.

    $ sudo podman run -d --name=edge-container -p 8080:8080 localhost/edge-container

    The podman run -d --name=edge-container command assigns a name to your container-based on the localhost/edge-container image.

  5. List containers:

    $ sudo podman ps -a
    CONTAINER ID  IMAGE                               	COMMAND	CREATED    	STATUS                	PORTS   NAMES
    2988198c4c4b  …./localhost/edge-container   /bin/bash  3 seconds ago  Up 2 seconds ago      	edge-container

As a result, Podman runs a container that serves an OSTree repository with the RHEL for Edge Container commit.

9.2. Creating a RHEL for Edge Installer image for non-network-based deployments

After you have built a running container to serve a repository with the RHEL for Edge Container commit, create an RHEL for Edge Installer (.iso) image. The RHEL for Edge Installer (.iso) pulls the commit served by RHEL for Edge Container (.tar). After the RHEL for Edge Container commit is loaded in Podman, it exposes the OSTree in the URL format.

To create the RHEL for Edge Installer image in the CLI, follow the steps:

Prerequisites

  • You created a blueprint for RHEL for Edge image.
  • You created a RHEL for Edge Edge Container image and deployed it using a web server.

Procedure

  1. Begin to create the RHEL for Edge Installer image.

    # composer-cli compose start-ostree --ref rhel/8/x86_64/edge --url URL-OSTree-repository blueprint-name image-type

    Where,

    • ref is the same value that customer used to build ostree repository
    • URL-OSTree-repository is the URL to the OSTree repository of the commit to embed in the image. For example, http://10.0.2.2:8080/repo/. See Creating a RHEL for Edge Container image for non-network-based deployments.
    • blueprint-name is the RHEL for Edge Installer blueprint name.
    • image-type is edge-installer.

      A confirmation that the composer process has been added to the queue appears. It also shows a Universally Unique Identifier (UUID) number for the image created. Use the UUID number to track your build. Also keep the UUID number handy for further tasks.

  2. Check the image compose status.

    # composer-cli compose status

    The command output displays the status in the following format:

    <UUID> RUNNING date blueprint-name blueprint-version image-type
    Note

    The image creation process takes a few minutes to complete.

    To interrupt the image creation process, run:

    # composer-cli compose cancel <UUID>

    To delete an existing image, run:

    # composer-cli compose delete <UUID>

    Image builder pulls the commit that is being served by the running container during the image build.

    After the image build is complete, you can download the resulting ISO image.

  3. Download the image. See Downloading a RHEL for Edge image.

    After the image is ready, you can use it for non-network deployments. See Installing the RHEL for Edge image for non-network-based deployments.

9.3. Installing the RHEL for Edge image for non-network-based deployments

To install the RHEL for Edge image, follow the steps:

Prerequisites

  • You created a RHEL for Edge Installer ISO image.
  • You stopped the running container.
  • A disk image to install the commit you created.
  • You installed the edk2-ovmf package.
  • You installed the virt-viewer package.
  • You customized your blueprint with a user account. See Creating a blueprint for a RHEL for Edge image using image builder in RHEL web console.

    Warning

    If you do not define a user account customization in your blueprint, you will not be able to login to the ISO image.

Procedure

  1. Create a qcow VM disk file to install the (.iso) image. That is an image of a hard disk drive for the virtual machine (VM). For example:

    $ qemu-img create -f qcow2 diskfile.qcow2 20G
  2. Use the virt-install command to boot the VM using the disk as a drive and the installer ISO image as a CD-ROM. For example:

    $ virt-install \
    --boot uefi \
    --name VM_NAME
    --memory 2048 \
    --vcpus 2 \
    --disk path=diskfile.qcow2
    --cdrom /var/lib/libvirt/images/UUID-installer.iso \
    --os-variant rhel9.0

    This command instructs virt-install to:

    • Instructs the VM to use UEFI to boot, instead of the BIOS.
    • Mount the installation ISO.
    • Use the hard disk drive image created in the first step.

      It gives you an Anaconda Installer. The RHEL Installer starts, loads the Kickstart file from the ISO and executes the commands, including the command to install the RHEL for Edge image commit. Once the installation is complete, the RHEL installer prompts for login details.

      Note

      Anaconda is preconfigured to use the Container commit during the installation. However, you need to set up system configurations, such as disk partition, timezone, between others.

  3. Connect to Anaconda GUI with virt-viewer to setup the system configuration:

    $ virt-viewer --connect qemu:///system --wait VM_NAME
  4. Reboot the system to finish the installation.
  5. On the login screen, specify your user account credentials and click Enter.

Verification steps

  1. Verify whether the RHEL for Edge image is successfully installed.

    $  rpm-ostree status

The command output provides the image commit ID and shows that the installation is successful.

Chapter 10. Managing RHEL for Edge images

To manage the RHEL for Edge images, you can perform any of the following administrative tasks:

  • Edit the RHEL for Edge image blueprint by using image builder in RHEL web console or in the command-line
  • Build a commit update by using image builder command-line
  • Update the RHEL for Edge images
  • Configure rpm-ostree remotes on nodes, to update node policy
  • Restore RHEL for Edge images manually or automatically by using greenboot

10.1. Editing a RHEL for Edge image blueprint by using image builder

You can edit the RHEL for Edge image blueprint to:

  • Add additional components that you might require
  • Modify the version of any existing component
  • Remove any existing component

10.1.1. Adding a component to RHEL for Edge image blueprint using image builder in RHEL web console

To add a component to a RHEL for Edge image blueprint, ensure that you have met the following prerequisites and then follow the procedure to edit the corresponding blueprint.

Prerequisites

  • On a RHEL system, you have accessed the image builder dashboard.
  • You have created a blueprint for RHEL for Edge image.

Procedure

  1. On the image builder dashboard, click the blueprint that you want to edit.

    To search for a specific blueprint, enter the blueprint name in the filter text box, and click Enter.

  2. On the upper right side of the blueprint, click Edit Packages.

    The Edit blueprints wizard opens.

  3. On the Details page, update the blueprint name and click Next.
  4. On the Packages page, follow the steps:

    1. In the Available Packages, enter the package name that you want to add in the filter text box, and click Enter.

      A list with the component name appears.

    2. Click > to add the component to the blueprint.
  5. On the Review page, click Save.

    The blueprint is now updated with the new package.

10.1.2. Removing a component from RHEL for Edge image blueprint using image builder in RHEL web console

To remove one or more unwanted components from a RHEL for Edge image blueprint that you created, ensure that you have met the following prerequisites and then follow the procedure.

Prerequisites

  • On a RHEL system, you have accessed the image builder dashboard.
  • You have created a blueprint for RHEL for Edge image.
  • You have added at least one component to the RHEL for Edge blueprint.

Procedure

  1. On the image builder dashboard, click the blueprint that you want to edit.

    To search for a specific blueprint, enter the blueprint name in the filter text box, and click Enter.

  2. On the upper right side of the blueprint, click Edit Packages.

    The Edit blueprints wizard opens.

  3. On the Details page, update the blueprint name and click Next.
  4. On the Packages page, follow the steps:

    1. From the Chosen packages, click < to remove the chosen component. You can also click << to remove all the packages at once.
  5. On the Review page, click Save.

    The blueprint is now updated.

10.1.3. Editing a RHEL for Edge image blueprint using command-line interface

You can change the specifications for your RHEL for Edge image blueprint using image builder command-line. To do so, ensure that you have met the following prerequisites and then follow the procedure to edit the corresponding blueprint.

Prerequisites

  • You have access to the image builder command-line.
  • You have created a RHEL for Edge image blueprint.

Procedure

  1. Save (export) the blueprint to a local text file:

    # composer-cli blueprints save BLUEPRINT-NAME
  2. Edit the BLUEPRINT-NAME.toml file with a text editor of your choice and make your changes.

    Before finishing with the edits, verify that the file is a valid blueprint:

  3. Increase the version number.

    Ensure that you use a Semantic Versioning scheme.

    Note

    if you do not change the version, the patch component of the version is increased automatically.

  4. Check if the contents are valid TOML specifications. See the TOML documentation for more information.

    Note

    TOML documentation is a community product and is not supported by Red Hat. You can report any issues with the tool at https://github.com/toml-lang/toml/issues.

  5. Save the file and close the editor.
  6. Push (import) the blueprint back into image builder command-line:

    # composer-cli blueprints push BLUEPRINT-NAME.toml
    Note

    When pushing the blueprint back into the image builder command-line, provide the file name including the .toml extension.

  7. Verify that the contents uploaded to image builder match your edits:

    # composer-cli blueprints show BLUEPRINT-NAME
  8. Check whether the components and versions listed in the blueprint and their dependencies are valid:

    # composer-cli blueprints depsolve BLUEPRINT-NAME

10.2. Updating RHEL for Edge images

10.2.1. How RHEL for Edge image updates are deployed

With RHEL for Edge images, you can either deploy the updates manually or can automate the deployment process. The updates are applied in an atomic manner, where the state of each update is known, and the updates are staged and applied only upon reboot. Because no changes are seen until you reboot the device, you can schedule a reboot to ensure the highest possible uptime.

During the image update, only the updated operating system content is transferred over the network. This makes the deployment process more efficient compared to transferring the entire image. The operating system binaries and libraries in /usr are read-only, and the read and write state is maintained in /var and /etc directories.

When moving to a new deployment, the /etc and the /var directories are copied to the new deployment with read and write permissions. The /usr directory is copied as a soft link to the new deployment directory, with read-only permissions.

The following diagram illustrates the RHEL for Edge image update deployment process:

Image Deployment

By default, the new system is booted using a procedure similar to a chroot operation, that is, the system enables control access to a filesystem while controls the exposure to the underlying server environment. The new /sysroot directory mainly has the following parts:

  • Repository database at the /sysroot/ostree/repo directory.
  • File system revisions at the /sysroot/ostree/deploy/rhel/deploy directory, which are created by each operation in the system update.
  • The /sysroot/ostree/boot directory, which links to deployments on the previous point. Note that /ostree is a soft link to /sysroot/ostree. The files from the /sysroot/ostree/boot directory are not duplicated. The same file is used if it is not changed during the deployment. The files are hard-links to another file stored in the /sysroot/ostree/repo/objects directory.

The operating system selects the deployment in the following way:

  1. The dracut tool parses the ostree kernel argument in the initramfs root file system and sets up the /usr directory as a read-only bind mount.
  2. Bind the deployment directory in /sysroot to / directory.
  3. Re-mount the operating system already mounted dirs using the MS_MOVE mount flag

If anything goes wrong, you can perform a deployment rollback by removing the old deployments with the rpm-ostree cleanup command. Each client machine contains an OSTree repository stored in /ostree/repo, and a set of deployments stored in /ostree/deploy/$STATEROOT/$CHECKSUM.

With the deployment updates in RHEL for Edge image, you can benefit from a better system consistency across multiple devices, easier reproducibility, and better isolation between the pre and post system states change.

10.2.2. Building a commit update

You can build a commit update after making a change in the blueprint, such as:

  • Adding an additional package that your system requires
  • Modifying the package version of any existing component
  • Removing any existing package.

Prerequisites

Procedure

  1. Start the compose of the new commit image, with the following arguments: --url, --ref, blueprint-name, edge-commit.

    # composer-cli compose start-ostree --ref rhel/8/x86_64/edge --url http://localhost:8080/repo <blueprint-name> edge-commit

    The command instructs the compose process to fetch the metadata from the OStree repo before starting the compose. The resulting new OSTree commit contains a reference of the original OSTree commit as a parent image.

  2. After the compose process finishes, fetch the .tar file.

    # composer-cli compose image <UUID>
  3. Extract the commit to a temporary directory, so that you can store the commit history in the OSTree repo.

    $ tar -xf UUID.tar -C /var/tmp
  4. Inspect the resulting OSTree repo commit, by using the tar -xf command. It extracts the tar file to disk so you can inspect the resulting OSTree repo:

    $ ostree --repo=/var/tmp/repo log rhel/8/x86_64/edge
    commit d523ef801e8b1df69ddbf73ce810521b5c44e9127a379a4e3bba5889829546fa
    Parent:  f47842de7e6859cee07d743d3c67949420874727883fa9dbbaeb5824ad949d91
    ContentChecksum:  f0f6703696331b661fa22d97358db48ba5f8b62711d9db83a00a79b3ae0dfe16
    Date:  2023-06-04 20:22:28 /+0000
    Version: 8

    In the output example, there is a single OSTree commit in the repo that references a parent commit. The parent commit is the same checksum from the original OSTree commit that you previously made.

  5. Merge the two commits by using the ostree pull-local command:

    $ sudo ostree --repo=/var/srv/httpd/repo pull-local /var/tmp/repo
    20 metadata, 22 content objects imported; 0 bytes content written

    This command copies any new metadata and content from the location on the disk, for example, /var/tmp, to a destination OSTree repo in /var/srv/httpd.

Verification

  1. Inspect the target OSTree repo:

    $ ostree --repo=/var/srv/httpd/repo log rhel/8/x86_64/edge
    commit d523ef801e8b1df69ddbf73ce810521b5c44e9127a379a4e3bba5889829546fa
    Parent:  f47842de7e6859cee07d743d3c67949420874727883fa9dbbaeb5824ad949d91
    ContentChecksum:  f0f6703696331b661fa22d97358db48ba5f8b62711d9db83a00a79b3ae0dfe16
    Date:  2023-06-04 20:22:28 /+0000
    Version: 8
    (no subject)
    
    commit f47842de7e6859cee07d743d3c67949420874727883fa9dbbaeb5824ad949d91
    ContentChecksum:  9054de3fe5f1210e3e52b38955bea0510915f89971e3b1ba121e15559d5f3a63
    Date:  2023-06-04 20:01:08 /+0000
    Version: 8
    (no subject)

    You can see that the target OSTree repo now contains two commits in the repository, in a logical order. After successful verification, you can update your RHEL for Edge system.

10.2.3. Deploying RHEL for Edge image updates manually

After you have edited a RHEL for Edge blueprint, you can update the image commit. Image builder generates a new commit for the updated RHEL for Edge image. Use this new commit to deploy the image with latest package versions or with additional packages.

To deploy RHEL for Edge images updates, ensure that you meet the prerequisites and then follow the procedure.

Prerequisites

  • On a RHEL system, you have accessed the image builder dashboard.
  • You have created a RHEL for Edge image blueprint.
  • You have edited the RHEL for Edge image blueprint.

Procedure

  1. On the image builder dashboard click Create Image.
  2. On the Create Image window, perform the following steps:

    1. In the Image output page:

      1. From the Select a blueprint dropdown list, select the blueprint that you edited.
      2. From the Image output type dropdown list, select RHEL for Edge Commit (.tar). Click Next.
    2. In the OSTree settings page, enter:

      1. In the Repository URL field, enter the URL to the OSTree repository of the commit to embed in the image. For example, http://10.0.2.2:8080/repo/. See Setting up a web server to install RHEL for Edge image.
      2. In the Parent commit field, specify the parent commit ID that was previously generated. See Extracting RHEL for Edge image commit.
      3. In the Ref field, you can either specify a name for your commit or leave it empty. By default, the web console specifies the Ref as rhel/8/arch_name/edge. Click Next.
    3. In the Review page, check the customizations and click Create image. Image builder starts to create a RHEL for Edge image for the updated blueprint. The image creation process takes a few minutes to complete.

      To view the RHEL for Edge image creation progress, click the blueprint name from the breadcrumbs, and then click the Images tab.

      The resulting image includes the latest packages that you have added, if any, and have the original commit ID as a parent.

  3. Download the resulting RHEL for Edge Commit (.tar) image.

    1. From the Images tab, click Download to save the RHEL for Edge Commit (.tar) image to your system.
  4. Extract the OSTree commit (.tar) file.

    # tar -xf UUID-commit.tar -C UPGRADE_FOLDER
  5. Upgrade the OSTree repo:

    # ostree pull-local --repo http://10.0.2.2:8080/repo UPGRADE_FOLDER/repo OSTREE_REF
    # ostree summary --update --repo http://10.0.2.2:8080/repo
  6. Build a docker container, serving the child commit ID this time.

    # podman build -t name-of-server --build-arg commit=UUID-child_commit.tar .
  7. Run the container.

    # podman run --rm -p 8000:80 name-of-server
  8. On the RHEL system provisioned, from the original edge image, verify the current status.

    $ rpm-ostree status

    If there is no new commit ID, run the following command to verify if there is any upgrade available:

    $ rpm-ostree upgrade --check

    The command output provides the current active OSTree commit ID.

  9. Update OSTree to make the new OSTree commit ID available.

    $ rpm-ostree upgrade

    OSTree verifies if there is an update on the repository. If yes, it fetches the update and requests you to reboot your system so that you can activate the deployment of this new commit update.

  10. Check the current status again:

    $ rpm-ostree status

    You can now see that there are 2 commits available:

    • The active parent commit.
    • A new commit that is not active and contains 1 added difference.
  11. To activate the new deployment and to make the new commit active, reboot your system.

    # systemctl reboot

    The Anaconda Installer reboots into the new deployment. On the login screen, you can see a new deployment available for you to boot.

  12. If you want to boot into the newest deployment (commit), the rpm-ostree upgrade command automatically orders the boot entries so that the new deployment is first in the list. Optionally, you can use the arrow key on your keyboard to select the GRUB menu entry and press Enter.
  13. Provide your login user account credentials.
  14. Verify the OSTree status:

    $ rpm-ostree status

    The command output provides the active commit ID.

  15. To view the changed packages, if any, run a diff between the parent commit and the new commit:

    $ rpm-ostree db diff parent_commit new_commit

    The update shows that the package you have installed is available and ready for use.

10.2.4. Deploying RHEL for Edge image updates manually using the command-line

After you have edited a RHEL for Edge blueprint, you can update the image commit. Image builder generates a new commit for the updated RHEL for Edge image. Use the new commit to deploy the image with latest package versions or with additional packages using the CLI.

To deploy RHEL for Edge image updates using the CLI, ensure that you meet the prerequisites, and then follow the procedure.

Prerequisites

Procedure

  1. Create the RHEL for Edge Commit (.tar) image with the following arguments:

    # composer-cli compose start-ostree --ref ostree_ref --url URL-OSTree-repository -blueprint_name_ image-type

    where

    • ref is the reference you provided during the creation of the RHEL for Edge Container commit. For example, rhel/8/x86_64/edge.
    • URL-OSTree-repository is the URL to the OSTree repository of the commit to embed in the image. For example, http://10.0.2.2:8080/repo/. See Setting up a web server to install RHEL for Edge image.
    • image-type is edge-commit.

      Image builder creates a RHEL for Edge image for the updated blueprint.

  2. Check the RHEL for Edge image creation progress:

    # composer-cli compose status
    Note

    The image creation processes can take up to ten to thirty minutes to complete.

    The resulting image includes the latest packages that you have added, if any, and has the original commit ID as a parent.

  3. Download the resulting RHEL for Edge image. For more information, see Downloading a RHEL for Edge image using the image builder command-line interface.
  4. Extract the OSTree commit.

    # tar -xf UUID-commit.tar -C UPGRADE_FOLDER
  5. Serve the OSTree commit by using httpd. See Setting up a web server to install RHEL for Edge image.
  6. Upgrade the OSTree repo:

    # ostree pull-local --repo http://10.0.2.2:8080/repo UPGRADE_FOLDER/repo OSTREE_REF
    # ostree summary --update --repo http://10.0.2.2:8080/repo
  7. On the RHEL system provisioned from the original edge image, verify the current status:

    $ rpm-ostree status

    If there is no new commit ID, run the following command to verify if there is any upgrade available:

    $ rpm-ostree upgrade --check

    The command output provides the current active OSTree commit ID.

  8. Update OSTree to make the new OSTree commit ID available:

    $ rpm-ostree upgrade

    OSTree verifies if there is an update on the repository. If yes, it fetches the update and requests you to reboot your system so that you can activate the deployment of the new commit update.

  9. Check the current status again:

    $ rpm-ostree status

    You should now see that there are 2 commits available:

    • The active parent commit
    • A new commit that is not active and contains 1 added difference
  10. To activate the new deployment and make the new commit active, reboot your system:

    # systemctl reboot

    The Anaconda Installer reboots into the new deployment. On the login screen, you can see a new deployment available for you to boot.

  11. If you want to boot into the newest deployment, the rpm-ostree upgrade command automatically orders the boot entries so that the new deployment is first in the list. Optionally, you can use the arrow key on your keyboard to select the GRUB menu entry and press Enter.
  12. Log in using your account credentials.
  13. Verify the OSTree status:

    $ rpm-ostree status

    The command output provides the active commit ID.

  14. To view the changed packages, if any, run a diff between the parent commit and the new commit:

    $ rpm-ostree db diff parent_commit new_commit

    The update shows that the package you have installed is available and ready for use.

10.2.5. Deploying RHEL for Edge image updates manually for non-network-base deployments

After editing a RHEL for Edge blueprint, you can update the image commit. Image builder generates a new commit for the updated RHEL for Edge image. Use this new commit to deploy the image with latest package versions or with additional packages.

To deploy RHEL for Edge images updates, ensure that you meet the prerequisites and then follow the procedure.

Prerequisites

  • An RHEL for Edge system is up and running.
  • An OSTree repository is being served over HTTP.
  • You have created a RHEL for Edge image blueprint.
  • You have edited the RHEL for Edge image blueprint.

Procedure

  1. On the image builder dashboard click Create Image.
  2. On the Create Image window, perform the following steps:

    1. In the Image output page:

      1. From the Select a blueprint dropdown list, select the blueprint that you edited.
      2. From the Image output type dropdown list, select RHEL for Edge Container (.tar). Click Next.
    2. In the OSTree settings page, enter:

      1. In the Repository URL field, enter the URL to the OSTree repository of the commit to embed in the image. For example, http://10.0.2.2:8080/repo/. See Setting up a web server to install RHEL for Edge image.
      2. In the Parent commit field, specify the parent commit ID that was previously generated. See Extracting RHEL for Edge image commit.
      3. In the Ref field, you can either specify a name for your commit or leave it empty. By default, the web console specifies the Ref as rhel/8/arch_name/edge. Click Next.
    3. In the Review page, check the customizations and click Create image.

      Image builder creates a RHEL for Edge image for the updated blueprint.

      To view the progress of RHEL for Edge image creation, click the Images tab.

      Note

      The image creation process takes a few minutes to complete.

      The resulting image includes the latest packages that you have added, if any, and has the original commit ID as a parent.

  3. Download the resulting RHEL for Edge image

    1. From the Images tab, click Download to save the RHEL for Edge Container (.tar) image to your system.
  4. Load the RHEL for Edge Container image into Podman, serving the child commit ID this time.

    $ cat ./child-commit_ID-container.tar | sudo podman load
  5. Run Podman.

    #  sudo podman run -p 8080:8080 localhost/edge-test
  6. Upgrade the OSTree repo:

    # ostree pull-local --repo http://10.0.2.2:8080/repo UPGRADE_FOLDER/repo OSTREE_REF
    # ostree summary --update --repo http://10.0.2.2:8080/repo
  7. On the RHEL system provisioned, from the original edge image, verify the current status.

    $ rpm-ostree status

    If there is no new commit ID, run the following command to verify if there is any upgrade available:

    $ rpm-ostree upgrade --check

    If there are updates available, the command output provides information about the available updates in the OSTree repository, such as the current active OSTree commit ID. Else, it prompts a message informing that there are no updates available.

  8. Update OSTree to make the new OSTree commit ID available.

    $ rpm-ostree upgrade

    OSTree verifies if there is an update on the repository. If yes, it fetches the update and requests you to reboot your system so that you can activate the deployment of this new commit update.

  9. Check the current status:

    $ rpm-ostree status

    You can now see that there are 2 commits available:

    • The active parent commit.
    • A new commit that is not active and contains 1 added difference.
  10. To activate the new deployment and to make the new commit active, reboot your system.

    # systemctl reboot

    The Anaconda Installer reboots into the new deployment. On the login screen, you can see a new deployment available for you to boot.

  11. If you want to boot into the newest commit, the rpm-ostree upgrade command automatically orders the boot entries so that the new deployment is first in the list. Optionally, you can use the arrow key on your keyboard to select the GRUB menu entry and press Enter.
  12. Provide your login user account credentials.
  13. Verify the OSTree status:

    $ rpm-ostree status

    The command output provides the active commit ID.

  14. To view the changed packages, if any, run a diff between the parent commit and the new commit:

    $ rpm-ostree db diff parent_commit new_commit

    The update shows that the package you have installed is available and ready for use.

10.3. Deploying RHEL for Edge automatic image updates

After you install a RHEL for Edge image on an Edge device, you can check for image updates available, if any, and can auto-apply them.

The rpm-ostreed-automatic.service (systemd service) and rpm-ostreed-automatic.timer (systemd timer) control the frequency of checks and upgrades. The available updates, if any, appear as staged deployments.

Deploying automatic image updates involves the following high-level steps:

  • Update the image update policy
  • Enable automatic download and staging of updates

10.3.1. Updating the RHEL for Edge image update policy

To update the image update policy, use the AutomaticUpdatePolicy and an IdleExitTimeout setting from the rpm-ostreed.conf file at /etc/rpm-ostreed.conf location on an Edge device.

The AutomaticUpdatePolicy settings controls the automatic update policy and has the following update checks options:

  • none: Disables automatic updates. By default, the AutomaticUpdatePolicy setting is set to none.
  • check: Downloads enough metadata to display available updates with rpm-ostree status.
  • stage: Downloads and unpacks the updates that are applied on a reboot.

The IdleExitTimeout setting controls the time in seconds of inactivity before the daemon exit and has the following options:

  • 0: Disables auto-exit.
  • 60: By default, the IdleExitTimeout setting is set to 60.

To enable automatic updates, perform the following steps:

Procedure

  1. In the /etc/rpm-ostreed.conf file, update the following:

    • Change the value of AutomaticUpdatePolicy to check.
    • To run the update checks, specify a value in seconds for IdleExitTimeout.
  2. Reload the rpm-ostreed service and enable the systemd timer.

    # systemctl reload rpm-ostreed
    # systemctl enable rpm-ostreed-automatic.timer --now
  3. Verify the rpm-ostree status to ensure the automatic update policy is configured and time is active.

    # rpm-ostree status

    The command output shows the following:

    State: idle; auto updates enabled (check; last run <minutes> ago)

    Additionally, the output also displays information about the available updates.

10.3.2. Enabling RHEL for Edge automatic download and staging of updates

After you update the image update policy to check for image updates, the updates if any are displayed along with the update details. If you decide to apply the updates, enable the policy to automatically download and stage the updates. The available image updates are then downloaded and staged for deployment. The updates are applied and take effect when you reboot the Edge device.

To enable the policy for automatic download and staging of updates, perform the following updates:

Procedure

  1. In the /etc/rpm-ostreed.conf file, update ‘AutomaticUpdatePolicy’ to stage.
  2. Reload the rpm-ostreed service.

    # systemctl enable rpm-ostreed-automatic.timer --now
  3. Verify the rpm-ostree status

    # rpm-ostree status

    The command output shows the following:

    State: idle
    AutomaticUpdates: stage; rpm-ostreed-automatic.timer: last run <time> ago
  4. To initiate the updates, you can either wait for the timer to initiate the updates, or can manually start the service.

    # systemctl start rpm-ostreed-automatic.service

    After the updates are initiated, the rpm-ostree status shows the following:

    # rpm-ostree status
    State: busy
    AutomaticUpdates: stage; rpm-ostreed-automatic.service: running
    Transaction: automatic (stage)

    When the update is complete, a new deployment is staged in the list of deployments, and the original booted deployment is left untouched. You can decide if you want to boot the system using the new deployment or can wait for the next update.

    To view the list of deployments, run the rpm-ostree status command.

    Following is a sample output.

    # rpm-ostree status
    State: idle
    AutomaticUpdates: stage; rpm-ostreed-automatic.timer: last run <time> ago
    Deployments:

    To view the list of deployments with the updated package details, run the rpm-ostree status -v command.

10.4. Rolling back RHEL for Edge images

Because RHEL for Edge applies transactional updates to the operating system, you can either manually or automatically roll back the unsuccessful updates to the last known good state, which prevents system failure during updates. You can automate the verification and rollback process by using the greenboot framework.

The greenboot health check framework leverages rpm-ostree to run custom health checks on system startup. In case of an issue, the system rolls back to the last working state. When you deploy a rpm-ostree update, it runs scripts to check that critical services can still work after the update. If the system does not work, for example, due to some failed package, you can roll back the system to a previous stable version of the system. This process ensures that your RHEL for Edge device is in an operational state.

After you update an image, it creates a new image deployment while preserving the previous image deployment. You can verify whether the update was successful. If the update is unsuccessful, for example, due to a failed package, you can roll back the system to a previous stable version.

10.4.1. Introducing the greenboot checks

Greenboot is a Generic Health Check Framework for systemd available on rpm-ostree based systems. It contains the following RPM packages that you can install on your system:

  • greenboot - a package that contains the following functionalities:

    • Checking provided scripts
    • Reboot the system if the check fails
    • Rollback to a previous deployment the reboot did not solve the issue.
  • greenboot-default-health-checks - a set of optional and selected health checks provided by your greenboot system maintainers.

Greenboot works in a RHEL for Edge system by using health check scripts that run on the system to assess the system health and automate a rollback to the last healthy state in case of some software fails. These health checks scripts are available in the /etc/greenboot/check/required.d directory. Greenboot supports shell scripts for the health checks. Having a health check framework is especially useful when you need to check for software problems and perform system rollbacks on edge devices where direct serviceability is either limited or non-existent. When you install and configure health check scripts, it triggers the health checks to run every time the system starts.

You can create your own health check scripts to assess the health of your workloads and applications. These additional health check scripts are useful components of software problem checks and automatic system rollbacks.

Note

You cannot use rollback in case of any health check failure on a system that is not using OSTree.

10.4.2. RHEL for Edge images roll back with greenboot

With RHEL for Edge images, only transactional updates are applied to the operating system. The transactional updates are atomic, which means that the updates are applied only if all the updates are successful, and there is support for rollbacks. With the transactional updates, you can easily rollback the unsuccessful updates to the last known good state, preventing system failure during updates.

Performing health checks is especially useful when you need to check for software problems and perform system rollbacks on edge devices where direct serviceability is limited or non-existent.

Note

You cannot use rollback in case of an update failure on a system that is not using OSTree, even if health checks might run.

You can use intelligent rollbacks with the greenboot health check framework to automatically assess system health every time the system starts. You can obtain pre-configured health from the greenboot-default-health-checks subpackage. These checks are located in the /usr/lib/greenboot/check read-only directory in rpm-ostree systems.

Greenboot leverages rpm-ostree and runs custom health checks that run on system startup. In case of an issue, the system rolls back the changes and preserves the last working state. When you deploy an rpm-ostree update, it runs scripts to check that critical services can still work after the update. If the system does not work, the update rolls back to the last known working version of the system. This process ensures that your RHEL for Edge device is in an operational state.

You can obtain pre-configured health from the greenboot-default-health-checks`subpackage. These checks are located in the `/usr/lib/greenboot/check read-only directory in rpm-ostree systems. You can also configure shell scripts as the following types of checks:

Example 10.1. The greenboot directory structure

etc
└─ greenboot
   ├─ check
   |   └─ required.d
   |   └─ init.py
   └─ green.d
   └─ red.d
Required
Contains the health checks that must not fail. Place required shell scripts in the /etc/greenboot/check/required.d`directory. If the scripts fail, greenboot retries them three times by default. You can configure the number of retries in the `/etc/greenboot/greenboot.conf file by setting the GREENBOOT_MAX_BOOTS parameter to the number of retries you want.

After all retries fail, greenboot automatically initiates a rollback if one is available. If a rollback is not available, the system log output shows that you need to perform a manual intervention.

Wanted
Contains the health checks that might fail without causing the system to roll back. Place wanted shell scripts in the /etc/greenboot/check/wanted.d`directory. `Greenboot informs that the script fails, the system health status remains unaffected and it does not perform a rollback neither a reboot.

You can also specify shell scripts that will run after a check:

Green
Contains the scripts to run after a successful boot. Place these scripts into the /etc/greenboot/green.d`directory. `Greenboot informs that the boot was successful.
Red
Contains the scripts to run after a failed boot. Place these scripts into the /etc/greenboot/red.d directory. The system attempts to boot three times and in case of failure, it executes the scripts. Greenboot informs that the boot failed.

The following diagram illustrates the RHEL for Edge image roll back process.

Image restoration process

After booting the updated operating system, greenboot runs the scripts in the required.d and wanted.d directories. If any of the scripts fail in the required.d directory, greenboot runs any scripts in the red.d directory, and then reboots the system.

Greenboot makes 2 more attempts to boot on the upgraded system. If during the third boot attempt the scripts in required.d are still failing, greenboot runs the red.d scripts one last time, to ensure that the script in the red.d directory tried to make a corrective action to fix the issue and this was not successful. Then, greenboot rollbacks the system from the current rpm-ostree deployment to the previous stable deployment.

10.4.3. Manually rolling back RHEL for Edge images

When you upgrade your operating system, a new deployment is created, and the rpm-ostree package also keeps the previous deployment. If there are issues on the updated version of the operating system, you can manually roll back to the previous deployment with a single rpm-ostree command, or by selecting the previous deployment in the GRUB boot loader.

To manually roll back to a previous version, perform the following steps.

Prerequisite

  1. You updated your system and it failed.

Procedure

  1. Optional: Check for the fail error message:

    $ journalctl -u greenboot-healthcheck.service.
  2. Run the rollback command:

    # rpm-ostree rollback

    The command output provides details about the commit ID that is being moved and indicates a completed transaction with the details of the package being removed.

  3. Reboot the system.

    # systemctl reboot

    The command activates the previous commit with the stable content. The changes are applied and the previous version is restored.

10.4.4. Rolling back RHEL for Edge images using an automated process

Greenboot checks provides a framework that is integrated into the boot process and can trigger rpm-ostree rollbacks when a health check fails. For the health checks, you can create a custom script that indicates whether a health check passed or failed. Based on the result, you can decide when a rollback should be triggered. The following procedure shows how to create an health check script example:

Procedure

  1. Create a script that returns a standard exit code 0.

    For example, the following script ensures that the configured DNS server is available:

    #!/bin/bash
    
    DNS_SERVER=$(grep ^nameserver /etc/resolv.conf | head -n 1 | cut -f2 -d" ")
    COUNT=0
    # check DNS server is available
    ping -c1 $DNS_SERVER
    while [ $? != '0' ] && [ $COUNT -lt 10 ]; do
    ((COUNT++))
    echo "Checking for DNS: Attempt $COUNT ."
    sleep 10
    ping -c 1 $DNS_SERVER
    done
  2. Include an executable file for the health checks at /etc/greenboot/check/required.d/.

    chmod +x check-dns.sh

    During the next reboot, the script is executed as part of the boot process, before the system enters the boot-complete.target unit. If the health checks are successful, no action is taken. If the health checks fail, the system will reboot several times, before marking the update as failed and rolling back to the previous update.

Verification steps

To check if the default gateway is reachable, run the following health check script:

  1. Create a script that returns a standard exit code 0.

    #!/bin/bash
    
    DEF_GW=$(ip r | awk '/^default/ {print $3}')
    SCRIPT=$(basename $0)
    
    count=10
    connected=0
    ping_timeout=5
    interval=5
    
    while [ $count -gt 0 -a $connected -eq 0 ]; do
      echo "$SCRIPT: Pinging default gateway $DEF_GW"
      ping -c 1 -q -W $ping_timeout $DEF_GW > /dev/null 2>&1 && connected=1 || sleep $interval
      ((--count))
    done
    
    if [ $connected -eq 1 ]; then
      echo "$SCRIPT: Default gateway $DEF_GW is reachable."
      exit 0
    else
      echo "$SCRIPT: Failed to ping default gateway $DEF_GW!" 1>&2
      exit 1
    fi
  2. Include an executable file for the health checks at /etc/greenboot/check/required.d/ directory.

    chmod +x check-gw.sh

Appendix A. Terminology and commands

Learn more about the rpm ostree terminology and commands.

A.1. OSTree and rpm-ostree terminology

Following are some helpful terms that are used in context to OSTree and rpm-ostree images.

Table A.1. OSTree and rpm-ostree terminology

TermDefinition

OSTree

A tool used for managing Linux-based operating system versions. The OSTree tree view is similar to Git and is based on similar concepts.

rpm-ostree

A hybrid image or system package that hosts operating system updates.

Commit

A release or image version of the operating system. Image builder generates an ostree commit for RHEL for Edge images. You can use these images to install or update RHEL on Edge servers.

Refs

Represents a branch in ostree. Refs always resolve to the latest commit. For example, rhel/8/x86_64/edge.

Revision (Rev)

SHA-256 for a specific commit.

Remote

The http or https endpoint that hosts the ostree content. This is analogous to the baseurl for a yum repository.

static-delta

Updates to ostree images are always delta updates. In case of RHEL for Edge images, the TCP overhead can be higher than expected due to the updates to number of files. To avoid TCP overhead, you can generate static-delta between specific commits, and send the update in a single connection. This optimization helps large deployments with constrained connectivity.

A.2. OSTree commands

The following table provides a few OSTree commands that you can use when installing or managing OSTree images.

Table A.2. ostree commands

ostree pull

ostree pull-local --repo [path] src

ostree pull-local <path> <rev> --repo=<repo-path>

ostree pull <URL> <rev> --repo=<repo-path>

ostree summary

ostree summary -u --repo=<repo-path>

View refs

ostree refs --repo ~/Code/src/osbuild-iot/build/repo/ --list

View commits in repo

ostree log --repo=/home/gicmo/Code/src/osbuild-iot/build/repo/ <REV>

Inspect a commit

ostree show --repo build/repo <REV>

List remotes of a repo

ostree remote list --repo <repo-path>

Resolve a REV

ostree rev-parse --repo ~/Code/src/osbuild-iot/build/repo fedora/x86_64/osbuild-demo

ostree rev-parse --repo ~/Code/src/osbuild-iot/build/repo b3a008eceeddd0cfd

Create static-delta

ostree static-delta generate --repo=[path] --from=REV --to=REV

Sign an existing ostree commit with a GPG key

ostree gpg-sign --repo=<repo-path> --gpg-homedir <gpg_home> COMMIT KEY-ID…

A.3. rpm-ostree commands

The following table provides a few rpm-ostree commands that you can use when installing or managing OSTree images.

Table A.3. rpm-ostree commands

CommandsDescription

rpm-ostree --repo=/home/gicmo/Code/src/osbuild-iot/build/repo/ db list <REV>

This command lists the packages existing in the <REV> commit into the repository.

rpm-ostree rollback

OSTree manages an ordered list of boot loader entries, called deployments. The entry at index 0 is the default boot loader entry. Each entry has a separate /etc directory, but all the entries share a single /var directory. You can use the boot loader to choose between entries by pressing Tab to interrupt startup. This rolls back to the previous state, that is, the default deployment changes places with the non-default one.

rpm-ostree status

This command gives information about the current deployment in use. Lists the names and refspecs of all possible deployments in order, such that the first deployment in the list is the default upon boot. The deployment marked with * is the current booted deployment, and marking with 'r' indicates the most recent upgrade.

rpm-ostree db list

Use this command to see which packages are within the commit or commits. You must specify at least one commit, but more than one or a range of commits also work.

rpm-ostree db diff

Use this command to show how the packages are different between the trees in two revs (revisions). If no revs are provided, the booted commit is compared to the pending commit. If only a single rev is provided, the booted commit is compared to that rev.

rpm-ostree upgrade

This command downloads the latest version of the current tree, and deploys it, setting up the current tree as the default for the next boot. This has no effect on your running filesystem tree. You must reboot for any changes to take effect.

Additional resources

  • The [citetitle] rpm-ostree man page.

A.4. FDO automatic onboarding terminology

Learn more about the FDO terminology.

Table A.4. FDO terminology

CommandsDescription

FDO

FIDO Device Onboarding.

Device

Any hardware, device, or computer.

Owner

The final owner of the device - a company or an IT department.

Manufacturer

The device manufacturer.

Manufacturer server

Creates the device credentials for the device.

Manufacturer client

Informs the location of the manufacturing server.

Ownership Voucher (OV)

Record of ownership of an individual device. Contains the following information:

* Owner (fdo-owner-onboarding-service)

* Rendezvous Server - FIDO server (fdo-rendezvous-server)

* Device (at least one combination) (fdo-manufacturing-service)

Device Credential (DC)

Key credential and rendezvous stored in the device at manufacture.

Keys

Keys to configure the manufacturing server

* key_path

* cert_path

* key_type

* mfg_string_type: device serial number

* allowed_key_storage_types: Filesystem and Trusted Platform Module (TPM) that protects the data used to authenticate the device you are using.

Rendezvous server

Link to a server used by the device and later on, used on the process to find out who is the owner of the device

Additional resources

A.5. FDO automatic onboarding technologies

Following are the technologies used in context to FDO automatic onboarding.

Table A.5. OSTree and rpm-ostree terminology

TechnologyDefinition

UEFI

Unified Extensible Firmware Interface.

RHEL

Red Hat® Enterprise Linux® operating system

rpm-ostree

Background image-based upgrades.

Greenboot

Healthcheck framework for systemd on rpm-ostree.

Osbuild

Pipeline-based build system for operating system artifacts.

Container

A Linux® container is a set of 1 or more processes that are isolated from the rest of the system.

Coreos-installer

Assists installation of RHEL images, boots systems with UEFI.

FIDO FDO

Specification protocol to provision configuration and onboarding devices.

Legal Notice

Copyright © 2023 Red Hat, Inc.
The text of and illustrations in this document are licensed by Red Hat under a Creative Commons Attribution–Share Alike 3.0 Unported license ("CC-BY-SA"). An explanation of CC-BY-SA is available at http://creativecommons.org/licenses/by-sa/3.0/. In accordance with CC-BY-SA, if you distribute this document or an adaptation of it, you must provide the URL for the original version.
Red Hat, as the licensor of this document, waives the right to enforce, and agrees not to assert, Section 4d of CC-BY-SA to the fullest extent permitted by applicable law.
Red Hat, Red Hat Enterprise Linux, the Shadowman logo, the Red Hat logo, JBoss, OpenShift, Fedora, the Infinity logo, and RHCE are trademarks of Red Hat, Inc., registered in the United States and other countries.
Linux® is the registered trademark of Linus Torvalds in the United States and other countries.
Java® is a registered trademark of Oracle and/or its affiliates.
XFS® is a trademark of Silicon Graphics International Corp. or its subsidiaries in the United States and/or other countries.
MySQL® is a registered trademark of MySQL AB in the United States, the European Union and other countries.
Node.js® is an official trademark of Joyent. Red Hat is not formally related to or endorsed by the official Joyent Node.js open source or commercial project.
The OpenStack® Word Mark and OpenStack logo are either registered trademarks/service marks or trademarks/service marks of the OpenStack Foundation, in the United States and other countries and are used with the OpenStack Foundation's permission. We are not affiliated with, endorsed or sponsored by the OpenStack Foundation, or the OpenStack community.
All other trademarks are the property of their respective owners.