Chapter 16. Creating and managing OSTree image updates

You can easily create and manage OStree image updates for your RHEL for Edge systems and make them immediately available to RHEL for Edge devices. With OSTree, you can use image builder to create RHEL for Edge Commit or RHEL for Edge Container images as .tar files that contain OSTree commits. The OSTree update versioning system works as a “Git repository” that stores and versions the OSTree commits. The rpm-ostree image and package system then assembles the commits on the client device. When you create a new image with RHEL image builder to perform an update, RHEL image builder pulls updates from these repositories.

16.1. Basic concepts for OSTree

Basic terms that OSTree and rpm-ostree use during image updates.

rpm-ostree
The technology on the edge device that handles how the OSTree commits are assembled on the device. It works as a hybrid between an image and a package system. With the rpm-ostree technology, you can make atomic upgrades and rollbacks to your system.
OSTree
OSTree is a technology that enables you to create commits and download bootable file system trees. You can also use it to deploy the trees and manage the bootloader configuration.
Commit
An OSTree commit contains a full operating system that is not directly bootable. To boot the system, you must deploy it, for example, with a RHEL Installable image.
Reference

It is also known as ref. An OSTree ref is similar to a Git branch and it is a name. The following reference names examples are valid:

  • rhel/9/x86_64/edge
  • ref-name
  • app/org.gnome.Calculator/x86_64/stable
  • ref-name-2

By default, image builder specifies rhel/9/$ARCH/edge as a path. The "$ARCH" value is determined by the host machine.

Parent
The parent argument is an OSTree commit that you can provide to build a new commit with image builder. You can use the parent argument to specify an existing ref that retrieves a parent commit for the new commit that you are building. You must specify the parent commit as a ref value to be resolved and pulled, for example rhel/9/x86_64/edge. You can use the --parent commit for the RHEL for Edge Commit (.tar) and RHEL for Edge Container (.tar) image types.
Remote
The http or https endpoint that hosts the OSTree content. This is analogous to the baseurl for a yum repository.
Static delta
Static deltas are a collection of updates generated between two OSTree commits. This enables the system client to fetch a smaller amount of files, which are larger in size. The static deltas updates are more network efficient because, when updating an ostree-based host, the system client will only fetch the objects from the new OSTree commit which do not exist on the system. Typically, the new OSTree commit contains many small files, which requires multiple TCP connections.
Summary
The summary file is a concise way of enumerating refs, checksums, and available static deltas in an OSTree repo. You can check the state of all the refs and static deltas available in an Ostree repo. However, you must generate the summary file every time a new ref, commit, or static-delta is added to the OSTree repo.

16.2. Creating OSTree repositories

You can create OSTree repos with RHEL image builder by using either RHEL for Edge Commit (.tar) or RHEL for Edge Container (.tar) image types. These image types contain an OSTree repo that contains a single OSTree commit.

  • You can extract the RHEL for Edge Commit (.tar) on a web server and it is ready to be served.
  • You must import the RHEL for Edge Container (.tar) to a local container image storage or push the image to a container registry. After you start the container, it serves the commit over an integrated nginx web server.

Use the RHEL for Edge Container (.tar) on a RHEL server with Podman to create an OSTree repo:

Prerequisite

  • You created a RHEL for Edge Container (.tar) image.

Procedure

  1. Download the container image from image builder: 

    $ composer-cli compose image _<UUID>

  2. Import the container into Podman: 

    $ skopeo copy oci-archive:_<UUID>_-container.tar containers-storage:localhost/ostree

  3. Start the container and make it available by using the port 8080: 

    $ podman run -rm -p 8080:8080 ostree

Verification

  • Check that the container is running: 

    $ podman ps -a

16.3. Managing a centralized OSTree mirror

For production environments, having a central OSTree mirror that serves all the commits has several advantages, including:

  • Deduplicating and minimizing disk storage
  • Optimizing the updates to clients by using static delta updates
  • Pointing to a single OSTree mirror for their deployment life.

To manage a centralized OSTree mirror, you must pull each commit from image builder into the centralized repository where it will be available to your users.

Note

You can also automate managing an OSTree mirror by using the osbuild.infra Ansible collection. See osbuild.infra Ansible.

To create a centralized repository you can run the following commands directly on a web server:

Procedure

  1. Create an empty blueprint, customizing it to use "rhel-92" as the distro: 

    name = "minimal-rhel92"
description = "minimal blueprint for ostree commit"
version = "1.0.0"
modules = []
groups = []
distro = "rhel-92"

  2. Push the blueprint to the server: 

    # composer-cli blueprints push minimal-rhel92.toml

  3. Build a RHEL for Edge Commit (.tar) image from the blueprint you created: 

    # composer-cli compose start-ostree minimal-rhel92 edge-commit

  4. Retrieve the .tar file and decompress it to the disk: 

    # composer-cli compose image _<rhel-92-uuid>
$ tar -xf <rhel-92-uuid>.tar -C /usr/share/nginx/html/

    The /usr/share/nginx/html/repo location on disk will become the single OSTree repo for all refs and commits.

  5. Create another empty blueprint, customizing it to use "rhel-87" as the distro: 

    name = "minimal-rhel87"
description = "minimal blueprint for ostree commit"
version = "1.0.0"
modules = []
groups = []
distro = "rhel-87"
  6. Push the blueprint and create another RHEL for Edge Commit (.tar) image: 
# *composer-cli blueprints push minimal-rhel87.toml*
# *composer-cli compose start-ostree minimal-rhel87 edge-commit*

  1. Retrieve the .tar file and decompress it to the disk: 

    # composer-cli compose image <rhel-87-uuid>
$ tar -xf <rhel-87-uuid>.tar

  2. Pull the commit to the local repo. By using ostree pull-local, you can copy the commit data from one local repo to another local repo. 

    # ostree --repo=/usr/share/nginx/html/repo pull-local repo

  3. Optional: Inspect the status of the OSTree repo. The following is an output example: 

    $ ostree --repo=/usr/share/nginx/html/repo refs

rhel/8/x86_64/edge
rhel/9/x86_64/edge

$ ostree --repo=/usr/share/nginx/html/repo show rhel/8/x86_64/edge
commit f7d4d95465fbd875f6358141f39d0c573df6a321627bafde68c73850667e5443
ContentChecksum:  41bf2f8b442a770e9bf03e096a46a286f5836e0a0702b7c3516ef4e0acec2dea
Date:  2023-09-15 16:17:04 +0000
Version: 8.7
(no subject)

$ ostree --repo=/usr/share/nginx/html/repo show rhel/9/x86_64/edge
commit 89290dbfd6f749700c77cbc434c121432defb0c1c367532368eee170d9e53ea9
ContentChecksum:  70235bfb9cae82c53f856183750e809becf0b9b076122b19c40fec92fc6d74c1
Date:  2023-09-15 15:30:24 +0000
Version: 9.2
(no subject)

  4. Update the RHEL 9.2 blueprint to include a new package and build a new commit, for example: 

    name = "minimal-rhel92"
description = "minimal blueprint for ostree commit"
version = "1.1.0"
modules = []
groups = []
distro = "rhel-92"

[[packages]]
name = "strace"
version = "*"

  5. Push the updated blueprint and create a new RHEL for Edge Commit (.tar) image, pointing the compose to the existing OSTree repo: 

    # composer-cli blueprints push minimal-rhel92.toml
# composer-cli compose start-ostree minimal-rhel92 edge-commit --url http://localhost/repo --ref rhel/9/x86_64/edge

  6. Retrieve the .tar file and decompress it to the disk: 

    # rm -rf repo
# composer-cli compose image <rhel-92-uuid>
# tar -xf <rhel-92-uuid>.tar

  7. Pull the commit to repo: 

    # ostree --repo=/usr/share/nginx/html/repo pull-local repo

  8. Optional: Inspect the OSTree repo status again: 

    $ ostree --repo=/usr/share/nginx/html/repo refs
rhel/8/x86_64/edge
rhel/9/x86_64/edge

$ ostree --repo=/usr/share/nginx/html/repo show rhel/8/x86_64/edge
commit f7d4d95465fbd875f6358141f39d0c573df6a321627bafde68c73850667e5443
ContentChecksum:  41bf2f8b442a770e9bf03e096a46a286f5836e0a0702b7c3516ef4e0acec2dea
Date:  2023-09-15 16:17:04 +0000
Version: 8.7
(no subject)

$ ostree --repo=/usr/share/nginx/html/repo show rhel/9/x86_64/edge
commit a35c3b1a9e731622f32396bb1aa84c73b16bd9b9b423e09d72efaca11b0411c9
Parent:  89290dbfd6f749700c77cbc434c121432defb0c1c367532368eee170d9e53ea9
ContentChecksum:  2335930df6551bf7808e49f8b35c45e3aa2a11a6c84d988623fd3f36df42a1f1
Date:  2023-09-15 18:21:31 +0000
Version: 9.2
(no subject)

$ ostree --repo=/usr/share/nginx/html/repo log rhel/9/x86_64/edge
commit a35c3b1a9e731622f32396bb1aa84c73b16bd9b9b423e09d72efaca11b0411c9
Parent:  89290dbfd6f749700c77cbc434c121432defb0c1c367532368eee170d9e53ea9
ContentChecksum:  2335930df6551bf7808e49f8b35c45e3aa2a11a6c84d988623fd3f36df42a1f1
Date:  2023-09-15 18:21:31 +0000
Version: 9.2
(no subject)

commit 89290dbfd6f749700c77cbc434c121432defb0c1c367532368eee170d9e53ea9
ContentChecksum:  70235bfb9cae82c53f856183750e809becf0b9b076122b19c40fec92fc6d74c1
Date:  2023-09-15 15:30:24 +0000
Version: 9.2
(no subject)

$ rpm-ostree db diff --repo=/usr/share/nginx/html/repo 89290dbfd6f749700c77cbc434c121432defb0c1c367532368eee170d9e53ea9 a35c3b1a9e731622f32396bb1aa84c73b16bd9b9b423e09d72efaca11b0411c9
ostree diff commit from: 89290dbfd6f749700c77cbc434c121432defb0c1c367532368eee170d9e53ea9
ostree diff commit to:   a35c3b1a9e731622f32396bb1aa84c73b16bd9b9b423e09d72efaca11b0411c9
Added:
  elfutils-default-yama-scope-0.188-3.el9.noarch
  elfutils-libs-0.188-3.el9.x86_64
  strace-5.18-2.el9.x86_64