Is it possible to customize RHCOS in OpenShift 4?

Resolution

Starting with OpenShift 4.13 the Red Hat Enterprise Linux CoreOS (RHCOS) image layering is generally available. This feature allows the extension of the functionality of the base RHCOS image by layering additional images onto the base image. The use of RHCOS image layers allows the installation of RPMs on the base image.

On OpenShift 4.12 and earlier versions: Customization of RHCOS, including installation of custom RPMs, is not supported.

• If one wants to deploy an agent of some kind, this should be accomplished through OpenShift, typically through DaemonSets or Operators.

• Debugging tools also must be run in a container:

  • Either use CoreOS toolbox for this :

    • See: https://github.com/coreos/toolbox
    • For a concrete example use case, see: https://access.redhat.com/solutions/4387261

    • Red Hat OpenShift Platform contains a built-in feature for node debugging which launches and connects directly to the RHEL toolbox container:

      [cloud-user@jump-server openshift]$ oc debug node/osc-tb9b7-worker-6j98d
      Starting pod/osc-tb9b7-worker-6j98d-debug ...
      To use host binaries, run `chroot /host`
      Pod IP: 192.168.0.27
      If you don't see a command prompt, try pressing enter.
      sh-4.2# cat /etc/redhat-release  
      Red Hat Enterprise Linux Server release 7.8 (Maipo)
      

  • Alternatively, it is possible to deploy custom pods. For example, it is possible to spawn a simple fedora pod using the following command (note that Red Hat Technical support does not support Fedora containers):

    oc run --generator=run-pod/v1 -it fedora --image=fedora:latest /bin/bash
    

• In specific cases, either Ignition (during bootstrap) or MachineConfig and the MachineConfigOperator (post installation) can be used to modify the RHCOS host. However, these features are intended to push configuration changes to the underlying OS, and not to install new software.

See also: Is custom network configuration supported in RHCOS?