Chapter 3. Setting up and configuring the registry

3.1. Configuring the registry for AWS user-provisioned infrastructure

3.1.1. Configuring a secret for the Image Registry Operator

In addition to the configs.imageregistry.operator.openshift.io and ConfigMap resources, configuration is provided to the Operator by a separate secret resource located within the openshift-image-registry namespace.

The image-registry-private-configuration-user secret provides credentials needed for storage access and management. It overrides the default credentials used by the Operator, if default credentials were found.

For S3 on AWS storage, the secret is expected to contain two keys:

  • REGISTRY_STORAGE_S3_ACCESSKEY
  • REGISTRY_STORAGE_S3_SECRETKEY

Procedure

  • Create an OpenShift Container Platform secret that contains the required keys.

    $ oc create secret generic image-registry-private-configuration-user --from-literal=REGISTRY_STORAGE_S3_ACCESSKEY=myaccesskey --from-literal=REGISTRY_STORAGE_S3_SECRETKEY=mysecretkey --namespace openshift-image-registry

3.1.2. Configuring registry storage for AWS with user-provisioned infrastructure

During installation, your cloud credentials are sufficient to create an Amazon S3 bucket and the Registry Operator will automatically configure storage.

If the Registry Operator cannot create an S3 bucket and automatically configure storage, you can create an S3 bucket and configure storage with the following procedure.

Prerequisites

  • You have a cluster on AWS with user-provisioned infrastructure.
  • For Amazon S3 storage, the secret is expected to contain two keys:

    • REGISTRY_STORAGE_S3_ACCESSKEY
    • REGISTRY_STORAGE_S3_SECRETKEY

Procedure

Use the following procedure if the Registry Operator cannot create an S3 bucket and automatically configure storage.

  1. Set up a Bucket Lifecycle Policy to abort incomplete multipart uploads that are one day old.
  2. Fill in the storage configuration in configs.imageregistry.operator.openshift.io/cluster:

    $ oc edit configs.imageregistry.operator.openshift.io/cluster

    Example configuration

    storage:
      s3:
        bucket: <bucket-name>
        region: <region-name>

Warning

To secure your registry images in AWS, block public access to the S3 bucket.

3.1.3. Image Registry Operator configuration parameters for AWS S3

The following configuration parameters are available for AWS S3 registry storage.

The image registry spec.storage.s3 configuration parameter holds the information to configure the registry to use the AWS S3 service for back-end storage. See the S3 storage driver documentation for more information.

ParameterDescription

bucket

Bucket is the bucket name in which you want to store the registry’s data. It is optional and is generated if not provided.

region

Region is the AWS region in which your bucket exists. It is optional and is set based on the installed AWS Region.

regionEndpoint

RegionEndpoint is the endpoint for S3 compatible storage services. It is optional and defaults based on the Region that is provided.

virtualHostedStyle

VirtualHostedStyle enables using S3 virtual hosted style bucket paths with a custom RegionEndpoint. It is optional and defaults to false.

Set this parameter to deploy OpenShift Container Platform to hidden regions.

encrypt

Encrypt specifies whether or not the registry stores the image in encrypted format. It is optional and defaults to false.

keyID

KeyID is the KMS key ID to use for encryption. It is optional. Encrypt must be true, or this parameter is ignored.

cloudFront

CloudFront configures Amazon Cloudfront as the storage middleware in a registry. It is optional.

trustedCA

The namespace for the config map referenced by trustedCA is openshift-config. The key for the bundle in the config map is ca-bundle.crt. It is optional.

Note

When the value of the regionEndpoint parameter is configured to a URL of a Rados Gateway, an explicit port must not be specified. For example:

regionEndpoint: http://rook-ceph-rgw-ocs-storagecluster-cephobjectstore.openshift-storage.svc.cluster.local

3.2. Configuring the registry for GCP user-provisioned infrastructure

3.2.1. Configuring a secret for the Image Registry Operator

In addition to the configs.imageregistry.operator.openshift.io and ConfigMap resources, configuration is provided to the Operator by a separate secret resource located within the openshift-image-registry namespace.

The image-registry-private-configuration-user secret provides credentials needed for storage access and management. It overrides the default credentials used by the Operator, if default credentials were found.

For GCS on GCP storage, the secret is expected to contain one key whose value is the contents of a credentials file provided by GCP:

  • REGISTRY_STORAGE_GCS_KEYFILE

Procedure

  • Create an OpenShift Container Platform secret that contains the required keys.

    $ oc create secret generic image-registry-private-configuration-user --from-file=REGISTRY_STORAGE_GCS_KEYFILE=<path_to_keyfile> --namespace openshift-image-registry

3.2.2. Configuring the registry storage for GCP with user-provisioned infrastructure

If the Registry Operator cannot create a Google Cloud Platform (GCP) bucket, you must set up the storage medium manually and configure the settings in the registry custom resource (CR).

Prerequisites

  • A cluster on GCP with user-provisioned infrastructure.
  • To configure registry storage for GCP, you need to provide Registry Operator cloud credentials.
  • For GCS on GCP storage, the secret is expected to contain one key whose value is the contents of a credentials file provided by GCP:

    • REGISTRY_STORAGE_GCS_KEYFILE

Procedure

  1. Set up an Object Lifecycle Management policy to abort incomplete multipart uploads that are one day old.
  2. Fill in the storage configuration in configs.imageregistry.operator.openshift.io/cluster:

    $ oc edit configs.imageregistry.operator.openshift.io/cluster

    Example configuration

    # ...
    storage:
      gcs:
        bucket: <bucket-name>
        projectID: <project-id>
        region: <region-name>
    # ...

Warning

You can secure your registry images that use a Google Cloud Storage bucket by setting public access prevention.

3.2.3. Image Registry Operator configuration parameters for GCP GCS

The following configuration parameters are available for GCP GCS registry storage.

ParameterDescription

bucket

Bucket is the bucket name in which you want to store the registry’s data. It is optional and is generated if not provided.

region

Region is the GCS location in which your bucket exists. It is optional and is set based on the installed GCS Region.

projectID

ProjectID is the Project ID of the GCP project that this bucket should be associated with. It is optional.

keyID

KeyID is the KMS key ID to use for encryption. It is optional because buckets are encrypted by default on GCP. This allows for the use of a custom encryption key.

3.3. Configuring the registry for OpenStack user-provisioned infrastructure

You can configure the registry of a cluster that runs on your own Red Hat OpenStack Platform (RHOSP) infrastructure.

3.3.1. Configuring the Image Registry Operator to trust Swift storage

You must configure the Image Registry Operator to trust Red Hat OpenStack Platform (RHOSP) Swift storage.

Procedure

  • From a command line, enter the following command to change the value of the spec.disableRedirect field in the config.imageregistry object to true:

    $ oc patch configs.imageregistry.operator.openshift.io cluster --type merge --patch '{"spec":{"disableRedirect":true}}'

3.3.2. Configuring a secret for the Image Registry Operator

In addition to the configs.imageregistry.operator.openshift.io and ConfigMap resources, configuration is provided to the Operator by a separate secret resource located within the openshift-image-registry namespace.

The image-registry-private-configuration-user secret provides credentials needed for storage access and management. It overrides the default credentials used by the Operator, if default credentials were found.

For Swift on Red Hat OpenStack Platform (RHOSP) storage, the secret is expected to contain the following two keys:

  • REGISTRY_STORAGE_SWIFT_USERNAME
  • REGISTRY_STORAGE_SWIFT_PASSWORD

Procedure

  • Create an OpenShift Container Platform secret that contains the required keys.

    $ oc create secret generic image-registry-private-configuration-user --from-literal=REGISTRY_STORAGE_SWIFT_USERNAME=<username> --from-literal=REGISTRY_STORAGE_SWIFT_PASSWORD=<password> -n openshift-image-registry

3.3.3. Registry storage for RHOSP with user-provisioned infrastructure

If the Registry Operator cannot create a Swift bucket, you must set up the storage medium manually and configure the settings in the registry custom resource (CR).

Prerequisites

  • A cluster on Red Hat OpenStack Platform (RHOSP) with user-provisioned infrastructure.
  • To configure registry storage for RHOSP, you need to provide Registry Operator cloud credentials.
  • For Swift on RHOSP storage, the secret is expected to contain the following two keys:

    • REGISTRY_STORAGE_SWIFT_USERNAME
    • REGISTRY_STORAGE_SWIFT_PASSWORD

Procedure

  • Fill in the storage configuration in configs.imageregistry.operator.openshift.io/cluster:

    $ oc edit configs.imageregistry.operator.openshift.io/cluster

    Example configuration

    # ...
    storage:
      swift:
        container: <container-id>
    # ...

3.3.4. Image Registry Operator configuration parameters for RHOSP Swift

The following configuration parameters are available for Red Hat OpenStack Platform (RHOSP) Swift registry storage.

ParameterDescription

authURL

Defines the URL for obtaining the authentication token. This value is optional.

authVersion

Specifies the Auth version of RHOSP, for example, authVersion: "3". This value is optional.

container

Defines the name of a Swift container for storing registry data. This value is optional.

domain

Specifies the RHOSP domain name for the Identity v3 API. This value is optional.

domainID

Specifies the RHOSP domain ID for the Identity v3 API. This value is optional.

tenant

Defines the RHOSP tenant name to be used by the registry. This value is optional.

tenantID

Defines the RHOSP tenant ID to be used by the registry. This value is optional.

regionName

Defines the RHOSP region in which the container exists. This value is optional.

3.4. Configuring the registry for Azure user-provisioned infrastructure

3.4.1. Configuring a secret for the Image Registry Operator

In addition to the configs.imageregistry.operator.openshift.io and ConfigMap resources, configuration is provided to the Operator by a separate secret resource located within the openshift-image-registry namespace.

The image-registry-private-configuration-user secret provides credentials needed for storage access and management. It overrides the default credentials used by the Operator, if default credentials were found.

For Azure registry storage, the secret is expected to contain one key whose value is the contents of a credentials file provided by Azure:

  • REGISTRY_STORAGE_AZURE_ACCOUNTKEY

Procedure

  • Create an OpenShift Container Platform secret that contains the required key.

    $ oc create secret generic image-registry-private-configuration-user --from-literal=REGISTRY_STORAGE_AZURE_ACCOUNTKEY=<accountkey> --namespace openshift-image-registry

3.4.2. Configuring registry storage for Azure

During installation, your cloud credentials are sufficient to create Azure Blob Storage, and the Registry Operator automatically configures storage.

Prerequisites

  • A cluster on Azure with user-provisioned infrastructure.
  • To configure registry storage for Azure, provide Registry Operator cloud credentials.
  • For Azure storage the secret is expected to contain one key:

    • REGISTRY_STORAGE_AZURE_ACCOUNTKEY

Procedure

  1. Create an Azure storage container.
  2. Fill in the storage configuration in configs.imageregistry.operator.openshift.io/cluster:

    $ oc edit configs.imageregistry.operator.openshift.io/cluster

    Example configuration

    storage:
      azure:
        accountName: <storage-account-name>
        container: <container-name>

3.4.3. Configuring registry storage for Azure Government

During installation, your cloud credentials are sufficient to create Azure Blob Storage, and the Registry Operator automatically configures storage.

Prerequisites

  • A cluster on Azure with user-provisioned infrastructure in a government region.
  • To configure registry storage for Azure, provide Registry Operator cloud credentials.
  • For Azure storage, the secret is expected to contain one key:

    • REGISTRY_STORAGE_AZURE_ACCOUNTKEY

Procedure

  1. Create an Azure storage container.
  2. Fill in the storage configuration in configs.imageregistry.operator.openshift.io/cluster:

    $ oc edit configs.imageregistry.operator.openshift.io/cluster

    Example configuration

    storage:
      azure:
        accountName: <storage-account-name>
        container: <container-name>
        cloudName: AzureUSGovernmentCloud 1

    1
    cloudName is the name of the Azure cloud environment, which can be used to configure the Azure SDK with the appropriate Azure API endpoints. Defaults to AzurePublicCloud. You can also set cloudName to AzureUSGovernmentCloud, AzureChinaCloud, or AzureGermanCloud with sufficient credentials.

3.5. Configuring the registry for RHOSP

3.5.1. Configuring an image registry with custom storage on clusters that run on RHOSP

After you install a cluster on Red Hat OpenStack Platform (RHOSP), you can use a Cinder volume that is in a specific availability zone for registry storage.

Procedure

  1. Create a YAML file that specifies the storage class and availability zone to use. For example:

    apiVersion: storage.k8s.io/v1
    kind: StorageClass
    metadata:
      name: custom-csi-storageclass
    provisioner: cinder.csi.openstack.org
    volumeBindingMode: WaitForFirstConsumer
    allowVolumeExpansion: true
    parameters:
      availability: <availability_zone_name>
    Note

    OpenShift Container Platform does not verify the existence of the availability zone you choose. Verify the name of the availability zone before you apply the configuration.

  2. From a command line, apply the configuration:

    $ oc apply -f <storage_class_file_name>

    Example output

    storageclass.storage.k8s.io/custom-csi-storageclass created

  3. Create a YAML file that specifies a persistent volume claim (PVC) that uses your storage class and the openshift-image-registry namespace. For example:

    apiVersion: v1
    kind: PersistentVolumeClaim
    metadata:
      name: csi-pvc-imageregistry
      namespace: openshift-image-registry 1
      annotations:
        imageregistry.openshift.io: "true"
    spec:
      accessModes:
      - ReadWriteOnce
      volumeMode: Filesystem
      resources:
        requests:
          storage: 100Gi 2
      storageClassName: <your_custom_storage_class> 3
    1
    Enter the namespace openshift-image-registry. This namespace allows the Cluster Image Registry Operator to consume the PVC.
    2
    Optional: Adjust the volume size.
    3
    Enter the name of the storage class that you created.
  4. From a command line, apply the configuration:

    $ oc apply -f <pvc_file_name>

    Example output

    persistentvolumeclaim/csi-pvc-imageregistry created

  5. Replace the original persistent volume claim in the image registry configuration with the new claim:

    $ oc patch configs.imageregistry.operator.openshift.io/cluster --type 'json' -p='[{"op": "replace", "path": "/spec/storage/pvc/claim", "value": "csi-pvc-imageregistry"}]'

    Example output

    config.imageregistry.operator.openshift.io/cluster patched

    Over the next several minutes, the configuration is updated.

Verification

To confirm that the registry is using the resources that you defined:

  1. Verify that the PVC claim value is identical to the name that you provided in your PVC definition:

    $ oc get configs.imageregistry.operator.openshift.io/cluster -o yaml

    Example output

    ...
    status:
        ...
        managementState: Managed
        pvc:
          claim: csi-pvc-imageregistry
    ...

  2. Verify that the status of the PVC is Bound:

    $ oc get pvc -n openshift-image-registry csi-pvc-imageregistry

    Example output

    NAME                   STATUS   VOLUME                                     CAPACITY   ACCESS MODES   STORAGECLASS             AGE
    csi-pvc-imageregistry  Bound    pvc-72a8f9c9-f462-11e8-b6b6-fa163e18b7b5   100Gi      RWO            custom-csi-storageclass  11m

3.6. Configuring the registry for bare metal

3.6.1. Image registry removed during installation

On platforms that do not provide shareable object storage, the OpenShift Image Registry Operator bootstraps itself as Removed. This allows openshift-installer to complete installations on these platform types.

After installation, you must edit the Image Registry Operator configuration to switch the managementState from Removed to Managed.

3.6.2. Changing the image registry’s management state

To start the image registry, you must change the Image Registry Operator configuration’s managementState from Removed to Managed.

Procedure

  • Change managementState Image Registry Operator configuration from Removed to Managed. For example:

    $ oc patch configs.imageregistry.operator.openshift.io cluster --type merge --patch '{"spec":{"managementState":"Managed"}}'

3.6.3. Image registry storage configuration

The Image Registry Operator is not initially available for platforms that do not provide default storage. After installation, you must configure your registry to use storage so that the Registry Operator is made available.

Instructions are shown for configuring a persistent volume, which is required for production clusters. Where applicable, instructions are shown for configuring an empty directory as the storage location, which is available for only non-production clusters.

Additional instructions are provided for allowing the image registry to use block storage types by using the Recreate rollout strategy during upgrades.

3.6.3.1. Configuring registry storage for bare metal and other manual installations

As a cluster administrator, following installation you must configure your registry to use storage.

Prerequisites

  • You have access to the cluster as a user with the cluster-admin role.
  • You have a cluster that uses manually-provisioned Red Hat Enterprise Linux CoreOS (RHCOS) nodes, such as bare metal.
  • You have provisioned persistent storage for your cluster, such as Red Hat OpenShift Data Foundation.

    Important

    OpenShift Container Platform supports ReadWriteOnce access for image registry storage when you have only one replica. ReadWriteOnce access also requires that the registry uses the Recreate rollout strategy. To deploy an image registry that supports high availability with two or more replicas, ReadWriteMany access is required.

  • Must have 100Gi capacity.

Procedure

  1. To configure your registry to use storage, change the spec.storage.pvc in the configs.imageregistry/cluster resource.

    Note

    When you use shared storage, review your security settings to prevent outside access.

  2. Verify that you do not have a registry pod:

    $ oc get pod -n openshift-image-registry -l docker-registry=default

    Example output

    No resources found in openshift-image-registry namespace

    Note

    If you do have a registry pod in your output, you do not need to continue with this procedure.

  3. Check the registry configuration:

    $ oc edit configs.imageregistry.operator.openshift.io

    Example output

    storage:
      pvc:
        claim:

    Leave the claim field blank to allow the automatic creation of an image-registry-storage PVC.

  4. Check the clusteroperator status:

    $ oc get clusteroperator image-registry

    Example output

    NAME             VERSION              AVAILABLE   PROGRESSING   DEGRADED   SINCE   MESSAGE
    image-registry   4.13                 True        False         False      6h50m

  5. Ensure that your registry is set to managed to enable building and pushing of images.

    • Run:

      $ oc edit configs.imageregistry/cluster

      Then, change the line

      managementState: Removed

      to

      managementState: Managed

3.6.3.2. Configuring storage for the image registry in non-production clusters

You must configure storage for the Image Registry Operator. For non-production clusters, you can set the image registry to an empty directory. If you do so, all images are lost if you restart the registry.

Procedure

  • To set the image registry storage to an empty directory:

    $ oc patch configs.imageregistry.operator.openshift.io cluster --type merge --patch '{"spec":{"storage":{"emptyDir":{}}}}'
    Warning

    Configure this option for only non-production clusters.

    If you run this command before the Image Registry Operator initializes its components, the oc patch command fails with the following error:

    Error from server (NotFound): configs.imageregistry.operator.openshift.io "cluster" not found

    Wait a few minutes and run the command again.

3.6.3.3. Configuring block registry storage for bare metal

To allow the image registry to use block storage types during upgrades as a cluster administrator, you can use the Recreate rollout strategy.

Important

Block storage volumes, or block persistent volumes, are supported but not recommended for use with the image registry on production clusters. An installation where the registry is configured on block storage is not highly available because the registry cannot have more than one replica.

If you choose to use a block storage volume with the image registry, you must use a filesystem persistent volume claim (PVC).

Procedure

  1. Enter the following command to set the image registry storage as a block storage type, patch the registry so that it uses the Recreate rollout strategy, and runs with only one (1) replica:

    $ oc patch config.imageregistry.operator.openshift.io/cluster --type=merge -p '{"spec":{"rolloutStrategy":"Recreate","replicas":1}}'
  2. Provision the PV for the block storage device, and create a PVC for that volume. The requested block volume uses the ReadWriteOnce (RWO) access mode.

    1. Create a pvc.yaml file with the following contents to define a VMware vSphere PersistentVolumeClaim object:

      kind: PersistentVolumeClaim
      apiVersion: v1
      metadata:
        name: image-registry-storage 1
        namespace: openshift-image-registry 2
      spec:
        accessModes:
        - ReadWriteOnce 3
        resources:
          requests:
            storage: 100Gi 4
      1
      A unique name that represents the PersistentVolumeClaim object.
      2
      The namespace for the PersistentVolumeClaim object, which is openshift-image-registry.
      3
      The access mode of the persistent volume claim. With ReadWriteOnce, the volume can be mounted with read and write permissions by a single node.
      4
      The size of the persistent volume claim.
    2. Enter the following command to create the PersistentVolumeClaim object from the file:

      $ oc create -f pvc.yaml -n openshift-image-registry
  3. Enter the following command to edit the registry configuration so that it references the correct PVC:

    $ oc edit config.imageregistry.operator.openshift.io -o yaml

    Example output

    storage:
      pvc:
        claim: 1

    1
    By creating a custom PVC, you can leave the claim field blank for the default automatic creation of an image-registry-storage PVC.

3.6.3.4. Configuring the Image Registry Operator to use Ceph RGW storage with Red Hat OpenShift Data Foundation

Red Hat OpenShift Data Foundation integrates multiple storage types that you can use with the OpenShift image registry:

  • Ceph, a shared and distributed file system and on-premises object storage
  • NooBaa, providing a Multicloud Object Gateway

This document outlines the procedure to configure the image registry to use Ceph RGW storage.

Prerequisites

  • You have access to the cluster as a user with the cluster-admin role.
  • You have access to the OpenShift Container Platform web console.
  • You installed the oc CLI.
  • You installed the OpenShift Data Foundation Operator to provide object storage and Ceph RGW object storage.

Procedure

  1. Create the object bucket claim using the ocs-storagecluster-ceph-rgw storage class. For example:

    cat <<EOF | oc apply -f -
    apiVersion: objectbucket.io/v1alpha1
    kind: ObjectBucketClaim
    metadata:
      name: rgwbucket
      namespace: openshift-storage 1
    spec:
      storageClassName: ocs-storagecluster-ceph-rgw
      generateBucketName: rgwbucket
    EOF
    1
    Alternatively, you can use the openshift-image-registry namespace.
  2. Get the bucket name by entering the following command:

    $ bucket_name=$(oc get obc -n openshift-storage rgwbucket -o jsonpath='{.spec.bucketName}')
  3. Get the AWS credentials by entering the following commands:

    $ AWS_ACCESS_KEY_ID=$(oc get secret -n openshift-storage rgwbucket -o jsonpath='{.data.AWS_ACCESS_KEY_ID}' | base64 --decode)
    $ AWS_SECRET_ACCESS_KEY=$(oc get secret -n openshift-storage rgwbucket -o jsonpath='{.data.AWS_SECRET_ACCESS_KEY}' | base64 --decode)
  4. Create the secret image-registry-private-configuration-user with the AWS credentials for the new bucket under openshift-image-registry project by entering the following command:

    $ oc create secret generic image-registry-private-configuration-user --from-literal=REGISTRY_STORAGE_S3_ACCESSKEY=${AWS_ACCESS_KEY_ID} --from-literal=REGISTRY_STORAGE_S3_SECRETKEY=${AWS_SECRET_ACCESS_KEY} --namespace openshift-image-registry
  5. Get the buckethost value by entering the following command:

    $ route_host=$(oc get objectbucket $bucket_name -n openshift-storage -o=jsonpath='{.spec.endpoint.bucketHost}')
  6. Create a config map that uses an ingress certificate by entering the following commands:

    $ oc extract secret/router-certs-default  -n openshift-ingress  --confirm
    $ oc create configmap image-registry-s3-bundle --from-file=ca-bundle.crt=./tls.crt  -n openshift-config
  7. Configure the image registry to use the Ceph RGW object storage by entering the following command:

    $ oc patch config.image/cluster -p '{"spec":{"managementState":"Managed","replicas":2,"storage":{"managementState":"Unmanaged","s3":{"bucket":'\"${bucket_name}\"',"region":"us-east-1","regionEndpoint":'\"https://${route_host}\"',"virtualHostedStyle":false,"encrypt":false,"trustedCA":{"name":"image-registry-s3-bundle"}}}}}' --type=merge

3.6.3.5. Configuring the Image Registry Operator to use Noobaa storage with Red Hat OpenShift Data Foundation

Red Hat OpenShift Data Foundation integrates multiple storage types that you can use with the OpenShift image registry:

  • Ceph, a shared and distributed file system and on-premises object storage
  • NooBaa, providing a Multicloud Object Gateway

This document outlines the procedure to configure the image registry to use Noobaa storage.

Prerequisites

  • You have access to the cluster as a user with the cluster-admin role.
  • You have access to the OpenShift Container Platform web console.
  • You installed the oc CLI.
  • You installed the OpenShift Data Foundation Operator to provide object storage and Noobaa object storage.

Procedure

  1. Create the object bucket claim using the openshift-storage.noobaa.io storage class. For example:

    cat <<EOF | oc apply -f -
    apiVersion: objectbucket.io/v1alpha1
    kind: ObjectBucketClaim
    metadata:
      name: noobaatest
      namespace: openshift-storage 1
    spec:
      storageClassName: openshift-storage.noobaa.io
      generateBucketName: noobaatest
    EOF
    1
    Alternatively, you can use the openshift-image-registry namespace.
  2. Get the bucket name by entering the following command:

    $ bucket_name=$(oc get obc -n openshift-storage noobaatest -o jsonpath='{.spec.bucketName}')
  3. Get the AWS credentials by entering the following commands:

    $ AWS_ACCESS_KEY_ID=$(oc get secret -n openshift-storage noobaatest -o yaml | grep -w "AWS_ACCESS_KEY_ID:" | head -n1 | awk '{print $2}' | base64 --decode)
    $ AWS_SECRET_ACCESS_KEY=$(oc get secret -n openshift-storage noobaatest -o yaml | grep -w "AWS_SECRET_ACCESS_KEY:" | head -n1 | awk '{print $2}' | base64 --decode)
  4. Create the secret image-registry-private-configuration-user with the AWS credentials for the new bucket under openshift-image-registry project by entering the following command:

    $ oc create secret generic image-registry-private-configuration-user --from-literal=REGISTRY_STORAGE_S3_ACCESSKEY=${AWS_ACCESS_KEY_ID} --from-literal=REGISTRY_STORAGE_S3_SECRETKEY=${AWS_SECRET_ACCESS_KEY} --namespace openshift-image-registry
  5. Get the route host by entering the following command:

    $ route_host=$(oc get route s3 -n openshift-storage -o=jsonpath='{.spec.host}')
  6. Create a config map that uses an ingress certificate by entering the following commands:

    $ oc extract secret/router-certs-default  -n openshift-ingress  --confirm
    $ oc create configmap image-registry-s3-bundle --from-file=ca-bundle.crt=./tls.crt  -n openshift-config
  7. Configure the image registry to use the Nooba object storage by entering the following command:

    $ oc patch config.image/cluster -p '{"spec":{"managementState":"Managed","replicas":2,"storage":{"managementState":"Unmanaged","s3":{"bucket":'\"${bucket_name}\"',"region":"us-east-1","regionEndpoint":'\"https://${route_host}\"',"virtualHostedStyle":false,"encrypt":false,"trustedCA":{"name":"image-registry-s3-bundle"}}}}}' --type=merge

3.6.4. Configuring the Image Registry Operator to use CephFS storage with Red Hat OpenShift Data Foundation

Red Hat OpenShift Data Foundation integrates multiple storage types that you can use with the OpenShift image registry:

  • Ceph, a shared and distributed file system and on-premises object storage
  • NooBaa, providing a Multicloud Object Gateway

This document outlines the procedure to configure the image registry to use CephFS storage.

Note

CephFS uses persistent volume claim (PVC) storage. It is not recommended to use PVCs for image registry storage if there are other options are available, such as Ceph RGW or Noobaa.

Prerequisites

  • You have access to the cluster as a user with the cluster-admin role.
  • You have access to the OpenShift Container Platform web console.
  • You installed the oc CLI.
  • You installed the OpenShift Data Foundation Operator to provide object storage and CephFS file storage.

Procedure

  1. Create a PVC to use the cephfs storage class. For example:

    cat <<EOF | oc apply -f -
    apiVersion: v1
    kind: PersistentVolumeClaim
    metadata:
     name: registry-storage-pvc
     namespace: openshift-image-registry
    spec:
     accessModes:
     - ReadWriteMany
     resources:
       requests:
         storage: 100Gi
     storageClassName: ocs-storagecluster-cephfs
    EOF
  2. Configure the image registry to use the CephFS file system storage by entering the following command:

    $ oc patch config.image/cluster -p '{"spec":{"managementState":"Managed","replicas":2,"storage":{"managementState":"Unmanaged","pvc":{"claim":"registry-storage-pvc"}}}}' --type=merge

3.6.5. Additional resources

3.7. Configuring the registry for vSphere

3.7.1. Image registry removed during installation

On platforms that do not provide shareable object storage, the OpenShift Image Registry Operator bootstraps itself as Removed. This allows openshift-installer to complete installations on these platform types.

After installation, you must edit the Image Registry Operator configuration to switch the managementState from Removed to Managed.

3.7.2. Changing the image registry’s management state

To start the image registry, you must change the Image Registry Operator configuration’s managementState from Removed to Managed.

Procedure

  • Change managementState Image Registry Operator configuration from Removed to Managed. For example:

    $ oc patch configs.imageregistry.operator.openshift.io cluster --type merge --patch '{"spec":{"managementState":"Managed"}}'

3.7.3. Image registry storage configuration

The Image Registry Operator is not initially available for platforms that do not provide default storage. After installation, you must configure your registry to use storage so that the Registry Operator is made available.

Instructions are shown for configuring a persistent volume, which is required for production clusters. Where applicable, instructions are shown for configuring an empty directory as the storage location, which is available for only non-production clusters.

Additional instructions are provided for allowing the image registry to use block storage types by using the Recreate rollout strategy during upgrades.

3.7.3.1. Configuring registry storage for VMware vSphere

As a cluster administrator, following installation you must configure your registry to use storage.

Prerequisites

  • Cluster administrator permissions.
  • A cluster on VMware vSphere.
  • Persistent storage provisioned for your cluster, such as Red Hat OpenShift Data Foundation.

    Important

    OpenShift Container Platform supports ReadWriteOnce access for image registry storage when you have only one replica. ReadWriteOnce access also requires that the registry uses the Recreate rollout strategy. To deploy an image registry that supports high availability with two or more replicas, ReadWriteMany access is required.

  • Must have "100Gi" capacity.
Important

Testing shows issues with using the NFS server on RHEL as storage backend for core services. This includes the OpenShift Container Registry and Quay, Prometheus for monitoring storage, and Elasticsearch for logging storage. Therefore, using RHEL NFS to back PVs used by core services is not recommended.

Other NFS implementations on the marketplace might not have these issues. Contact the individual NFS implementation vendor for more information on any testing that was possibly completed against these OpenShift Container Platform core components.

Procedure

  1. To configure your registry to use storage, change the spec.storage.pvc in the configs.imageregistry/cluster resource.

    Note

    When you use shared storage, review your security settings to prevent outside access.

  2. Verify that you do not have a registry pod:

    $ oc get pod -n openshift-image-registry -l docker-registry=default

    Example output

    No resourses found in openshift-image-registry namespace

    Note

    If you do have a registry pod in your output, you do not need to continue with this procedure.

  3. Check the registry configuration:

    $ oc edit configs.imageregistry.operator.openshift.io

    Example output

    storage:
      pvc:
        claim: 1

    1
    Leave the claim field blank to allow the automatic creation of an image-registry-storage persistent volume claim (PVC). The PVC is generated based on the default storage class. However, be aware that the default storage class might provide ReadWriteOnce (RWO) volumes, such as a RADOS Block Device (RBD), which can cause issues when you replicate to more than one replica.
  4. Check the clusteroperator status:

    $ oc get clusteroperator image-registry

    Example output

    NAME             VERSION                              AVAILABLE   PROGRESSING   DEGRADED   SINCE   MESSAGE
    image-registry   4.7                                  True        False         False      6h50m

3.7.3.2. Configuring storage for the image registry in non-production clusters

You must configure storage for the Image Registry Operator. For non-production clusters, you can set the image registry to an empty directory. If you do so, all images are lost if you restart the registry.

Procedure

  • To set the image registry storage to an empty directory:

    $ oc patch configs.imageregistry.operator.openshift.io cluster --type merge --patch '{"spec":{"storage":{"emptyDir":{}}}}'
    Warning

    Configure this option for only non-production clusters.

    If you run this command before the Image Registry Operator initializes its components, the oc patch command fails with the following error:

    Error from server (NotFound): configs.imageregistry.operator.openshift.io "cluster" not found

    Wait a few minutes and run the command again.

3.7.3.3. Configuring block registry storage for VMware vSphere

To allow the image registry to use block storage types such as vSphere Virtual Machine Disk (VMDK) during upgrades as a cluster administrator, you can use the Recreate rollout strategy.

Important

Block storage volumes are supported but not recommended for use with image registry on production clusters. An installation where the registry is configured on block storage is not highly available because the registry cannot have more than one replica.

Procedure

  1. Enter the following command to set the image registry storage as a block storage type, patch the registry so that it uses the Recreate rollout strategy, and runs with only 1 replica:

    $ oc patch config.imageregistry.operator.openshift.io/cluster --type=merge -p '{"spec":{"rolloutStrategy":"Recreate","replicas":1}}'
  2. Provision the PV for the block storage device, and create a PVC for that volume. The requested block volume uses the ReadWriteOnce (RWO) access mode.

    1. Create a pvc.yaml file with the following contents to define a VMware vSphere PersistentVolumeClaim object:

      kind: PersistentVolumeClaim
      apiVersion: v1
      metadata:
        name: image-registry-storage 1
        namespace: openshift-image-registry 2
      spec:
        accessModes:
        - ReadWriteOnce 3
        resources:
          requests:
            storage: 100Gi 4
      1
      A unique name that represents the PersistentVolumeClaim object.
      2
      The namespace for the PersistentVolumeClaim object, which is openshift-image-registry.
      3
      The access mode of the persistent volume claim. With ReadWriteOnce, the volume can be mounted with read and write permissions by a single node.
      4
      The size of the persistent volume claim.
    2. Enter the following command to create the PersistentVolumeClaim object from the file:

      $ oc create -f pvc.yaml -n openshift-image-registry
  3. Enter the following command to edit the registry configuration so that it references the correct PVC:

    $ oc edit config.imageregistry.operator.openshift.io -o yaml

    Example output

    storage:
      pvc:
        claim: 1

    1
    By creating a custom PVC, you can leave the claim field blank for the default automatic creation of an image-registry-storage PVC.

For instructions about configuring registry storage so that it references the correct PVC, see Configuring the registry for vSphere.

3.7.3.4. Configuring the Image Registry Operator to use Ceph RGW storage with Red Hat OpenShift Data Foundation

Red Hat OpenShift Data Foundation integrates multiple storage types that you can use with the OpenShift image registry:

  • Ceph, a shared and distributed file system and on-premises object storage
  • NooBaa, providing a Multicloud Object Gateway

This document outlines the procedure to configure the image registry to use Ceph RGW storage.

Prerequisites

  • You have access to the cluster as a user with the cluster-admin role.
  • You have access to the OpenShift Container Platform web console.
  • You installed the oc CLI.
  • You installed the OpenShift Data Foundation Operator to provide object storage and Ceph RGW object storage.

Procedure

  1. Create the object bucket claim using the ocs-storagecluster-ceph-rgw storage class. For example:

    cat <<EOF | oc apply -f -
    apiVersion: objectbucket.io/v1alpha1
    kind: ObjectBucketClaim
    metadata:
      name: rgwbucket
      namespace: openshift-storage 1
    spec:
      storageClassName: ocs-storagecluster-ceph-rgw
      generateBucketName: rgwbucket
    EOF
    1
    Alternatively, you can use the openshift-image-registry namespace.
  2. Get the bucket name by entering the following command:

    $ bucket_name=$(oc get obc -n openshift-storage rgwbucket -o jsonpath='{.spec.bucketName}')
  3. Get the AWS credentials by entering the following commands:

    $ AWS_ACCESS_KEY_ID=$(oc get secret -n openshift-storage rgwbucket -o jsonpath='{.data.AWS_ACCESS_KEY_ID}' | base64 --decode)
    $ AWS_SECRET_ACCESS_KEY=$(oc get secret -n openshift-storage rgwbucket -o jsonpath='{.data.AWS_SECRET_ACCESS_KEY}' | base64 --decode)
  4. Create the secret image-registry-private-configuration-user with the AWS credentials for the new bucket under openshift-image-registry project by entering the following command:

    $ oc create secret generic image-registry-private-configuration-user --from-literal=REGISTRY_STORAGE_S3_ACCESSKEY=${AWS_ACCESS_KEY_ID} --from-literal=REGISTRY_STORAGE_S3_SECRETKEY=${AWS_SECRET_ACCESS_KEY} --namespace openshift-image-registry
  5. Get the buckethost value by entering the following command:

    $ route_host=$(oc get objectbucket $bucket_name -n openshift-storage -o=jsonpath='{.spec.endpoint.bucketHost}')
  6. Create a config map that uses an ingress certificate by entering the following commands:

    $ oc extract secret/router-certs-default  -n openshift-ingress  --confirm
    $ oc create configmap image-registry-s3-bundle --from-file=ca-bundle.crt=./tls.crt  -n openshift-config
  7. Configure the image registry to use the Ceph RGW object storage by entering the following command:

    $ oc patch config.image/cluster -p '{"spec":{"managementState":"Managed","replicas":2,"storage":{"managementState":"Unmanaged","s3":{"bucket":'\"${bucket_name}\"',"region":"us-east-1","regionEndpoint":'\"https://${route_host}\"',"virtualHostedStyle":false,"encrypt":false,"trustedCA":{"name":"image-registry-s3-bundle"}}}}}' --type=merge

3.7.3.5. Configuring the Image Registry Operator to use Noobaa storage with Red Hat OpenShift Data Foundation

Red Hat OpenShift Data Foundation integrates multiple storage types that you can use with the OpenShift image registry:

  • Ceph, a shared and distributed file system and on-premises object storage
  • NooBaa, providing a Multicloud Object Gateway

This document outlines the procedure to configure the image registry to use Noobaa storage.

Prerequisites

  • You have access to the cluster as a user with the cluster-admin role.
  • You have access to the OpenShift Container Platform web console.
  • You installed the oc CLI.
  • You installed the OpenShift Data Foundation Operator to provide object storage and Noobaa object storage.

Procedure

  1. Create the object bucket claim using the openshift-storage.noobaa.io storage class. For example:

    cat <<EOF | oc apply -f -
    apiVersion: objectbucket.io/v1alpha1
    kind: ObjectBucketClaim
    metadata:
      name: noobaatest
      namespace: openshift-storage 1
    spec:
      storageClassName: openshift-storage.noobaa.io
      generateBucketName: noobaatest
    EOF
    1
    Alternatively, you can use the openshift-image-registry namespace.
  2. Get the bucket name by entering the following command:

    $ bucket_name=$(oc get obc -n openshift-storage noobaatest -o jsonpath='{.spec.bucketName}')
  3. Get the AWS credentials by entering the following commands:

    $ AWS_ACCESS_KEY_ID=$(oc get secret -n openshift-storage noobaatest -o yaml | grep -w "AWS_ACCESS_KEY_ID:" | head -n1 | awk '{print $2}' | base64 --decode)
    $ AWS_SECRET_ACCESS_KEY=$(oc get secret -n openshift-storage noobaatest -o yaml | grep -w "AWS_SECRET_ACCESS_KEY:" | head -n1 | awk '{print $2}' | base64 --decode)
  4. Create the secret image-registry-private-configuration-user with the AWS credentials for the new bucket under openshift-image-registry project by entering the following command:

    $ oc create secret generic image-registry-private-configuration-user --from-literal=REGISTRY_STORAGE_S3_ACCESSKEY=${AWS_ACCESS_KEY_ID} --from-literal=REGISTRY_STORAGE_S3_SECRETKEY=${AWS_SECRET_ACCESS_KEY} --namespace openshift-image-registry
  5. Get the route host by entering the following command:

    $ route_host=$(oc get route s3 -n openshift-storage -o=jsonpath='{.spec.host}')
  6. Create a config map that uses an ingress certificate by entering the following commands:

    $ oc extract secret/router-certs-default  -n openshift-ingress  --confirm
    $ oc create configmap image-registry-s3-bundle --from-file=ca-bundle.crt=./tls.crt  -n openshift-config
  7. Configure the image registry to use the Nooba object storage by entering the following command:

    $ oc patch config.image/cluster -p '{"spec":{"managementState":"Managed","replicas":2,"storage":{"managementState":"Unmanaged","s3":{"bucket":'\"${bucket_name}\"',"region":"us-east-1","regionEndpoint":'\"https://${route_host}\"',"virtualHostedStyle":false,"encrypt":false,"trustedCA":{"name":"image-registry-s3-bundle"}}}}}' --type=merge

3.7.4. Configuring the Image Registry Operator to use CephFS storage with Red Hat OpenShift Data Foundation

Red Hat OpenShift Data Foundation integrates multiple storage types that you can use with the OpenShift image registry:

  • Ceph, a shared and distributed file system and on-premises object storage
  • NooBaa, providing a Multicloud Object Gateway

This document outlines the procedure to configure the image registry to use CephFS storage.

Note

CephFS uses persistent volume claim (PVC) storage. It is not recommended to use PVCs for image registry storage if there are other options are available, such as Ceph RGW or Noobaa.

Prerequisites

  • You have access to the cluster as a user with the cluster-admin role.
  • You have access to the OpenShift Container Platform web console.
  • You installed the oc CLI.
  • You installed the OpenShift Data Foundation Operator to provide object storage and CephFS file storage.

Procedure

  1. Create a PVC to use the cephfs storage class. For example:

    cat <<EOF | oc apply -f -
    apiVersion: v1
    kind: PersistentVolumeClaim
    metadata:
     name: registry-storage-pvc
     namespace: openshift-image-registry
    spec:
     accessModes:
     - ReadWriteMany
     resources:
       requests:
         storage: 100Gi
     storageClassName: ocs-storagecluster-cephfs
    EOF
  2. Configure the image registry to use the CephFS file system storage by entering the following command:

    $ oc patch config.image/cluster -p '{"spec":{"managementState":"Managed","replicas":2,"storage":{"managementState":"Unmanaged","pvc":{"claim":"registry-storage-pvc"}}}}' --type=merge

3.7.5. Additional resources

3.8. Configuring the registry for Red Hat OpenShift Data Foundation

To configure the OpenShift image registry on bare metal and vSphere to use Red Hat OpenShift Data Foundation storage, you must install OpenShift Data Foundation and then configure image registry using Ceph or Noobaa.

3.8.1. Configuring the Image Registry Operator to use Ceph RGW storage with Red Hat OpenShift Data Foundation

Red Hat OpenShift Data Foundation integrates multiple storage types that you can use with the OpenShift image registry:

  • Ceph, a shared and distributed file system and on-premises object storage
  • NooBaa, providing a Multicloud Object Gateway

This document outlines the procedure to configure the image registry to use Ceph RGW storage.

Prerequisites

  • You have access to the cluster as a user with the cluster-admin role.
  • You have access to the OpenShift Container Platform web console.
  • You installed the oc CLI.
  • You installed the OpenShift Data Foundation Operator to provide object storage and Ceph RGW object storage.

Procedure

  1. Create the object bucket claim using the ocs-storagecluster-ceph-rgw storage class. For example:

    cat <<EOF | oc apply -f -
    apiVersion: objectbucket.io/v1alpha1
    kind: ObjectBucketClaim
    metadata:
      name: rgwbucket
      namespace: openshift-storage 1
    spec:
      storageClassName: ocs-storagecluster-ceph-rgw
      generateBucketName: rgwbucket
    EOF
    1
    Alternatively, you can use the openshift-image-registry namespace.
  2. Get the bucket name by entering the following command:

    $ bucket_name=$(oc get obc -n openshift-storage rgwbucket -o jsonpath='{.spec.bucketName}')
  3. Get the AWS credentials by entering the following commands:

    $ AWS_ACCESS_KEY_ID=$(oc get secret -n openshift-storage rgwbucket -o jsonpath='{.data.AWS_ACCESS_KEY_ID}' | base64 --decode)
    $ AWS_SECRET_ACCESS_KEY=$(oc get secret -n openshift-storage rgwbucket -o jsonpath='{.data.AWS_SECRET_ACCESS_KEY}' | base64 --decode)
  4. Create the secret image-registry-private-configuration-user with the AWS credentials for the new bucket under openshift-image-registry project by entering the following command:

    $ oc create secret generic image-registry-private-configuration-user --from-literal=REGISTRY_STORAGE_S3_ACCESSKEY=${AWS_ACCESS_KEY_ID} --from-literal=REGISTRY_STORAGE_S3_SECRETKEY=${AWS_SECRET_ACCESS_KEY} --namespace openshift-image-registry
  5. Get the buckethost value by entering the following command:

    $ route_host=$(oc get objectbucket $bucket_name -n openshift-storage -o=jsonpath='{.spec.endpoint.bucketHost}')
  6. Create a config map that uses an ingress certificate by entering the following commands:

    $ oc extract secret/router-certs-default  -n openshift-ingress  --confirm
    $ oc create configmap image-registry-s3-bundle --from-file=ca-bundle.crt=./tls.crt  -n openshift-config
  7. Configure the image registry to use the Ceph RGW object storage by entering the following command:

    $ oc patch config.image/cluster -p '{"spec":{"managementState":"Managed","replicas":2,"storage":{"managementState":"Unmanaged","s3":{"bucket":'\"${bucket_name}\"',"region":"us-east-1","regionEndpoint":'\"https://${route_host}\"',"virtualHostedStyle":false,"encrypt":false,"trustedCA":{"name":"image-registry-s3-bundle"}}}}}' --type=merge

3.8.2. Configuring the Image Registry Operator to use Noobaa storage with Red Hat OpenShift Data Foundation

Red Hat OpenShift Data Foundation integrates multiple storage types that you can use with the OpenShift image registry:

  • Ceph, a shared and distributed file system and on-premises object storage
  • NooBaa, providing a Multicloud Object Gateway

This document outlines the procedure to configure the image registry to use Noobaa storage.

Prerequisites

  • You have access to the cluster as a user with the cluster-admin role.
  • You have access to the OpenShift Container Platform web console.
  • You installed the oc CLI.
  • You installed the OpenShift Data Foundation Operator to provide object storage and Noobaa object storage.

Procedure

  1. Create the object bucket claim using the openshift-storage.noobaa.io storage class. For example:

    cat <<EOF | oc apply -f -
    apiVersion: objectbucket.io/v1alpha1
    kind: ObjectBucketClaim
    metadata:
      name: noobaatest
      namespace: openshift-storage 1
    spec:
      storageClassName: openshift-storage.noobaa.io
      generateBucketName: noobaatest
    EOF
    1
    Alternatively, you can use the openshift-image-registry namespace.
  2. Get the bucket name by entering the following command:

    $ bucket_name=$(oc get obc -n openshift-storage noobaatest -o jsonpath='{.spec.bucketName}')
  3. Get the AWS credentials by entering the following commands:

    $ AWS_ACCESS_KEY_ID=$(oc get secret -n openshift-storage noobaatest -o yaml | grep -w "AWS_ACCESS_KEY_ID:" | head -n1 | awk '{print $2}' | base64 --decode)
    $ AWS_SECRET_ACCESS_KEY=$(oc get secret -n openshift-storage noobaatest -o yaml | grep -w "AWS_SECRET_ACCESS_KEY:" | head -n1 | awk '{print $2}' | base64 --decode)
  4. Create the secret image-registry-private-configuration-user with the AWS credentials for the new bucket under openshift-image-registry project by entering the following command:

    $ oc create secret generic image-registry-private-configuration-user --from-literal=REGISTRY_STORAGE_S3_ACCESSKEY=${AWS_ACCESS_KEY_ID} --from-literal=REGISTRY_STORAGE_S3_SECRETKEY=${AWS_SECRET_ACCESS_KEY} --namespace openshift-image-registry
  5. Get the route host by entering the following command:

    $ route_host=$(oc get route s3 -n openshift-storage -o=jsonpath='{.spec.host}')
  6. Create a config map that uses an ingress certificate by entering the following commands:

    $ oc extract secret/router-certs-default  -n openshift-ingress  --confirm
    $ oc create configmap image-registry-s3-bundle --from-file=ca-bundle.crt=./tls.crt  -n openshift-config
  7. Configure the image registry to use the Nooba object storage by entering the following command:

    $ oc patch config.image/cluster -p '{"spec":{"managementState":"Managed","replicas":2,"storage":{"managementState":"Unmanaged","s3":{"bucket":'\"${bucket_name}\"',"region":"us-east-1","regionEndpoint":'\"https://${route_host}\"',"virtualHostedStyle":false,"encrypt":false,"trustedCA":{"name":"image-registry-s3-bundle"}}}}}' --type=merge

3.8.3. Configuring the Image Registry Operator to use CephFS storage with Red Hat OpenShift Data Foundation

Red Hat OpenShift Data Foundation integrates multiple storage types that you can use with the OpenShift image registry:

  • Ceph, a shared and distributed file system and on-premises object storage
  • NooBaa, providing a Multicloud Object Gateway

This document outlines the procedure to configure the image registry to use CephFS storage.

Note

CephFS uses persistent volume claim (PVC) storage. It is not recommended to use PVCs for image registry storage if there are other options are available, such as Ceph RGW or Noobaa.

Prerequisites

  • You have access to the cluster as a user with the cluster-admin role.
  • You have access to the OpenShift Container Platform web console.
  • You installed the oc CLI.
  • You installed the OpenShift Data Foundation Operator to provide object storage and CephFS file storage.

Procedure

  1. Create a PVC to use the cephfs storage class. For example:

    cat <<EOF | oc apply -f -
    apiVersion: v1
    kind: PersistentVolumeClaim
    metadata:
     name: registry-storage-pvc
     namespace: openshift-image-registry
    spec:
     accessModes:
     - ReadWriteMany
     resources:
       requests:
         storage: 100Gi
     storageClassName: ocs-storagecluster-cephfs
    EOF
  2. Configure the image registry to use the CephFS file system storage by entering the following command:

    $ oc patch config.image/cluster -p '{"spec":{"managementState":"Managed","replicas":2,"storage":{"managementState":"Unmanaged","pvc":{"claim":"registry-storage-pvc"}}}}' --type=merge

3.8.4. Additional resources

3.9. Configuring the registry for Nutanix

By following the steps outlined in this documentation, users can optimize container image distribution, security, and access controls, enabling a robust foundation for Nutanix applications on OpenShift Container Platform

3.9.1. Image registry removed during installation

On platforms that do not provide shareable object storage, the OpenShift Image Registry Operator bootstraps itself as Removed. This allows openshift-installer to complete installations on these platform types.

After installation, you must edit the Image Registry Operator configuration to switch the managementState from Removed to Managed.

3.9.2. Changing the image registry’s management state

To start the image registry, you must change the Image Registry Operator configuration’s managementState from Removed to Managed.

Procedure

  • Change managementState Image Registry Operator configuration from Removed to Managed. For example:

    $ oc patch configs.imageregistry.operator.openshift.io cluster --type merge --patch '{"spec":{"managementState":"Managed"}}'

3.9.3. Image registry storage configuration

The Image Registry Operator is not initially available for platforms that do not provide default storage. After installation, you must configure your registry to use storage so that the Registry Operator is made available.

Instructions are shown for configuring a persistent volume, which is required for production clusters. Where applicable, instructions are shown for configuring an empty directory as the storage location, which is available for only non-production clusters.

Additional instructions are provided for allowing the image registry to use block storage types by using the Recreate rollout strategy during upgrades.

3.9.3.1. Configuring registry storage for Nutanix

As a cluster administrator, following installation you must configure your registry to use storage.

Prerequisites

  • You have access to the cluster as a user with the cluster-admin role.
  • You have a cluster on Nutanix.
  • You have provisioned persistent storage for your cluster, such as Red Hat OpenShift Data Foundation.

    Important

    OpenShift Container Platform supports ReadWriteOnce access for image registry storage when you have only one replica. ReadWriteOnce access also requires that the registry uses the Recreate rollout strategy. To deploy an image registry that supports high availability with two or more replicas, ReadWriteMany access is required.

  • You must have 100 Gi capacity.

Procedure

  1. To configure your registry to use storage, change the spec.storage.pvc in the configs.imageregistry/cluster resource.

    Note

    When you use shared storage, review your security settings to prevent outside access.

  2. Verify that you do not have a registry pod:

    $ oc get pod -n openshift-image-registry -l docker-registry=default

    Example output

    No resourses found in openshift-image-registry namespace

    Note

    If you do have a registry pod in your output, you do not need to continue with this procedure.

  3. Check the registry configuration:

    $ oc edit configs.imageregistry.operator.openshift.io

    Example output

    storage:
      pvc:
        claim: 1

    1
    Leave the claim field blank to allow the automatic creation of an image-registry-storage persistent volume claim (PVC). The PVC is generated based on the default storage class. However, be aware that the default storage class might provide ReadWriteOnce (RWO) volumes, such as a RADOS Block Device (RBD), which can cause issues when you replicate to more than one replica.
  4. Check the clusteroperator status:

    $ oc get clusteroperator image-registry

    Example output

    NAME             VERSION                              AVAILABLE   PROGRESSING   DEGRADED   SINCE   MESSAGE
    image-registry   4.13                                  True        False         False      6h50m

3.9.3.2. Configuring storage for the image registry in non-production clusters

You must configure storage for the Image Registry Operator. For non-production clusters, you can set the image registry to an empty directory. If you do so, all images are lost if you restart the registry.

Procedure

  • To set the image registry storage to an empty directory:

    $ oc patch configs.imageregistry.operator.openshift.io cluster --type merge --patch '{"spec":{"storage":{"emptyDir":{}}}}'
    Warning

    Configure this option for only non-production clusters.

    If you run this command before the Image Registry Operator initializes its components, the oc patch command fails with the following error:

    Error from server (NotFound): configs.imageregistry.operator.openshift.io "cluster" not found

    Wait a few minutes and run the command again.

3.9.3.3. Configuring block registry storage for Nutanix volumes

To allow the image registry to use block storage types such as Nutanix volumes during upgrades as a cluster administrator, you can use the Recreate rollout strategy.

Important

Block storage volumes, or block persistent volumes, are supported but not recommended for use with the image registry on production clusters. An installation where the registry is configured on block storage is not highly available because the registry cannot have more than one replica.

If you choose to use a block storage volume with the image registry, you must use a filesystem persistent volume claim (PVC).

Procedure

  1. Enter the following command to set the image registry storage as a block storage type, patch the registry so that it uses the Recreate rollout strategy, and runs with only one (1) replica:

    $ oc patch config.imageregistry.operator.openshift.io/cluster --type=merge -p '{"spec":{"rolloutStrategy":"Recreate","replicas":1}}'
  2. Provision the PV for the block storage device, and create a PVC for that volume. The requested block volume uses the ReadWriteOnce (RWO) access mode.

    1. Create a pvc.yaml file with the following contents to define a Nutanix PersistentVolumeClaim object:

      kind: PersistentVolumeClaim
      apiVersion: v1
      metadata:
        name: image-registry-storage 1
        namespace: openshift-image-registry 2
      spec:
        accessModes:
        - ReadWriteOnce 3
        resources:
          requests:
            storage: 100Gi 4
      1
      A unique name that represents the PersistentVolumeClaim object.
      2
      The namespace for the PersistentVolumeClaim object, which is openshift-image-registry.
      3
      The access mode of the persistent volume claim. With ReadWriteOnce, the volume can be mounted with read and write permissions by a single node.
      4
      The size of the persistent volume claim.
    2. Enter the following command to create the PersistentVolumeClaim object from the file:

      $ oc create -f pvc.yaml -n openshift-image-registry
  3. Enter the following command to edit the registry configuration so that it references the correct PVC:

    $ oc edit config.imageregistry.operator.openshift.io -o yaml

    Example output

    storage:
      pvc:
        claim: 1

    1
    By creating a custom PVC, you can leave the claim field blank for the default automatic creation of an image-registry-storage PVC.

3.9.3.4. Configuring the Image Registry Operator to use Ceph RGW storage with Red Hat OpenShift Data Foundation

Red Hat OpenShift Data Foundation integrates multiple storage types that you can use with the OpenShift image registry:

  • Ceph, a shared and distributed file system and on-premises object storage
  • NooBaa, providing a Multicloud Object Gateway

This document outlines the procedure to configure the image registry to use Ceph RGW storage.

Prerequisites

  • You have access to the cluster as a user with the cluster-admin role.
  • You have access to the OpenShift Container Platform web console.
  • You installed the oc CLI.
  • You installed the OpenShift Data Foundation Operator to provide object storage and Ceph RGW object storage.

Procedure

  1. Create the object bucket claim using the ocs-storagecluster-ceph-rgw storage class. For example:

    cat <<EOF | oc apply -f -
    apiVersion: objectbucket.io/v1alpha1
    kind: ObjectBucketClaim
    metadata:
      name: rgwbucket
      namespace: openshift-storage 1
    spec:
      storageClassName: ocs-storagecluster-ceph-rgw
      generateBucketName: rgwbucket
    EOF
    1
    Alternatively, you can use the openshift-image-registry namespace.
  2. Get the bucket name by entering the following command:

    $ bucket_name=$(oc get obc -n openshift-storage rgwbucket -o jsonpath='{.spec.bucketName}')
  3. Get the AWS credentials by entering the following commands:

    $ AWS_ACCESS_KEY_ID=$(oc get secret -n openshift-storage rgwbucket -o jsonpath='{.data.AWS_ACCESS_KEY_ID}' | base64 --decode)
    $ AWS_SECRET_ACCESS_KEY=$(oc get secret -n openshift-storage rgwbucket -o jsonpath='{.data.AWS_SECRET_ACCESS_KEY}' | base64 --decode)
  4. Create the secret image-registry-private-configuration-user with the AWS credentials for the new bucket under openshift-image-registry project by entering the following command:

    $ oc create secret generic image-registry-private-configuration-user --from-literal=REGISTRY_STORAGE_S3_ACCESSKEY=${AWS_ACCESS_KEY_ID} --from-literal=REGISTRY_STORAGE_S3_SECRETKEY=${AWS_SECRET_ACCESS_KEY} --namespace openshift-image-registry
  5. Get the buckethost value by entering the following command:

    $ route_host=$(oc get objectbucket $bucket_name -n openshift-storage -o=jsonpath='{.spec.endpoint.bucketHost}')
  6. Create a config map that uses an ingress certificate by entering the following commands:

    $ oc extract secret/router-certs-default  -n openshift-ingress  --confirm
    $ oc create configmap image-registry-s3-bundle --from-file=ca-bundle.crt=./tls.crt  -n openshift-config
  7. Configure the image registry to use the Ceph RGW object storage by entering the following command:

    $ oc patch config.image/cluster -p '{"spec":{"managementState":"Managed","replicas":2,"storage":{"managementState":"Unmanaged","s3":{"bucket":'\"${bucket_name}\"',"region":"us-east-1","regionEndpoint":'\"https://${route_host}\"',"virtualHostedStyle":false,"encrypt":false,"trustedCA":{"name":"image-registry-s3-bundle"}}}}}' --type=merge

3.9.3.5. Configuring the Image Registry Operator to use Noobaa storage with Red Hat OpenShift Data Foundation

Red Hat OpenShift Data Foundation integrates multiple storage types that you can use with the OpenShift image registry:

  • Ceph, a shared and distributed file system and on-premises object storage
  • NooBaa, providing a Multicloud Object Gateway

This document outlines the procedure to configure the image registry to use Noobaa storage.

Prerequisites

  • You have access to the cluster as a user with the cluster-admin role.
  • You have access to the OpenShift Container Platform web console.
  • You installed the oc CLI.
  • You installed the OpenShift Data Foundation Operator to provide object storage and Noobaa object storage.

Procedure

  1. Create the object bucket claim using the openshift-storage.noobaa.io storage class. For example:

    cat <<EOF | oc apply -f -
    apiVersion: objectbucket.io/v1alpha1
    kind: ObjectBucketClaim
    metadata:
      name: noobaatest
      namespace: openshift-storage 1
    spec:
      storageClassName: openshift-storage.noobaa.io
      generateBucketName: noobaatest
    EOF
    1
    Alternatively, you can use the openshift-image-registry namespace.
  2. Get the bucket name by entering the following command:

    $ bucket_name=$(oc get obc -n openshift-storage noobaatest -o jsonpath='{.spec.bucketName}')
  3. Get the AWS credentials by entering the following commands:

    $ AWS_ACCESS_KEY_ID=$(oc get secret -n openshift-storage noobaatest -o yaml | grep -w "AWS_ACCESS_KEY_ID:" | head -n1 | awk '{print $2}' | base64 --decode)
    $ AWS_SECRET_ACCESS_KEY=$(oc get secret -n openshift-storage noobaatest -o yaml | grep -w "AWS_SECRET_ACCESS_KEY:" | head -n1 | awk '{print $2}' | base64 --decode)
  4. Create the secret image-registry-private-configuration-user with the AWS credentials for the new bucket under openshift-image-registry project by entering the following command:

    $ oc create secret generic image-registry-private-configuration-user --from-literal=REGISTRY_STORAGE_S3_ACCESSKEY=${AWS_ACCESS_KEY_ID} --from-literal=REGISTRY_STORAGE_S3_SECRETKEY=${AWS_SECRET_ACCESS_KEY} --namespace openshift-image-registry
  5. Get the route host by entering the following command:

    $ route_host=$(oc get route s3 -n openshift-storage -o=jsonpath='{.spec.host}')
  6. Create a config map that uses an ingress certificate by entering the following commands:

    $ oc extract secret/router-certs-default  -n openshift-ingress  --confirm
    $ oc create configmap image-registry-s3-bundle --from-file=ca-bundle.crt=./tls.crt  -n openshift-config
  7. Configure the image registry to use the Nooba object storage by entering the following command:

    $ oc patch config.image/cluster -p '{"spec":{"managementState":"Managed","replicas":2,"storage":{"managementState":"Unmanaged","s3":{"bucket":'\"${bucket_name}\"',"region":"us-east-1","regionEndpoint":'\"https://${route_host}\"',"virtualHostedStyle":false,"encrypt":false,"trustedCA":{"name":"image-registry-s3-bundle"}}}}}' --type=merge

3.9.4. Configuring the Image Registry Operator to use CephFS storage with Red Hat OpenShift Data Foundation

Red Hat OpenShift Data Foundation integrates multiple storage types that you can use with the OpenShift image registry:

  • Ceph, a shared and distributed file system and on-premises object storage
  • NooBaa, providing a Multicloud Object Gateway

This document outlines the procedure to configure the image registry to use CephFS storage.

Note

CephFS uses persistent volume claim (PVC) storage. It is not recommended to use PVCs for image registry storage if there are other options are available, such as Ceph RGW or Noobaa.

Prerequisites

  • You have access to the cluster as a user with the cluster-admin role.
  • You have access to the OpenShift Container Platform web console.
  • You installed the oc CLI.
  • You installed the OpenShift Data Foundation Operator to provide object storage and CephFS file storage.

Procedure

  1. Create a PVC to use the cephfs storage class. For example:

    cat <<EOF | oc apply -f -
    apiVersion: v1
    kind: PersistentVolumeClaim
    metadata:
     name: registry-storage-pvc
     namespace: openshift-image-registry
    spec:
     accessModes:
     - ReadWriteMany
     resources:
       requests:
         storage: 100Gi
     storageClassName: ocs-storagecluster-cephfs
    EOF
  2. Configure the image registry to use the CephFS file system storage by entering the following command:

    $ oc patch config.image/cluster -p '{"spec":{"managementState":"Managed","replicas":2,"storage":{"managementState":"Unmanaged","pvc":{"claim":"registry-storage-pvc"}}}}' --type=merge

3.9.5. Additional resources