Chapter 3. Configuring registry storage

3.1. Configuring registry storage for AWS user-provisioned infrastructure

3.1.1. Configuring a secret for the Image Registry Operator

In addition to the configs.imageregistry.operator.openshift.io and ConfigMap resources, configuration is provided to the Operator by a separate secret resource located within the openshift-image-registry namespace.

The image-registry-private-configuration-user secret provides credentials needed for storage access and management. It overrides the default credentials used by the Operator, if default credentials were found.

For S3 on AWS storage the secret is expected to contain two keys:

  • REGISTRY_STORAGE_S3_ACCESSKEY
  • REGISTRY_STORAGE_S3_SECRETKEY

Procedure

  • Create an OpenShift Container Platform secret that contains the required keys. 

    $ oc create secret generic image-registry-private-configuration-user --from-literal=REGISTRY_STORAGE_S3_ACCESSKEY=myaccesskey --from-literal=REGISTRY_STORAGE_S3_SECRETKEY=mysecretkey --namespace openshift-image-registry

3.1.2. Configuring registry storage for AWS with user-provisioned infrastructure

During installation, your cloud credentials are sufficient to create an S3 bucket and the Registry Operator will automatically configure storage.

If the Registry Operator cannot create an S3 bucket, and automatically configure storage, you can create an S3 bucket and configure storage with the following procedure.

Prerequisites

  • A cluster on AWS with user-provisioned infrastructure.

  • For S3 on AWS storage the secret is expected to contain two keys:

    • REGISTRY_STORAGE_S3_ACCESSKEY
    • REGISTRY_STORAGE_S3_SECRETKEY

Procedure

Use the following procedure if the Registry Operator cannot create an S3 bucket and automatically configure storage.

  1. Set up a Bucket Lifecycle Policy to abort incomplete multipart uploads that are one day old.

  2. Fill in the storage configuration in configs.imageregistry.operator.openshift.io/cluster: 

    $ oc edit configs.imageregistry.operator.openshift.io/cluster

storage:
  s3:
    bucket: <bucket-name>
    region: <region-name>
Warning

To secure your registry images in AWS, block public access to the S3 bucket.

3.1.3. Image Registry Operator configuration parameters for AWS S3

The following configuration parameters are available for AWS S3 registry storage.

ParameterDescription

bucket

Bucket is the bucket name in which you want to store the registry’s data. It is optional and is generated if not provided.

region

Region is the AWS region in which your bucket exists. It is optional and is set based on the installed AWS Region.

regionEndpoint

RegionEndpoint is the endpoint for S3 compatible storage services. It is optional and defaults based on the Region that is provided.

encrypt

Encrypt specifies whether or not the registry stores the image in encrypted format. It is optional and defaults to false.

keyID

KeyID is the KMS key ID to use for encryption. It is optional. Encrypt must be true, or this parameter is ignored.

ImageRegistryConfigStorageS3CloudFront

CloudFront configures Amazon Cloudfront as the storage middleware in a registry. It is optional.

3.2. Configuring registry storage for GCP user-provisioned infrastructure

3.2.1. Configuring a secret for the Image Registry Operator

In addition to the configs.imageregistry.operator.openshift.io and ConfigMap resources, configuration is provided to the Operator by a separate secret resource located within the openshift-image-registry namespace.

The image-registry-private-configuration-user secret provides credentials needed for storage access and management. It overrides the default credentials used by the Operator, if default credentials were found.

For GCS on GCP storage the secret is expected to contain one key whose value is the contents of a credentials file provided by GCP:

  • REGISTRY_STORAGE_GCS_KEYFILE

Procedure

  • Create an OpenShift Container Platform secret that contains the required keys. 

    $ oc create secret generic image-registry-private-configuration-user --from-file=REGISTRY_STORAGE_GCS_KEYFILE=<path_to_keyfile> --namespace openshift-image-registry

3.2.2. Registry storage for GCP with user-provisioned infrastructure

You must set up the storage medium manually and configure the settings in the registry CRD.

Prerequisites

  • A cluster on GCP with user-provisioned infrastructure.
  • To configure registry storage for GCP, you need to provide Registry Operator cloud credentials.

  • For GCS on GCP storage the secret is expected to contain one key whose value is the contents of a credentials file provided by GCP:

    • REGISTRY_STORAGE_GCS_KEYFILE

3.2.3. Image Registry Operator configuration parameters for GCP GCS

Procedure

The following configuration parameters are available for GCP GCS registry storage.

ParameterDescription

bucket

Bucket is the bucket name in which you want to store the registry’s data. It is optional and is generated if not provided.

region

Region is the GCS location in which your bucket exists. It is optional and is set based on the installed GCS Region.

projectID

ProjectID is the Project ID of the GCP project that this bucket should be associated with. It is optional.

keyID

KeyID is the KMS key ID to use for encryption. It is optional because buckets are encrypted by default on GCP. This allows for the use of a custom encryption key.

3.3. Configuring registry storage for bare metal

3.3.1. Configuring registry storage for bare metal

As a cluster administrator, following installation you must configure your registry to use storage.

Prerequisites

  • Cluster administrator permissions.
  • A cluster on bare metal.
  • Provision persistent storage for your cluster, such as Red Hat OpenShift Container Storage. To deploy a private image registry, your storage must provide ReadWriteMany access mode.
  • Must have "100Gi" capacity.

Procedure

  1. To configure your registry to use storage, change the spec.storage.pvc in the configs.imageregistry/cluster resource.

  2. Verify you do not have a registry Pod: 

    $ oc get pod -n openshift-image-registry
    Note

    If the storage type is emptyDIR, the replica number cannot be greater than 1. If the storage type is NFS, and you want to scale up the registry Pod by setting replica>1 you must enable the no_wdelay mount option. For example: 

    # cat /etc/exports
/mnt/data *(rw,sync,no_wdelay,no_root_squash,insecure,fsid=0)
sh-4.3# exportfs -rv
exporting *:/mnt/data

  3. Check the registry configuration: 

    $ oc edit configs.imageregistry.operator.openshift.io

storage:
  pvc:
    claim:

    Leave the claim field blank to allow the automatic creation of an image-registry-storage PVC.

  4. Check the clusteroperator status: 

    $ oc get clusteroperator image-registry

3.3.2. Additional resources

For more details on configuring registry storage for bare metal, see Recommended configurable storage technology.

3.4. Configuring registry storage for vSphere

3.4.1. Configuring registry storage for VMware vSphere

As a cluster administrator, following installation you must configure your registry to use storage.

Prerequisites

  • Cluster administrator permissions.
  • A cluster on VMware vSphere.

  • A provisioned persistent volume (PV) with ReadWriteMany access mode, such as NFS.

    Important

    vSphere volumes do not support the ReadWriteMany access mode. You must use a different storage backend, such as NFS, to configure the registry storage.

  • Must have "100Gi" capacity.

Procedure

  1. To configure your registry to use storage, change the spec.storage.pvc in the configs.imageregistry/cluster resource.

  2. Verify you do not have a registry Pod: 

    $ oc get pod -n openshift-image-registry
    Note

    If the storage type is emptyDIR, the replica number cannot be greater than 1. If the storage type is NFS, and you want to scale up the registry Pod by setting replica>1 you must enable the no_wdelay mount option. For example: 

    # cat /etc/exports
/mnt/data *(rw,sync,no_wdelay,no_root_squash,insecure,fsid=0)
sh-4.3# exportfs -rv
exporting *:/mnt/data

  3. Check the registry configuration: 

    $ oc edit configs.imageregistry.operator.openshift.io

storage:
  pvc:
    claim:

    Leave the claim field blank to allow the automatic creation of an image-registry-storage PVC.

  4. Optional: Add a new storage class to a PV:

    1. Create the PV: 

      $ oc create -f -
      apiVersion: v1
kind: PersistentVolume
metadata:
  name: image-registry-pv
spec:
  accessModes:
    ReadWriteMany
    capacity:
      storage: 100Gi
  nfs:
    path: /registry
    server: 172.16.231.181
  persistentVolumeReclaimPolicy: Retain
  storageClassName: nfs01
      $ oc get pv

    2. Create the PVC: 

      $ oc create -n openshift-image-registry -f -
      apiVersion: "v1"
kind: "PersistentVolumeClaim"
metadata:
  name: "image-registry-pvc"
spec:
  accessModes:
    - ReadWriteMany
  resources:
    requests:
      storage: 100Gi
  storageClassName: nfs01
  volumeMode: Filesystem
      $ oc get pvc -n openshift-image-registry

      Finally, add the name of your PVC: 

      $ oc edit configs.imageregistry.operator.openshift.io -o yaml
      storage:
  pvc:
    claim: image-registry-pvc 1
      1
      Creating a custom PVC allows you to leave the claim field blank for default automatic creation of an image-registry-storage PVC.

  5. Check the clusteroperator status: 

    $ oc get clusteroperator image-registry

3.4.2. Additional resources

For more details on configuring registry storage for vSphere, see Recommended configurable storage technology.