6.2. Converting the Openshift Container Platform Registry with Container-Native Storage

This section provides the steps to create a Red Hat Gluster Storage volume and use it to provide storage for the integrated registry.
Setting up a Red Hat Gluster Storage Persistent Volume

Execute the following commands to create a Red Hat Gluster Storage volume to store the registry data and create a persistent volume.

Note

The commands must be executed in the default project.
  1. Login to the default project:
    # oc project default
    For example:
    # oc project default
    Now using project "default" on server "https://cns30.rh73:8443"
  2. Execute the following command to create the gluster-registry-endpoints.yaml file:
    # oc get endpoints heketi-storage-endpoints -o yaml --namespace=storage-project > gluster-registry-endpoints.yaml

    Note

    You must create an endpoint for each project from which you want to utilize the Red Hat Gluster Storage registry. Hence, you will have a service and an endpoint in both the default project and the new project (storage-project) created in earlier steps.
  3. Edit the gluster-registry-endpoints.yaml file. Remove all the metadata except for name, leaving everything else the same.
    # cat gluster-registry-endpoints.yaml
    apiVersion: v1
    kind: Endpoints
    metadata:
      name: gluster-registry-endpoints
    subsets:
    - addresses:
      - ip: 192.168.124.114
      - ip: 192.168.124.52
      - ip: 192.168.124.83
      ports:
      - port: 1
        protocol: TCP
  4. Execute the following command to create the endpoint:
    # oc create -f gluster-registry-endpoints.yaml
    endpoints "gluster-registry-endpoints" created
  5. To verify the creation of the endpoint, execute the following command:
    # oc get endpoints
    NAME                       ENDPOINTS                                                                 AGE
    docker-registry            10.129.0.8:5000,10.130.0.5:5000                                           28d
    gluster-registry-endpoints  192.168.124.114:1,192.168.124.52:1,192.168.124.83:1                       10s
    kubernetes                 192.168.124.250:8443,192.168.124.250:8053,192.168.124.250:8053            28d
    registry-console           10.131.0.6:9090                                                           28d
    router                     192.168.124.114:443,192.168.124.83:443,192.168.124.114:1936 + 3 more...   28d
  6. Execute the following command to create the gluster-registry-service.yaml file:
    # oc get services heketi-storage-endpoints -o yaml --namespace=storage-project > gluster-registry-service.yaml
  7. Edit the gluster-registry-service.yaml file. Remove all the metadata except for name. Also, remove the specific cluster IP addresses:
    # cat gluster-registry-service.yaml
    apiVersion: v1
    kind: Service
    metadata:
      name: gluster-registry-service
    spec:
      ports:
      - port: 1
        protocol: TCP
        targetPort: 1
      sessionAffinity: None
      type: ClusterIP
    status:
      loadBalancer: {}
  8. Execute the following command to create the service:
    # oc create -f gluster-registry-service.yaml
    services "gluster-registry-service" created
  9. Execute the following command to verify if the service are running:
    # oc get services
    NAME                       CLUSTER-IP       EXTERNAL-IP   PORT(S)                   AGE
    docker-registry            172.30.197.118   <none>        5000/TCP                  28d
    gluster-registry-service   172.30.0.183     <none>        1/TCP                     6s
    kubernetes                 172.30.0.1       <none>        443/TCP,53/UDP,53/TCP     29d
    registry-console           172.30.146.178   <none>        9000/TCP                  28d
    router                     172.30.232.238   <none>        80/TCP,443/TCP,1936/TCP   28d
  10. Execute the following command to obtain the fsGroup GID of the existing docker-registry pods:
    # export GID=$(oc get po --selector="docker-registry=default" -o go-template --template='{{printf "%.0f" ((index .items 0).spec.securityContext.fsGroup)}}')
    
  11. Execute the following command to create a volume
    # heketi-cli volume create --size=5 --name=gluster-registry-volume --gid=${GID}
  12. Create the persistent volume file for the Red Hat Gluster Storage volume:
    # cat gluster-registry-volume.yaml
    kind: PersistentVolume
    apiVersion: v1
    metadata:
      name: gluster-registry-volume
      labels:
        glusterfs: registry-volume
    spec:
      capacity:
        storage: 5Gi
      glusterfs:
        endpoints: gluster-registry-endpoints
        path: gluster-registry-volume
      accessModes:
      - ReadWriteMany
      persistentVolumeReclaimPolicy: Retain
    
  13. Execute the following command to create the persistent volume:
    # oc create -f gluster-registry-volume.yaml
  14. Execute the following command to verify and get the details of the created persistent volume:
    # oc get pv/gluster-registry-volume
    NAME                      CAPACITY   ACCESSMODES   RECLAIMPOLICY   STATUS      CLAIM     REASON    AGE
    gluster-registry-volume   5Gi        RWX           Retain          Available                       21m
  15. Create a new persistent volume claim. Following is a sample Persistent Volume Claim that will be used to replace the existing registry-storage volume claim.
    # cat gluster-registry-claim.yaml
    apiVersion: v1
    kind: PersistentVolumeClaim
    metadata:
      name: gluster-registry-claim
    spec:
      accessModes:
       - ReadWriteMany
      resources:
        requests:
          storage: 5Gi
      selector:
          matchLabels:
              glusterfs: registry-volume
  16. Create the persistent volume claim by executing the following command:
    # oc create -f gluster-registry-claim.yaml
    For example:
    # oc create -f gluster-registry-claim.yaml
    persistentvolumeclaim "gluster-registry-claim" created
  17. Execute the following command to verify if the claim is bound:
    # oc get pvc/gluster-registry-claim
    For example:
    # oc get pvc/gluster-registry-claim
    NAME                     STATUS    VOLUME                    CAPACITY   ACCESSMODES   AGE
    gluster-registry-claim   Bound     gluster-registry-volume   5Gi        RWX           22s
  18. If you want to migrate the data from the old registry to the Red Hat Gluster Storage registry, then execute the following commands:

    Note

    These steps are optional.
    1. Make the old registry readonly by executing the following command:
      # oc set env dc/docker-registry REGISTRY_STORAGE_MAINTENANCE_READONLY_ENABLED=true
    2. Add the Red Hat Gluster Storage registry to the old registry deployment configuration (dc) by executing the following command:
      # oc volume dc/docker-registry --add --name=gluster-registry-storage -m /gluster-registry -t pvc --claim-name=gluster-registry-claim
    3. Save the Registry pod name by executing the following command:
      # export REGISTRY_POD=$(oc get po --selector="docker-registry=default" -o go-template --template='{{printf "%s" ((index .items 0).metadata.name)}}')
    4. Run rsync of data from old registry to the Red Hat Gluster Storage registry by executing the following command:
      # oc rsync $REGISTRY_POD:/registry/ $REGISTRY_POD:/gluster-registry/
    5. Remove the Red Hat Gluster Storage registry form the old dc registry by executing the following command:
      # oc volume dc/docker-registry --remove --name=gluster-registry-storage
    6. Swap the existing registry storage volume for the new Red Hat Gluster Storage volume by executing the following command:
      # oc volume dc/docker-registry --add --name=registry-storage -t pvc --claim-name=gluster-registry-claim --overwrite
    7. Make the registry read write by executing the following command:
      # oc set env dc/docker-registry REGISTRY_STORAGE_MAINTENANCE_READONLY_ENABLED-