Chapter 9. Integrate Red Hat Quay into OpenShift with the Bridge Operator

Using the Quay Bridge Operator, you can replace the integrated container registry in OpenShift with a Red Hat Quay registry. By doing this, your integrated OpenShift registry becomes a highly available, enterprise-grade Red Hat Quay registry with enhanced role based access control (RBAC) features.

The primary goals of the Bridge Operator is to duplicate the features of the integrated OpenShift registry in the new Red Hat Quay registry. The features enabled with this Operator include:

  • Synchronizing OpenShift namespaces as Red Hat Quay organizations.

    • Creating Robot accounts for each default namespace service account
    • Creating Secrets for each created Robot Account (associating each Robot Secret to a Service Account as Mountable and Image Pull Secret)
    • Synchronizing OpenShift ImageStreams as Quay Repositories
  • Automatically rewriting new Builds making use of ImageStreams to output to Red Hat Quay
  • Automatically importing an ImageStream tag once a build completes

Using this procedure with the Quay Bridge Operator, you enable bi-directional communication between your Red Hat Quay and OpenShift clusters.

Warning

You cannot have more than one OpenShift Container Platform cluster pointing to the same Red Hat Quay instance from a Quay Bridge Operator. If you did, it would prevent you from creating namespaces of the same name on the two clusters.

9.1. Running the Quay Bridge Operator

9.1.1. Prerequisites

Before setting up the Bridge Operator, have the following in place:

  • An existing Red Hat Quay environment for which you have superuser permissions
  • A Red Hat OpenShift Container Platform environment (4.2 or later is recommended) for which you have cluster administrator permissions
  • An OpenShift command line tool (oc command)

9.1.2. Setting up and configuring OpenShift and Red Hat Quay

Both Red Hat Quay and OpenShift configuration is required:

9.1.2.1. Red Hat Quay setup

Create a dedicated Red Hat Quay organization, and from a new application you create within that organization, generate an OAuth token to be used with the Quay Bridge Operator in OpenShift

  1. Log in to Red Hat Quay as a user with superuser access and select the organization for which the external application will be configured.
  2. In the left navigation, select Applications.
  3. Select Create New Application and entering a name for the new application (for example, openshift).
  4. With the new application displayed, select it.
  5. In the left navigation, select Generate Token to create a new OAuth2 token.
  6. Select all checkboxes to grant the access needed for the integration.
  7. Review the assigned permissions and then select Authorize Application, then confirm it.
  8. Copy and save the generated Access Token that appears to use in the next section.

9.1.2.2. OpenShift Setup

Setting up OpenShift for the Quay Bridge Operator requires several steps, including:

  • Creating an OpenShift secret: Using the OAuth token created earlier in Quay, create an OpenShift secret.
  • Adding MutatingWebhookConfiguration support: To support Red Hat Quay integration with OpenShift, any new Build requests should be intercepted so that the output can be modified to target Red Hat Quay instead of OpenShift’s integrated registry.

Support for dynamic interception of API requests that are performed as part of OpenShift’s typical build process is facilitated through a MutatingWebhookConfiguration. A MutatingWebhookConfiguration allows for invoking an API running within a project on OpenShift when certain API requests are received.

Kubernetes requires that the webhook endpoint is secured via SSL using a certificate that makes use of the certificate authority for the cluster. Fortunately, OpenShift provides support for generating a certificate signed by the cluster.

  1. Using the OpenShift oc command line tool, log in to OpenShift as a cluster administrator.
  2. Choose an OpenShift namespace to use, such as openshift-operators or create a new one.

  3. Create an OpenShift secret, replacing <access_token> with the Access Token obtained earlier from Red Hat Quay. For example, this creates a secret with your <access_token> called quay-integration with a key called token: 

    $ oc create secret generic quay-integration --from-literal=token=<access_token>

    The result places the newly created private key and certificate within a secret specified. The secret will be mounted into the appropriate located within the operator as declared in the Deployment of the Operator.

  4. Create a Service for the Operator’s webhook endpoint:

    quay-webhook.yaml

    apiVersion: v1
kind: Service
metadata:
  labels:
    name: quay-bridge-operator
  name: quay-bridge-operator
  namespace: openshift-operators
spec:
  ports:
    - name: https
      port: 443
      protocol: TCP
      targetPort: 8443
  selector:
    name: quay-bridge-operator
  sessionAffinity: None
  type: ClusterIP

  5. Create the webhook service as follows: 

    $ oc create -f quay-webhook.yaml
  6. Download the webhook-create-signed-cert.sh script, so you can use it to generate a certificate signed by a Kubernetes certificate authority.

  7. Execute the following command to request the certificate: 

    $ ./webhook-create-signed-cert.sh --namespace openshift-operators \
   --secret quay-bridge-operator-webhook-certs \
   --service quay-bridge-operator

  8. Execute the following command to retrieve the CA and format the result as a single line so that it can be entered into the MutatingWebhookConfiguration resource: 

    $ oc get configmap -n kube-system \
   extension-apiserver-authentication \
   -o=jsonpath='{.data.client-ca-file}' | base64 | tr -d '\n'

  9. Replace the ${CA_BUNDLE} variable in the following MutatingWebhookConfiguration YAML:

    quay-mutating-webhook.yaml

    apiVersion: admissionregistration.k8s.io/v1beta1
kind: MutatingWebhookConfiguration
metadata:
  name: quay-bridge-operator
webhooks:
  - name: quayintegration.redhatcop.redhat.io
    clientConfig:
      service:
        namespace: openshift-operators
        name: quay-bridge-operator
        path: "/admissionwebhook"
      caBundle: "${CA_BUNDLE}" 1
    rules:
    - operations:  [ "CREATE" ]
      apiGroups: [ "build.openshift.io" ]
      apiVersions: ["v1" ]
      resources: [ "builds" ]
    failurePolicy: Fail

    1
    Replace ${CA_BUNDLE} with the output of the previous step. It will appear as one long line that you copy and paste to replace ${CA_BUNDLE}.

  10. Create the MutatingWebhookConfiguration as follows: 

    $ oc create -f quay-mutating-webhook.yaml

    Until the operator is running, new requests for builds will fail since the webserver the MutatingWebhookConfiguration invokes is not available and a proper is response is required in order for the object to be persisted in etcd.

  11. Go to the OpenShift console and install the Quay Bridge Operator as follows:

    • Select OperatorHub and search for Quay Bridge Operator.
    • Select Install
    • Choose Installation Mode (all namespaces), Update Channel, and Approval Strategy (Automatic or Manual).
    • Select Subscribe

  12. Create the custom resource (CR) called QuayIntegration. For example:

    quay-integration.yaml

    apiVersion: redhatcop.redhat.io/v1alpha1
kind: QuayIntegration
metadata:
  name: example-quayintegration
spec:
  clusterID: openshift  1
  credentialsSecretName: openshift-operators/quay-integration 2
  quayHostname: https://<QUAY_URL>   3
  whitelistNamespaces: 4
    - default
  insecureRegistry: false 5

    1
    The clusterID value should be unique across the entire ecosystem. This value is optional and defaults to openshift.
    2
    For credentialsSecretName, replace openshift-operators/quay-integration with the name of the namespace and the secret containing the token you created earlier.
    3
    Replace QUAY_URL with the hostname of your Red Hat Quay instance.
    4
    The whitelistNamespaces is optional. If not used, the Bridge Operator will sync all namespaces to Red Hat Quay except the openshift prefixed project. In this example, the white listed namespace (default) will now have an associated Red Hat Quay organization. Use any namespace you like here.
    5
    If Quay is using self signed certificates, set the property insecureRegistry: true.

    The result is that organizations within Red Hat Quay should be created for the related namespaces in OpenShift.

  13. Create the QuayIntegration as follows: 

    $ oc create -f quay-integration.yaml

At this point a Quay integration resource is created, linking the OpenShift cluster to the Red Hat Quay instance.

The whitelisted namespace you created should now have a Red Hat Quay organization. If you were to use a command such as oc new-app to create a new application in that namespace, you would see a new Red Hat Quay repository created for it instead of using the internal registry.