Chapter 13. Red Hat Quay build enhancements

Procedure

1. Log in to your Red Hat Quay cluster using a cluster administrator account.

2. Create a new project where your virtual builders will be run, for example, virtual-builders, by running the following command:

   $ oc new-project virtual-builders

3. Create a ServiceAccount in the project that will be used to run builds by entering the following command:

   $ oc create sa -n virtual-builders quay-builder

4. Provide the created service account with editing permissions so that it can run the build:

   $ oc adm policy -n virtual-builders add-role-to-user edit system:serviceaccount:virtual-builders:quay-builder

5. Grant the Quay builder anyuid scc permissions by entering the following command:

   $ oc adm policy -n virtual-builders add-scc-to-user anyuid -z quay-builder

   Note

   This action requires cluster admin privileges. This is required because builders must run as the Podman user for unprivileged or rootless builds to work.

6. Obtain the token for the Quay builder service account.

   1. If using OpenShift Container Platform 4.10 or an earlier version, enter the following command:

      oc sa get-token -n virtual-builders quay-builder

   2. If using OpenShift Container Platform 4.11 or later, enter the following command:

      $ oc create token quay-builder -n virtual-builders

      Note

      When the token expires you will need to request a new token. Optionally, you can also add a custom expiration. For example, specify --duration 20160m to retain the token for two weeks.

      Example output

      eyJhbGciOiJSUzI1NiIsImtpZCI6IldfQUJkaDVmb3ltTHZ0dGZMYjhIWnYxZTQzN2dJVEJxcDJscldSdEUtYWsifQ...

7. Determine the builder route by entering the following command:

   $ oc get route -n quay-enterprise

   Example output

   NAME                                  HOST/PORT                                                                    PATH   SERVICES                              PORT   TERMINATION     WILDCARD
   ...
   example-registry-quay-builder         example-registry-quay-builder-quay-enterprise.apps.docs.quayteam.org                example-registry-quay-app             grpc   edge/Redirect   None
   ...

8. Generate a self-signed SSL/TlS certificate with the .crt extension by entering the following command:

   $ oc extract cm/kube-root-ca.crt -n openshift-apiserver

   Example output

   ca.crt

9. Rename the ca.crt file to extra_ca_cert_build_cluster.crt by entering the following command:

   $ mv ca.crt extra_ca_cert_build_cluster.crt

10. Locate the secret for you configuration bundle in the Console, and select Actions → Edit Secret and add the appropriate builder configuration:

    FEATURE_USER_INITIALIZE: true
    BROWSER_API_CALLS_XHR_ONLY: false
    SUPER_USERS:
    - <superusername>
    FEATURE_USER_CREATION: false
    FEATURE_QUOTA_MANAGEMENT: true
    FEATURE_BUILD_SUPPORT: True
    BUILDMAN_HOSTNAME: <sample_build_route> 1
    BUILD_MANAGER:
      - ephemeral
      - ALLOWED_WORKER_COUNT: 1
        ORCHESTRATOR_PREFIX: buildman/production/
        JOB_REGISTRATION_TIMEOUT: 3600 2
        ORCHESTRATOR:
          REDIS_HOST: <sample_redis_hostname> 3
          REDIS_PASSWORD: ""
          REDIS_SSL: false
          REDIS_SKIP_KEYSPACE_EVENT_SETUP: false
        EXECUTORS:
          - EXECUTOR: kubernetesPodman
            NAME: openshift
            BUILDER_NAMESPACE: <sample_builder_namespace> 4
            SETUP_TIME: 180
            MINIMUM_RETRY_THRESHOLD: 0
            BUILDER_CONTAINER_IMAGE: <sample_builder_container_image> 5
            # Kubernetes resource options
            K8S_API_SERVER: <sample_k8s_api_server> 6
            K8S_API_TLS_CA: <sample_crt_file> 7
            VOLUME_SIZE: 8G
            KUBERNETES_DISTRIBUTION: openshift
            CONTAINER_MEMORY_LIMITS: 300m 8
            CONTAINER_CPU_LIMITS: 1G 9
            CONTAINER_MEMORY_REQUEST: 300m 10
            CONTAINER_CPU_REQUEST: 1G 11
            NODE_SELECTOR_LABEL_KEY: ""
            NODE_SELECTOR_LABEL_VALUE: ""
            SERVICE_ACCOUNT_NAME: <sample_service_account_name>
            SERVICE_ACCOUNT_TOKEN: <sample_account_token> 12

    1

    The build route is obtained by running oc get route -n with the name of your OpenShift Operator’s namespace. A port must be provided at the end of the route, and it should use the following format: [quayregistry-cr-name]-quay-builder-[ocp-namespace].[ocp-domain-name]:443.

    2

    If the JOB_REGISTRATION_TIMEOUT parameter is set too low, you might receive the following error: failed to register job to build manager: rpc error: code = Unauthenticated desc = Invalid build token: Signature has expired. It is suggested that this parameter be set to at least 240.

    3

    If your Redis host has a password or SSL/TLS certificates, you must update accordingly.

    4

    Set to match the name of your virtual builders namespace, for example, virtual-builders.

    5

    For early access, the BUILDER_CONTAINER_IMAGE is currently quay.io/projectquay/quay-builder:3.7.0-rc.2. Note that this might change during the early access window. If this happens, customers are alerted.

    6

    The K8S_API_SERVER is obtained by running oc cluster-info.

    7

    You must manually create and add your custom CA cert, for example, K8S_API_TLS_CA: /conf/stack/extra_ca_certs/build_cluster.crt.

    8

    Defaults to 5120Mi if left unspecified.

    9

    For virtual builds, you must ensure that there are enough resources in your cluster. Defaults to 1000m if left unspecified.

    10

    Defaults to 3968Mi if left unspecified.

    11

    Defaults to 500m if left unspecified.

    12

    Obtained when running oc create sa.

    Sample configuration

    FEATURE_USER_INITIALIZE: true
    BROWSER_API_CALLS_XHR_ONLY: false
    SUPER_USERS:
    - quayadmin
    FEATURE_USER_CREATION: false
    FEATURE_QUOTA_MANAGEMENT: true
    FEATURE_BUILD_SUPPORT: True
    BUILDMAN_HOSTNAME: example-registry-quay-builder-quay-enterprise.apps.docs.quayteam.org:443
    BUILD_MANAGER:
      - ephemeral
      - ALLOWED_WORKER_COUNT: 1
        ORCHESTRATOR_PREFIX: buildman/production/
        JOB_REGISTRATION_TIMEOUT: 3600
        ORCHESTRATOR:
          REDIS_HOST: example-registry-quay-redis
          REDIS_PASSWORD: ""
          REDIS_SSL: false
          REDIS_SKIP_KEYSPACE_EVENT_SETUP: false
        EXECUTORS:
          - EXECUTOR: kubernetesPodman
            NAME: openshift
            BUILDER_NAMESPACE: virtual-builders
            SETUP_TIME: 180
            MINIMUM_RETRY_THRESHOLD: 0
            BUILDER_CONTAINER_IMAGE: quay.io/projectquay/quay-builder:3.7.0-rc.2
            # Kubernetes resource options
            K8S_API_SERVER: api.docs.quayteam.org:6443
            K8S_API_TLS_CA: /conf/stack/extra_ca_certs/build_cluster.crt
            VOLUME_SIZE: 8G
            KUBERNETES_DISTRIBUTION: openshift
            CONTAINER_MEMORY_LIMITS: 1G
            CONTAINER_CPU_LIMITS: 1080m
            CONTAINER_MEMORY_REQUEST: 1G
            CONTAINER_CPU_REQUEST: 580m
            NODE_SELECTOR_LABEL_KEY: ""
            NODE_SELECTOR_LABEL_VALUE: ""
            SERVICE_ACCOUNT_NAME: quay-builder
            SERVICE_ACCOUNT_TOKEN: "eyJhbGciOiJSUzI1NiIsImtpZCI6IldfQUJkaDVmb3ltTHZ0dGZMYjhIWnYxZTQzN2dJVEJxcDJscldSdEUtYWsifQ"

Procedure

1. Log in to your Red Hat Quay repository.
2. Click Create New Repository and create a new registry, for example, testrepo.

3. On the Repositories page, click the Builds tab on the navigation pane. Alternatively, use the corresponding URL directly:

   https://example-registry-quay-quay-enterprise.apps.docs.quayteam.org/repository/quayadmin/testrepo?tab=builds

   Important

   In some cases, the builder might have issues resolving hostnames. This issue might be related to the dnsPolicy being set to default on the job object. Currently, there is no workaround for this issue. It will be resolved in a future version of Red Hat Quay.

4. Click Create Build Trigger → Custom Git Repository Push.

5. Enter the HTTPS or SSH style URL used to clone your Git repository, then click Continue. For example:

   https://github.com/gabriel-rh/actions_test.git

6. Check Tag manifest with the branch or tag name and then click Continue.
7. Enter the location of the Dockerfile to build when the trigger is invoked, for example, /Dockerfile and click Continue.
8. Enter the location of the context for the Docker build, for example, /, and click Continue.
9. If warranted, create a Robot Account. Otherwise, click Continue.
10. Click Continue to verify the parameters.
11. On the Builds page, click Options icon of your Trigger Name, and then click Run Trigger Now.
12. Enter a commit SHA from the Git repository and click Start Build.

13. You can check the status of your build by clicking the commit in the Build History page, or by running oc get pods -n virtual-builders. For example:

    $ oc get pods -n virtual-builders

    Example output

    NAME                                               READY   STATUS    RESTARTS   AGE
    f192fe4a-c802-4275-bcce-d2031e635126-9l2b5-25lg2   1/1     Running   0          7s

    $ oc get pods -n virtual-builders

    Example output

    NAME                                               READY   STATUS        RESTARTS   AGE
    f192fe4a-c802-4275-bcce-d2031e635126-9l2b5-25lg2   1/1     Terminating   0          9s

    $ oc get pods -n virtual-builders

    Example output

    No resources found in virtual-builders namespace.

14. When the build is finished, you can check the status of the tag under Tags on the navigation pane.

    Note

    With early access, full build logs and timestamps of builds are currently unavailable.

Procedure

1. Log in to your AWS console at s3.console.aws.com.
2. In the search bar, search for S3 and then click S3.
3. Click the name of your bucket, for example, myawsbucket.
4. Click the Permissions tab.

5. Under Cross-origin resource sharing (CORS), include the following parameters:

     [
         {
             "AllowedHeaders": [
                 "Authorization"
             ],
             "AllowedMethods": [
                 "GET"
             ],
             "AllowedOrigins": [
                 "*"
             ],
             "ExposeHeaders": [],
             "MaxAgeSeconds": 3000
         },
         {
             "AllowedHeaders": [
                 "Content-Type",
                 "x-amz-acl",
                 "origin"
             ],
             "AllowedMethods": [
                 "PUT"
             ],
             "AllowedOrigins": [
                 "*"
             ],
             "ExposeHeaders": [],
             "MaxAgeSeconds": 3000
         }
     ]

Procedure

1. Use the following reference to create a JSON file for your specific CORS needs. For example:

   $ cat gcp_cors.json

   Example output

   [
       {
         "origin": ["*"],
         "method": ["GET"],
         "responseHeader": ["Authorization"],
         "maxAgeSeconds": 3600
       },
       {
         "origin": ["*"],
         "method": ["PUT"],
         "responseHeader": [
                 "Content-Type",
                 "x-goog-acl",
                 "origin"],
         "maxAgeSeconds": 3600
       }
   ]

2. Enter the following command to update your GCP storage bucket:

   $ gcloud storage buckets update gs://<bucket_name> --cors-file=./gcp_cors.json

   Example output

   Updating
     Completed 1

3. You can display the updated CORS configuration of your GCP bucket by running the following command:

   $ gcloud storage buckets describe gs://<bucket_name>  --format="default(cors)"

   Example output

   cors:
   - maxAgeSeconds: 3600
     method:
     - GET
     origin:
     - '*'
     responseHeader:
     - Authorization
   - maxAgeSeconds: 3600
     method:
     - PUT
     origin:
     - '*'
     responseHeader:
     - Content-Type
     - x-goog-acl
     - origin