Preparing to deploy in a disconnected environment
Prerequisite configuration for disconnected deployments
storage-docs@redhat.com
Abstract
Preface
When your Red Hat OpenShift Container Storage environment is not directly connected to the internet, some additional configuration is required to provide the Operator Lifecycle Manager (OLM) with alternatives to the default Operator Hub and image registries.
See the OpenShift Container Platform documentation for more general information: Using Operator Lifecycle Manager on restricted networks.
To configure your cluster for disconnected operation:
When these steps are complete, continue with deployment as usual.
Chapter 1. Adding mirror registry authentication details
Prerequisites
- Verify that your existing disconnected cluster uses OpenShift Container Platform 4.3 or higher.
-
Verify that you have an
oc clientversion of 4.4 or higher. - Prepare a mirror host with a mirror registry. See Preparing your mirror host for details.
Procedure
-
Log in to the OpenShift Container Platform cluster using the
cluster-adminrole. Locate your
auth.jsonfile.This file is generated when you use podman or docker to log in to a registry. It is located in one of the following locations:
-
~/.docker/auth.json -
/run/user/<UID>/containers/auth.json -
/var/run/containers/<UID>/auth.json
-
Obtain your unique Red Hat registry pull secret and paste it into your
auth.jsonfile. It will look something like this.{ "auths": { "cloud.openshift.com": { "auth": "*****************", "email": "user@example.com" }, "quay.io": { "auth": "*****************", "email": "user@example.com" }, "registry.connect.redhat.com": { "auth": "*****************", "email": "user@example.com" }, "registry.redhat.io": { "auth": "*****************", "email": "user@example.com" } } }Export environment variables with the appropriate details for your setup.
$ export AUTH_FILE="<location_of_auth.json>" $ export MIRROR_REGISTRY_DNS="<your_registry_url>:<port>"
Use
podmanto log in to the mirror registry and store the credentials in the${AUTH_FILE}.$ podman login ${MIRROR_REGISTRY_DNS} --tls-verify=false --authfile ${AUTH_FILE}This adds the mirror registry to the
auth.jsonfile.{ "auths": { "cloud.openshift.com": { "auth": "*****************", "email": "user@example.com" }, "quay.io": { "auth": "*****************", "email": "user@example.com" }, "registry.connect.redhat.com": { "auth": "*****************", "email": "user@example.com" }, "registry.redhat.io": { "auth": "*****************", "email": "user@example.com" }, "<mirror_registry>": { "auth": "*****************", } } }
Chapter 2. Building and mirroring the Red Hat operator catalog
Follow this process on a host that has access to Red Hat registries to create a mirror of those registries.
Prerequisites
- Run these commands as a cluster administrator.
-
Be aware that mirroring the
redhat-operatorcatalog can take hours to complete, and requires substantial available disk space on the mirror host.
Procedure
Build the catalog for
redhat-operators.Match the tag of the
ose-operator-registryin the--fromflag to the major and minor versions of the OpenShift Container Platform cluster (for example, 4.5).$ oc adm catalog build --appregistry-org redhat-operators \ --from=registry.redhat.io/openshift4/ose-operator-registry:v4.5 \ --to=${MIRROR_REGISTRY_DNS}/olm/redhat-operators:v1 \ --registry-config=${AUTH_FILE} \ --filter-by-os="linux/amd64" --insecureMirror the catalog for
redhat-operators.This is a long operation and can take 1-5 hours. Make sure there is 100 GB available disk space on the mirror host.
$ oc adm catalog mirror ${MIRROR_REGISTRY_DNS}/olm/redhat-operators:v1 \ ${MIRROR_REGISTRY_DNS} --registry-config=${AUTH_FILE} --insecureDisable the default
OperatorSourcesby addingdisableAllDefaultSources: trueto thespecfile for the Operator Hub.$ oc patch OperatorHub cluster --type json -p '[{"op": "add", "path": "/spec/disableAllDefaultSources", "value": true}]'
Chapter 3. Creating Operator imageContentSourcePolicy
After the oc adm catalog mirror command is completed, the imageContentSourcePolicy.yaml file gets created. The output directory for this file is usually, ./[catalog image name]-manifests). Use this procedure to add any missing entries to the .yaml file and apply them to cluster.
Procedure
Check the content of this file for the mirrors mapping shown as follows:
spec: repositoryDigestMirrors: - mirrors: - <your_registry>/ocs4 source: registry.redhat.io/ocs4 - mirrors: - <your_registry>/rhceph source: registry.redhat.io/rhceph - mirrors: - <your_registry>/openshift4 source: registry.redhat.io/openshift4 - mirrors: - <your_registry>/rhscl source: registry.redhat.io/rhscl-
Add any missing entries to the end of the
imageContentSourcePolicy.yamlfile. Apply the imageContentSourcePolicy.yaml file to the cluster.
$ oc apply -f ./[output dir]/imageContentSourcePolicy.yaml
Once the Image Content Source Policy is updated, all the nodes (master, infra, and workers) in the cluster need to be updated and rebooted. This process is automatically handled through the Machine Config Pool operator and take up to 30 minutes although the exact elapsed time might vary based on the number of nodes in your OpenShift cluster. You can monitor the update process by using the
oc get mcpcommand or theoc get nodecommand.
Chapter 4. Creating a CatalogSource from a mirrored catalog
Procedure
Create a
CatalogSourceobject that references the catalog image forredhat-operators.Save the following in a
redhat-operator-catalogsource.yamlfile, remembering to replace<your_registry>with your mirror registry URL:apiVersion: operators.coreos.com/v1alpha1 kind: CatalogSource metadata: name: redhat-operators namespace: openshift-marketplace spec: sourceType: grpc icon: base64data: PHN2ZyBpZD0iTGF5ZXJfMSIgZGF0YS1uYW1lPSJMYXllciAxIiB4bWxucz0iaHR0cDovL3d3dy53My5vcmcvMjAwMC9zdmciIHZpZXdCb3g9IjAgMCAxOTIgMTQ1Ij48ZGVmcz48c3R5bGU+LmNscy0xe2ZpbGw6I2UwMDt9PC9zdHlsZT48L2RlZnM+PHRpdGxlPlJlZEhhdC1Mb2dvLUhhdC1Db2xvcjwvdGl0bGU+PHBhdGggZD0iTTE1Ny43Nyw2Mi42MWExNCwxNCwwLDAsMSwuMzEsMy40MmMwLDE0Ljg4LTE4LjEsMTcuNDYtMzAuNjEsMTcuNDZDNzguODMsODMuNDksNDIuNTMsNTMuMjYsNDIuNTMsNDRhNi40Myw2LjQzLDAsMCwxLC4yMi0xLjk0bC0zLjY2LDkuMDZhMTguNDUsMTguNDUsMCwwLDAtMS41MSw3LjMzYzAsMTguMTEsNDEsNDUuNDgsODcuNzQsNDUuNDgsMjAuNjksMCwzNi40My03Ljc2LDM2LjQzLTIxLjc3LDAtMS4wOCwwLTEuOTQtMS43My0xMC4xM1oiLz48cGF0aCBjbGFzcz0iY2xzLTEiIGQ9Ik0xMjcuNDcsODMuNDljMTIuNTEsMCwzMC42MS0yLjU4LDMwLjYxLTE3LjQ2YTE0LDE0LDAsMCwwLS4zMS0zLjQybC03LjQ1LTMyLjM2Yy0xLjcyLTcuMTItMy4yMy0xMC4zNS0xNS43My0xNi42QzEyNC44OSw4LjY5LDEwMy43Ni41LDk3LjUxLjUsOTEuNjkuNSw5MCw4LDgzLjA2LDhjLTYuNjgsMC0xMS42NC01LjYtMTcuODktNS42LTYsMC05LjkxLDQuMDktMTIuOTMsMTIuNSwwLDAtOC40MSwyMy43Mi05LjQ5LDI3LjE2QTYuNDMsNi40MywwLDAsMCw0Mi41Myw0NGMwLDkuMjIsMzYuMywzOS40NSw4NC45NCwzOS40NU0xNjAsNzIuMDdjMS43Myw4LjE5LDEuNzMsOS4wNSwxLjczLDEwLjEzLDAsMTQtMTUuNzQsMjEuNzctMzYuNDMsMjEuNzdDNzguNTQsMTA0LDM3LjU4LDc2LjYsMzcuNTgsNTguNDlhMTguNDUsMTguNDUsMCwwLDEsMS41MS03LjMzQzIyLjI3LDUyLC41LDU1LC41LDc0LjIyYzAsMzEuNDgsNzQuNTksNzAuMjgsMTMzLjY1LDcwLjI4LDQ1LjI4LDAsNTYuNy0yMC40OCw1Ni43LTM2LjY1LDAtMTIuNzItMTEtMjcuMTYtMzAuODMtMzUuNzgiLz48L3N2Zz4= mediatype: image/svg+xml image: <your_registry>/olm/redhat-operators:v1 displayName: Redhat Operators Catalog publisher: Red HatCreate a
catalogsourceusing theredhat-operator-catalogsource.yamlfile:$ oc apply -f redhat-operator-catalogsource.yaml
Verification
Run the following command to verify that
catalogsourceand pod were created correctly.$ oc get catalogsource,pod -n openshift-marketplace | grep redhat-operators
Chapter 5. Continue to deployment
After your alternative catalog source is configured, you can continue to the appropriate deployment process: