Chapter 7. Configuring managed and hub clusters

7.1. Configuring SSL access between S3 endpoints

Configure network (SSL) access between the s3 endpoints so that metadata can be stored on the alternate cluster in a MCG object bucket using a secure transport protocol and in addition, the Hub cluster needs to verify access to the object buckets.

Note

If all of your OpenShift clusters are deployed using a signed and valid set of certificates for your environment then this section can be skipped.

Procedure

  1. Extract the ingress certificate for the Primary managed cluster and save the output to primary.crt.

    $ oc get cm default-ingress-cert -n openshift-config-managed -o jsonpath="{['data']['ca-bundle\.crt']}" > primary.crt
  2. Extract the ingress certificate for the Secondary managed cluster and save the output to secondary.crt.

    $ oc get cm default-ingress-cert -n openshift-config-managed -o jsonpath="{['data']['ca-bundle\.crt']}" > secondary.crt
  3. Create a new ConfigMap to hold the remote cluster’s certificate bundle with filename cm-clusters-crt.yaml on the Primary managed cluster, Secondary managed cluster, and the Hub cluster.

    Note

    There could be more or less than three certificates for each cluster as shown in this example file. Also, ensure that the certificate contents are correctly indented after you copy and paste from the primary.crt and secondary.crt files that were created before.

    apiVersion: v1
    data:
      ca-bundle.crt: |
        -----BEGIN CERTIFICATE-----
        <copy contents of cert1 from primary.crt here>
        -----END CERTIFICATE-----
    
        -----BEGIN CERTIFICATE-----
        <copy contents of cert2 from primary.crt here>
        -----END CERTIFICATE-----
    
        -----BEGIN CERTIFICATE-----
        <copy contents of cert3 primary.crt here>
        -----END CERTIFICATE----
    
        -----BEGIN CERTIFICATE-----
        <copy contents of cert1 from secondary.crt here>
        -----END CERTIFICATE-----
    
        -----BEGIN CERTIFICATE-----
        <copy contents of cert2 from secondary.crt here>
        -----END CERTIFICATE-----
    
        -----BEGIN CERTIFICATE-----
        <copy contents of cert3 from secondary.crt here>
        -----END CERTIFICATE-----
    kind: ConfigMap
    metadata:
      name: user-ca-bundle
      namespace: openshift-config
  4. Create the ConfigMap file on the Primary managed cluster, Secondary managed cluster, and the Hub cluster.

    $ oc create -f cm-clusters-crt.yaml

    Example output:

    configmap/user-ca-bundle created
    Important

    For the Hub cluster to verify access to the object buckets using the DRPolicy resource, the same ConfigMap cm-clusters-crt.yaml must also be created on the Hub cluster.

  5. Patch the default proxy resource on the Primary managed cluster, Secondary managed cluster, and the Hub cluster.

    $ oc patch proxy cluster --type=merge  --patch='{"spec":{"trustedCA":{"name":"user-ca-bundle"}}}'

    Example output:

    proxy.config.openshift.io/cluster patched

7.2. Creating object buckets and S3StoreProfiles

OpenShift DR requires S3 stores to store relevant cluster data of a workload from the managed clusters and to orchestrate a recovery of the workload during failover or relocate actions. These instructions are applicable for creating the necessary object bucket(s) using Multicloud Object Gateway (MCG). MCG should already be installed as a result of installing OpenShift Data Foundation.

Procedure

  1. Create MCG object bucket or OBC to be used for storing persistent volume metadata on both the Primary and Secondary managed clusters.

    1. Copy the following YAML file to filename odrbucket.yaml.

      apiVersion: objectbucket.io/v1alpha1
      kind: ObjectBucketClaim
      metadata:
        name: odrbucket
        namespace: openshift-storage
      spec:
        generateBucketName: "odrbucket"
        storageClassName: openshift-storage.noobaa.io
    2. Create a MCG bucket odrbucket on both the Primary managed cluster and the Secondary managed cluster.

      $ oc create -f odrbucket.yaml

      Example output:

      objectbucketclaim.objectbucket.io/odrbucket created
  2. Extract the odrbucket OBC access key for each managed cluster as their base-64 encoded values by using the following command.

    $ oc get secret odrbucket -n openshift-storage -o jsonpath='{.data.AWS_ACCESS_KEY_ID}{"\n"}'

    Example output:

    cFpIYTZWN1NhemJjbEUyWlpwN1E=
  3. Extract the odrbucket OBC secret key for each managed cluster as their base-64 encoded values by using the following command.

    $ oc get secret odrbucket -n openshift-storage -o jsonpath='{.data.AWS_SECRET_ACCESS_KEY}{"\n"}'

    Example output:

    V1hUSnMzZUoxMHRRTXdGMU9jQXRmUlAyMmd5bGwwYjNvMHprZVhtNw==
Important

The access key and secret key must be retrieved for the odrbucket OBC on both the Primary managed cluster and Secondary managed cluster.

7.3. Creating S3 secrets for Multicloud Object Gateway object buckets

Now that the necessary information has been extracted for the object buckets in the previous section, there must be new Secrets created on the Hub cluster. These new Secrets will store the MCG object bucket access key and secret key for both managed clusters on the Hub cluster.

Procedure

  1. Copy the following S3 secret YAML format for the Primary managed cluster to filename odr-s3secret-primary.yaml.

    apiVersion: v1
    data:
      AWS_ACCESS_KEY_ID: <primary cluster base-64 encoded access key>
      AWS_SECRET_ACCESS_KEY: <primary cluster base-64 encoded secret access key>
    kind: Secret
    metadata:
      name: odr-s3secret-primary
      namespace: openshift-dr-system
  2. Create this secret on the Hub cluster.

    $ oc create -f odr-s3secret-primary.yaml

    Example output:

    secret/odr-s3secret-primary created
  3. Copy the following S3 secret YAML format for the Secondary managed cluster to filename odr-s3secret-secondary.yaml.

    apiVersion: v1
    data:
      AWS_ACCESS_KEY_ID: <secondary cluster base-64 encoded access key>
      AWS_SECRET_ACCESS_KEY: <secondary cluster base-64 encoded secret access key>
    kind: Secret
    metadata:
      name: odr-s3secret-secondary
      namespace: openshift-dr-system
  4. Create this secret on the Hub cluster.

    $ oc create -f odr-s3secret-secondary.yaml

    Example output:

    secret/odr-s3secret-secondary created
Important

The values for the access key and secret key must be base-64 encoded. The encoded values for the keys were retrieved in the prior section.

7.4. Configure OpenShift DR Hub operator s3StoreProfiles

To find the s3CompatibleEndpoint or route for MCG, execute the following command on the Primary managed cluster and the Secondary managed cluster:

Procedure

  1. Search for the external S3 endpoint s3CompatibleEndpoint or route for MCG on each managed cluster by using the following command.

    $ oc get route s3 -n openshift-storage -o jsonpath --template="https://{.spec.host}{'\n'}"

    Example output:

    https://s3-openshift-storage.apps.perf1.example.com
    Important

    The unique s3CompatibleEndpoint route or s3-openshift-storage.apps.<primary clusterID>.<baseDomain> and s3-openshift-storage.apps.<secondary clusterID>.<baseDomain> must be retrieved for both the Primary managed cluster and Secondary managed cluster respectively.

  2. Search for the odrbucket OBC exact bucket name.

    $ oc get configmap odrbucket -n openshift-storage -o jsonpath='{.data.BUCKET_NAME}{"\n"}'

    Example output:

    odrbucket-2f2d44e4-59cb-4577-b303-7219be809dcd
    Important

    The unique s3Bucket name odrbucket-<your value1> and odrbucket-<your value2> must be retrieved on both the Primary managed cluster and Secondary managed cluster respectively.

  3. Modify the ConfigMap ramen-hub-operator-config on the Hub cluster to add the new content.

    $ oc edit configmap ramen-hub-operator-config -n openshift-dr-system
  4. Add the following new content starting at s3StoreProfiles to the ConfigMap on the Hub cluster only.

    [...]
    data:
      ramen_manager_config.yaml: |
        apiVersion: ramendr.openshift.io/v1alpha1
        kind: RamenConfig
    [...]
        ramenControllerType: "dr-hub"
        ### Start of new content to be added
        s3StoreProfiles:
        - s3ProfileName: s3-primary
          s3CompatibleEndpoint: https://s3-openshift-storage.apps.<primary clusterID>.<baseDomain>
          s3Region: primary
          s3Bucket: odrbucket-<your value1>
          s3SecretRef:
            name: odr-s3secret-primary
            namespace: openshift-dr-system
        - s3ProfileName: s3-secondary
          s3CompatibleEndpoint: https://s3-openshift-storage.apps.<secondary clusterID>.<baseDomain>
          s3Region: secondary
          s3Bucket: odrbucket-<your value2>
          s3SecretRef:
            name: odr-s3secret-secondary
            namespace: openshift-dr-system
    [...]