Chapter 6. Updating the OpenShift Data Foundation external secret

Procedure

1. Download the OpenShift Data Foundation version of the ceph-external-cluster-details-exporter.py python script.

   # oc get csv $(oc get csv -n openshift-storage | grep ocs-operator | awk '{print $1}') -n openshift-storage -o jsonpath='{.metadata.annotations.external\.features\.ocs\.openshift\.io/export-script}' | base64 --decode > ceph-external-cluster-details-exporter.py

2. Update permission caps on the external Red Hat Ceph Storage cluster by running ceph-external-cluster-details-exporter.py on any client node in the external Red Hat Ceph Storage cluster. You may need to ask your Red Hat Ceph Storage administrator to do this.

   # python3 ceph-external-cluster-details-exporter.py --upgrade

   The updated permissions for the user are set as:

   client.csi-cephfs-node
   key: AQCYz0piYgu/IRAAipji4C8+Lfymu9vOrox3zQ==
   caps: [mds] allow rw
   caps: [mgr] allow rw
   caps: [mon] allow r, allow command 'osd blocklist'
   caps: [osd] allow rw tag cephfs =
   client.csi-cephfs-provisioner
   key: AQCYz0piDUMSIxAARuGUyhLXFO9u4zQeRG65pQ==
   caps: [mgr] allow rw
   caps: [mon] allow r, allow command 'osd blocklist'
   caps: [osd] allow rw tag cephfs metadata=*
   client.csi-rbd-node
   key: AQCYz0pi88IKHhAAvzRN4fD90nkb082ldrTaHA==
   caps: [mon] profile rbd, allow command 'osd blocklist'
   caps: [osd] profile rbd
   client.csi-rbd-provisioner
   key: AQCYz0pi6W8IIBAAgRJfrAW7kZfucNdqJqS9dQ==
   caps: [mgr] allow rw
   caps: [mon] profile rbd, allow command 'osd blocklist'
   caps: [osd] profile rbd

3. Run the previously downloaded python script and save the JSON output that gets generated, from the external Red Hat Ceph Storage cluster.

   1. Run the previously downloaded python script:

      # python3 ceph-external-cluster-details-exporter.py --rbd-data-pool-name <rbd block pool name> --monitoring-endpoint <ceph mgr prometheus exporter endpoint> --monitoring-endpoint-port <ceph mgr prometheus exporter port> --rgw-endpoint <rgw endpoint> --run-as-user <ocs_client_name>  [optional arguments]

      --rbd-data-pool-name

      Is a mandatory parameter used for providing block storage in OpenShift Data Foundation.

      --rgw-endpoint

      Is optional. Provide this parameter if object storage is to be provisioned through Ceph Rados Gateway for OpenShift Data Foundation. Provide the endpoint in the following format: <ip_address>:<port>.

      --monitoring-endpoint

      Is optional. It accepts comma separated list of IP addresses of active and standby mgrs reachable from the OpenShift Container Platform cluster. If not provided, the value is automatically populated.

      --monitoring-endpoint-port

      Is optional. It is the port associated with the ceph-mgr Prometheus exporter specified by --monitoring-endpoint. If not provided, the value is automatically populated.

      --run-as-user

      The client name used during OpenShift Data Foundation cluster deployment. Use the default client name client.healthchecker if a different client name was not set.

      Note

      Ensure that all the parameters, including the optional arguments, except for monitoring-endpoint and monitoring-endpoint-port, are the same as what was used during the deployment of OpenShift Data Foundation in external mode.

   2. Save the JSON output generated after running the script in the previous step.

      Example output:

      [{"name": "rook-ceph-mon-endpoints", "kind": "ConfigMap", "data": {"data": "xxx.xxx.xxx.xxx:xxxx", "maxMonId": "0", "mapping": "{}"}}, {"name": "rook-ceph-mon", "kind": "Secret", "data": {"admin-secret": "admin-secret", "fsid": "<fs-id>", "mon-secret": "mon-secret"}}, {"name": "rook-ceph-operator-creds", "kind": "Secret", "data": {"userID": "<user-id>", "userKey": "<user-key>"}}, {"name": "rook-csi-rbd-node", "kind": "Secret", "data": {"userID": "csi-rbd-node", "userKey": "<user-key>"}}, {"name": "ceph-rbd", "kind": "StorageClass", "data": {"pool": "<pool>"}}, {"name": "monitoring-endpoint", "kind": "CephCluster", "data": {"MonitoringEndpoint": "xxx.xxx.xxx.xxxx", "MonitoringPort": "xxxx"}}, {"name": "rook-ceph-dashboard-link", "kind": "Secret", "data": {"userID": "ceph-dashboard-link", "userKey": "<user-key>"}}, {"name": "rook-csi-rbd-provisioner", "kind": "Secret", "data": {"userID": "csi-rbd-provisioner", "userKey": "<user-key>"}}, {"name": "rook-csi-cephfs-provisioner", "kind": "Secret", "data": {"adminID": "csi-cephfs-provisioner", "adminKey": "<admin-key>"}}, {"name": "rook-csi-cephfs-node", "kind": "Secret", "data": {"adminID": "csi-cephfs-node", "adminKey": "<admin-key>"}}, {"name": "cephfs", "kind": "StorageClass", "data": {"fsName": "cephfs", "pool": "cephfs_data"}}, {"name": "ceph-rgw", "kind": "StorageClass", "data": {"endpoint": "xxx.xxx.xxx.xxxx", "poolPrefix": "default"}}, {"name": "rgw-admin-ops-user", "kind": "Secret", "data": {"accessKey": "<access-key>", "secretKey": "<secret-key>"}}]

4. Upload the generated JSON file.

   1. Log in to the OpenShift Web Console.
   2. Click Workloads → Secrets.
   3. Set project to openshift-storage.
   4. Click rook-ceph-external-cluster-details.
   5. Click Actions (⋮) → Edit Secret.
   6. Click Browse and upload the JSON file.
   7. Click Save.