Chapter 4. Creating an OpenShift Container Storage Cluster service for external mode

You need to create a new OpenShift Container Storage cluster service after you install OpenShift Container Storage operator on OpenShift Container Platform deployed on user provisioned infrastructures VMware vSphere or Bare metal platform.

Prerequisites

  • You must be logged into the working OpenShift Container Platform version 4.5.4 or above.
  • OpenShift Container Storage operator must be installed. For more information, see Installing OpenShift Container Storage Operator using the Operator Hub.
  • Red Hat Ceph Storage version 4.1.1 or later is required for the external cluster. For more information, see this knowledge base article on Red Hat Ceph Storage releases and corresponding Ceph package versions.

    If you have updated the Red Hat Ceph Storage cluster to version 4.1.1 or later from a previous release and is not a freshly deployed cluster, you must manually set the application type for CephFS pool on the Red Hat Ceph Storage cluster to enable CephFS PVC creation in external mode.

    For more details, see Troubleshooting CephFS PVC creation in external mode.

  • It is recommended that the external Red Hat Ceph Storage cluster has the PG Autoscaler enabled, with a target_size_ratio of 0.49. For more information, see The placement group autoscaler section in the Red Hat Ceph Storage documentation.
  • The external Ceph cluster should have an existing RBD pool pre-configured for use. If it does not exist, contact your Red Hat Ceph Storage administrator to create one before you move ahead with OpenShift Container Storage deployment.

Procedure

  1. Click Operators → Installed Operators from the OpenShift Web Console to view the installed operators. Ensure that the Project selected is openshift-storage.
  2. On the Installed Operators page, Click Openshift Container Storage.

    Figure 4.1. OpenShift Container Storage Operator page

    Screenshot of OpenShift Container Storage operator dashboard.
  3. On the Installed Operators → Operator Details page, perform either of the following to create a Storage Cluster Service.

    1. On the Details tab → Provided APIs → OCS Storage Cluster, click Create Instance.

      Figure 4.2. Operator Details Page

      Screenshot of Operator Details Page.
    2. Alternatively, select the Storage cluster tab and click Create OCS Cluster Service.

      Figure 4.3. Storage Cluster tab

      Screenshot of Storage Cluster tab on OpenShift Container Storage Operator dashboard.
  4. On the Create Storage Cluster page, ensure that the following options are selected:

    Figure 4.4. Connect to external cluster section on Create Storage Cluster form

    Screenshot shows connect to external cluster section after you select mode as external where you can download the python script and then upload the RHCS json file.
    1. Select Mode as External. By default, Internal is selected as deployment mode.
    2. In the Connect to external cluster section, click on the Download Script link to download the python script for extracting Ceph cluster details.
    3. For extracting the Red Hat Ceph Storage (RHCS) cluster details, contact the RHCS admin to run the downloaded python script on a Red Hat Ceph Storage client node.

      • Run the following command on the RHCS client node to view the list of available arguments.

        # python3 ceph-external-cluster-details-exporter.py --help
        Important

        Use python instead of python3 if the Red Hat Ceph Storage 4.x cluster is deployed on Red Hat Enterprise Linux 7.x (RHEL 7.x) cluster.

        Note

        If you do not have access to the RHCS client node, you can also run the script from inside a MON container (containerized deployment) or from a MON node (rpm deployment).

      • To retrieve the external cluster details from the RHCS cluster, run the following command

        # python3 ceph-external-cluster-details-exporter.py --rbd-data-pool-name <rbd block pool name> [optional arguments]

        For example

        # python3 ceph-external-cluster-details-exporter.py --rbd-data-pool-name ceph-rbd --rgw-endpoint xxx.xxx.xxx.xxx:xxxx --run-as-user client.ocs

        In the above example,

        • --rbd-data-pool-name is a mandatory parameter used for providing Block Storage in OpenShift Container Storage.
        • --rgw-endpoint is optional. Provide this parameter if object storage is to be provisioned through Ceph Rados Gateway for OpenShift Container Storage.
        • -- run-as-user is an optional parameter used for providing a name for the Ceph user which is created by the script. If this parameter is not specified, a default user name client.healthchecker is created. The permissions for the new user is set as:

          • caps: [mon] allow r, allow command quorum_status
          • caps: [osd] allow rwx pool=RGW_POOL_PREFIX.rgw.meta, allow r pool=.rgw.root, allow rw pool=RGW_POOL_PREFIX.rgw.control, allow x pool=RGW_POOL_PREFIX.rgw.buckets.index

            Example of JSON output generated using the python script:

            [{"name": "rook-ceph-mon-endpoints", "kind": "ConfigMap", "data": {"data": "ceph-mon-node=xxx.xxx.xxx.xxx:xxxx", "maxMonId": "0", "mapping": "{}"}}, {"name": "rook-ceph-mon", "kind": "Secret", "data": {"admin-secret": "<admin-secret>", "cluster-name": "openshift-storage", "fsid": "<fs-id>", "mon-secret": "<mon-secret>"}}, {"name": "rook-ceph-operator-creds", "kind": "Secret", "data": {"userID": "client.healthchecker", "userKey": "<user-key>"}}, {"name": "rook-csi-rbd-node", "kind": "Secret", "data": {"userID": "csi-rbd-node", "userKey": "<user-key>"}}, {"name": "rook-csi-rbd-provisioner", "kind": "Secret", "data": {"userID": "csi-rbd-provisioner", "userKey": "<user-key>"}}, {"name": "rook-csi-cephfs-node", "kind": "Secret", "data": {"adminID": "csi-cephfs-node", "adminKey": "<admin-key>"}}, {"name": "rook-csi-cephfs-provisioner", "kind": "Secret", "data": {"adminID": "csi-cephfs-provisioner", "adminKey": "<admin-key>"}}, {"name": "ceph-rbd", "kind": "StorageClass", "data": {"pool": "ceph-rbd"}}, {"name": "cephfs", "kind": "StorageClass", "data": {"fsName": "cephfs", "pool": "cephfs_data"}}, {"name": "ceph-rgw", "kind": "StorageClass", "data": {"endpoint": "xxx.xxx.xxx.xxx:xxxx"}}]

      • Save the JSON output to a file with .json extension

        Note

        For OpenShift Container Storage to work seamlessly, ensure that the parameters (RGW endpoint, CephFS details, RBD pool, etc.) to be uploaded using the JSON file remains unchanged on the RHCS external cluster, post storage cluster creation.

    4. Click External cluster metadata → Browse to select and upload the json file. The json file content will be populated and displayed in the text box.

      Figure 4.5. Json file content

      Screenshot shows the content of the credentials file after it is uploaded
  5. Click Create.

    Note

    The Create button is enabled only after you upload the .json file.

Verification steps