Chapter 1. Deploying using local storage devices

Procedure

1. Click Operators → OperatorHub.
2. Use Filter by keyword text box or the filter list to search for OpenShift Container Storage from the list of operators.
3. Click OpenShift Container Storage.
4. Click Install on the OpenShift Container Storage operator page.

5. On the Install Operator page, the following required options are selected by default:

   1. Update Channel as stable-4.6
   2. Installation Mode as A specific namespace on the cluster
   3. Installed Namespace as Operator recommended namespace openshift-storage. If Namespace openshift-storage does not exist, it will be created during the operator installation.
6. Select Enable operator recommended cluster monitoring on this namespace checkbox as this is required for cluster monitoring.

7. Approval Strategy is set to Automatic by default.

   Figure 1.1. Install Operator page

   
8. Click Install.

Procedure

1. Log in to the OpenShift Web Console.
2. Click Operators → OperatorHub.
3. Search for Local Storage Operator from the list of operators and click on it.

4. Click Install.

   Figure 1.2. Install Operator page

   

5. Set the following options on the Install Operator page:

   1. Update Channel as stable-4.6
   2. Installation Mode as A specific namespace on the cluster
   3. Installed Namespace as Operator recommended namespace openshift-local-storage.
   4. Approval Strategy as Automatic
6. Click Install.
7. Verify that the Local Storage Operator shows the Status as Succeeded.

Procedure

1. List and verify the name of the worker nodes with the OpenShift Container Storage label.

   $ oc get nodes -n openshift-storage

   Example output:

   NAME          STATUS   ROLES    AGE     VERSION
   bmworker01    Ready    worker   6h45m   v1.16.2
   bmworker02    Ready    worker   6h45m   v1.16.2
   bmworker03    Ready    worker   6h45m   v1.16.2

2. Log in to each worker node that is used for OpenShift Container Storage resources and find the unique by-id device name for each available raw block device.

   $ oc debug node/<Nodename>

   Example output:

   $ oc debug node/bmworker01
   Starting pod/bmworker01-debug ...
   To use host binaries, run `chroot /host`
   Pod IP: 10.0.135.71
   If you don't see a command prompt, try pressing enter.
   sh-4.2# chroot /host
   sh-4.4# lsblk
   NAME                         MAJ:MIN RM   SIZE RO TYPE MOUNTPOINT
   xvda                         202:0    0   120G  0 disk
   |-xvda1                      202:1    0   384M  0 part /boot
   |-xvda2                      202:2    0   127M  0 part /boot/efi
   |-xvda3                      202:3    0     1M  0 part
   `-xvda4                      202:4    0 119.5G  0 part
     `-coreos-luks-root-nocrypt 253:0    0 119.5G  0 dm   /sysroot
   nvme0n1                      259:0    0   931G  0 disk

   In this example, for bmworker01, the available local device is nvme0n1.

3. Identify the unique ID for each of the devices selected in Step 2.

   sh-4.4#  ls -l /dev/disk/by-id/ | grep nvme0n1
   lrwxrwxrwx. 1 root root 13 Mar 17 16:24 nvme-INTEL_SSDPE2KX010T7_PHLF733402LM1P0GGN -> ../../nvme0n1

   In the above example, the ID for the local device nvme0n1

   nvme-INTEL_SSDPE2KX010T7_PHLF733402LM1P0GGN

4. Repeat the above step to identify the device ID for all the other nodes that have the storage devices to be used by OpenShift Container Storage. See this Knowledge Base article for more details.

Procedure

1. Create local persistent volumes (PVs) on the storage nodes using LocalVolume custom resource (CR).

   Example of LocalVolume CR local-storage-block.yaml using OpenShift Container Storage label as node selector.

   apiVersion: local.storage.openshift.io/v1
   kind: LocalVolume
   metadata:
     name: local-block
     namespace: local-storage
     labels:
       app: ocs-storagecluster
   spec:
     nodeSelector:
       nodeSelectorTerms:
       - matchExpressions:
           - key: cluster.ocs.openshift.io/openshift-storage
             operator: In
             values:
             - ""
     storageClassDevices:
       - storageClassName: localblock
         volumeMode: Block
         devicePaths:
           - /dev/disk/by-id/nvme-INTEL_SSDPEKKA128G7_BTPY81260978128A   # <-- modify this line
           - /dev/disk/by-id/nvme-INTEL_SSDPEKKA128G7_BTPY80440W5U128A   # <-- modify this line
           - /dev/disk/by-id/nvme-INTEL_SSDPEKKA128G7_BTPYB85AABDE128A   # <-- modify this line

2. Create the LocalVolume CR for block PVs.

   $ oc create -f local-storage-block.yaml

3. Check if the pods are created.

   Example output:

   NAME                                    READY   STATUS  RESTARTS AGE
   local-block-local-diskmaker-cmfql       1/1     Running 0        31s
   local-block-local-diskmaker-g6fzr       1/1     Running 0        31s
   local-block-local-diskmaker-jkqxt       1/1     Running 0        31s
   local-block-local-provisioner-jgqcc     1/1     Running 0        31s
   local-block-local-provisioner-mx49d     1/1     Running 0        31s
   local-block-local-provisioner-qbcvp     1/1     Running 0        31s
   local-storage-operator-54bc7566c6-ddbrt 1/1     Running 0        12m

4. Check if the PVs are created.

   $ oc get pv

   Example output:

   NAME                CAPACITY   ACCESS MODES   RECLAIM POLICY   STATUS      CLAIM   STORAGECLASS   REASON   AGE
   local-pv-150fdc87   931Gi      RWO            Delete           Available           localblock              2m11s
   local-pv-183bfc0a   931Gi      RWO            Delete           Available           localblock              2m15s
   local-pv-b2f5cb25   931Gi      RWO            Delete           Available           localblock              2m21s

5. Check for the new localblock StorageClass.

   $ oc get sc | grep localblock

   Example output:

   NAME            PROVISIONER                    RECLAIMPOLICY
   VOLUMEBINDINGMODE ALLOWVOLUMEEXPANSION     AGE
   localblock      kubernetes.io/no-provisioner   Delete
   WaitForFirstConsumer  false                2m10s

6. Create the OpenShift Container Storage Cluster Service that uses the localblock Storage Class.

   1. Log into the OpenShift Web Console.

   2. Click Installed Operators from the left pane of the OpenShift Web Console to view the installed operators.

      Figure 1.3. OpenShift Container Storage Operator page

      
   3. Click the OpenShift Container Storage installed operator.

   4. On the Operator Details page, click the Storage Cluster link.

      Figure 1.4. Storage Cluster tab

      

   5. Click Create Storage Cluster.

      
7. Select Internal-Attached devices for the Select Mode.

8. Perform the steps to create the storage cluster based on any of the following possible scenarios that you are in.

   Local Storage Operator is not installed and storage class not created

   1. You are prompted to install the Local Storage Operator. Click Install and install the operator as described in Installing Local Storage Operator.
   2. Proceed to the storage class creation from the wizard as described in the next steps.

   Local Storage Operator is installed but the storage class is not created

   Perform the following steps in the wizard:

   Discover Disks

   Figure 1.5. Discovery Disks wizard page

   

   1. Choose one of the following:

      • All nodes to discover disks in all the nodes.

      • Select nodes to choose a subset of nodes from the nodes listed.

        To find specific worker nodes in the cluster, you can filter nodes on the basis of Name or Label. Name allows you to search by name of the node and Label allows you to search by selecting the predefined label.

        It is recommended that the worker nodes are spread across three different physical nodes, racks or failure domains for high availability.

        Note

        Ensure OpenShift Container Storage rack labels are aligned with physical racks in the datacenter to prevent a double node failure at the failure domain level.

        If the nodes selected do not match the OpenShift Container Storage cluster requirement of an aggregated 30 CPUs and 72 GiB of RAM, a minimal cluster will be deployed. For minimum starting node requirements, see Resource requirements section in Planning guide.

        Note

        If the nodes to be selected are tainted and not discovered in the wizard, follow the steps provided in the Red Hat Knowledgebase Solution as a workaround.

   2. Click Next.

   Create Storage Class

   Figure 1.6. Create Storage Class wizard page

   

   1. Enter the Volume set name.

   2. Enter the Storage class name. By default, the Volume set name appears for the storage class name.

      You might need to wait for a couple of seconds for the nodes to get populated.

   3. To select the nodes from the discovered nodes, you can choose one of the following:

      • All nodes to select all the nodes for which you discovered the available disks.
      • Select nodes to choose a subset of the nodes for which you discovered the available disks.

   4. (Optional) You can view the selected capacity of the disks on the selected nodes using the donut chart. Also, you can click on the Nodes and Disks links on the chart which brings up the list of nodes and disks. You can view more details about the selected nodes and disks.

      
      
   5. Select the Disk type.

   6. In the Advanced section, you can set the following options:

      Disk Mode	Block is selected by default.
      Disk Size	Minimum and maximum size of the device that needs to be included in Max and Min respectively.
      Max Disk Limit	This indicates the maximum number of persistent volumes (PVs) that can be created on a node. If this field is left empty, then PVs are created for all the available disks on the matching nodes.

   7. Click Next.

   8. Click Yes in the message alert to confirm the creation of the storage class.

      After the local volume set and storage class are created, it is not possible to go back to the step.

   Local Storage Operator is installed and the storage class is created

   Create Storage Cluster

   Figure 1.7. Create Storage Cluster wizard page

   

   1. Select the required storage class.

      You might need to wait a couple of minutes for the storage nodes corresponding to the selected storage class to get populated.

   2. (Optional) In the Encryption section, set the toggle to Enabled to enable data encryption on the cluster.
   3. The nodes corresponding to the storage class are displayed based on the storage class that you selected from the drop down list.

   4. Click Create.

      The Create button is enabled only when three nodes are selected. A new storage cluster of three volumes will be created with one volume per worker node. The default configuration uses a replication factor of 3.

      To expand the capacity of the initial cluster, see Scaling Storage guide.