Chapter 2. Scaling up storage capacity
Depending on the type of your deployment, you can choose one of the following procedures to scale up storage capacity.
- For AWS, VMware, Red Hat Virtualization or Azure infrastructures using dynamic or automated provisioning of storage devices, see Section 2.2, “Scaling up storage by adding capacity to your OpenShift Data Foundation nodes”.
- For bare metal, VMware or Red Hat Virtualization infrastructures using local storage devices, see Section 2.3, “Scaling up storage by adding capacity to your OpenShift Data Foundation nodes using local storage devices”.
- For IBM Z or LinuxONE infrastructures using local storage devices, see Section 2.4, “Scaling up storage by adding capacity to your OpenShift Data Foundation nodes on IBM Z or LinuxONE infrastructure”.
- For IBM Power using local storage devices, see Section 2.5, “Scaling up storage by adding capacity to your OpenShift Data Foundation nodes on IBM Power infrastructure using local storage devices”.
If you want to scale using a storage class other than the one provisioned during deployment, you must also define an additional storage class before you scale. See Creating a storage class for details.
OpenShift Data Foundation does not support heterogeneous OSD sizes.
2.1. Creating a Storage Class
You can define a new storage class to dynamically provision storage from an existing provider.
Prerequisites
- Administrator access to OpenShift web console.
Procedure
- Log in to OpenShift Web Console.
- Click Storage → StorageClasses.
Click Create Storage Class.
- Enter the storage class Name and Description.
- Select the required Reclaim Policy and Provisioner.
- Click Create to create the Storage Class.
Verification steps
- Click Storage → StorageClasses and verify that you can see the new storage class.
2.2. Scaling up storage by adding capacity to your OpenShift Data Foundation nodes
You can add storage capacity and performance to your configured Red Hat OpenShift Data Foundation worker nodes on the following infrastructures:
- AWS
- VMware vSphere
- Red Hat Virtualization
- Microsoft Azure
Prerequisites
- A running OpenShift Data Foundation Platform.
- Administrative privileges on the OpenShift Web Console.
- To scale using a storage class other than the one provisioned during deployment, first define an additional storage class. See Creating a storage class for details.
Procedure
- Log in to the OpenShift Web Console.
- Click Operators → Installed Operators.
- Click OpenShift Data Foundation Operator.
Click the Storage Systems tab.
- Click the Action Menu (⋮) on the far right of the storage system name to extend the options menu.
- Select Add Capacity from the options menu.
Select the Storage Class.
Set the storage class to
gp2
on AWS,thin
on VMware,ovirt-csi-sc
on Red Hat Virtualization ormanaged_premium
on Microsoft Azure if you are using the default storage class generated during deployment. If you have created other storage classes, select whichever is appropriate.ImportantUsing storage classes other than the default for your provider is a Technology Preview feature.
Technology Preview features are not supported with Red Hat production service level agreements (SLAs) and might not be functionally complete. Red Hat does not recommend using them in production. These features provide early access to upcoming product features, enabling customers to test functionality and provide feedback during the development process.
For more information, see Technology Preview Features Support Scope.
The Raw Capacity field shows the size set during storage class creation. The total amount of storage consumed is three times this amount, because OpenShift Data Foundation uses a replica count of 3.
- Click Add.
-
To check the status, navigate to Storage → OpenShift Data Foundation and verify that
Storage System
in the Status card has a green tick.
Verification steps
Verify the Raw Capacity card.
- In the OpenShift Web Console, click Storage → OpenShift Data Foundation.
- In the Status card of the Overview tab, click Storage System and then click the storage system link from the pop up that appears.
In the Block and File tab, check the Raw Capacity card.
Note that the capacity increases based on your selections.
NoteThe raw capacity does not take replication into account and shows the full capacity.
Verify that the new OSDs and their corresponding new Persistent Volume Claims (PVCs) are created.
To view the state of the newly created OSDs:
- Click Workloads → Pods from the OpenShift Web Console.
Select
openshift-storage
from the Project drop-down list.NoteIf the Show default projects option is disabled, use the toggle button to list all the default projects.
To view the state of the PVCs:
- Click Storage → Persistent Volume Claims from the OpenShift Web Console.
Select
openshift-storage
from the Project drop-down list.NoteIf the Show default projects option is disabled, use the toggle button to list all the default projects.
Optional: If cluster-wide encryption is enabled on the cluster, verify that the new OSD devices are encrypted.
Identify the nodes where the new OSD pods are running.
$ oc get -o=custom-columns=NODE:.spec.nodeName pod/<OSD-pod-name>
<OSD-pod-name>
Is the name of the OSD pod.
For example:
oc get -o=custom-columns=NODE:.spec.nodeName pod/rook-ceph-osd-0-544db49d7f-qrgqm
For each of the nodes identified in the previous step, do the following:
Create a debug pod and open a chroot environment for the selected hosts.
$ oc debug node/<node-name>
<node-name>
Is the name of the node.
$ chroot /host
Check for the
crypt
keyword beside theocs-deviceset
names.$ lsblk
Cluster reduction is supported only with the Red Hat Support Team’s assistance..
2.3. Scaling up storage by adding capacity to your OpenShift Data Foundation nodes using local storage devices
You can add storage capacity (additional storage devices) to your configured local storage based OpenShift Data Foundation worker nodes on the following infrastructures:
- Bare metal
- VMware
- Red Hat Virtualization
Prerequisites
- You must be logged into the OpenShift Container Platform cluster.
You must have installed the local storage operator. Use any of the following procedures applicable to your infrastructure:
-
If you upgraded to OpenShift Data Foundation version 4.9 from a previous version, and have not already created the
LocalVolumeDiscovery
andLocalVolumeSet
objects, follow the procedure described in Post-update configuration changes for clusters backed by local storage. - You must have three OpenShift Container Platform worker nodes with the same storage type and size attached to each node (for example, 2TB NVMe drive) as the original OpenShift Data Foundation StorageCluster was created with.
Procedure
To add capacity, you can either use a storage class that you provisioned during the deployment or any other storage class that matches the filter.
- In the OpenShift Web Console, click Operators → Installed Operators.
- Click OpenShift Data Foundation Operator.
Click Storage Systems tab.
- Click the Action menu (⋮) next to the visible list to extend the options menu.
- Select Add Capacity from the options menu.
- Select the Storage Class for which you added disks or the new storage class depending on your requirement. Available Capacity displayed is based on the local disks available in storage class.
- Click Add.
- To check the status, navigate to Storage → OpenShift Data Foundation and verify that Storage System in the Status card has a green tick.
Verification steps
Verify the Raw Capacity card.
- In the OpenShift Web Console, click Storage → OpenShift Data Foundation.
- In the Status card of the Overview tab, click Storage System and then click the storage system link from the pop up that appears.
In the Block and File tab, check the Raw Capacity card.
Note that the capacity increases based on your selections.
NoteThe raw capacity does not take replication into account and shows the full capacity.
Verify that the new OSDs and their corresponding new Persistent Volume Claims (PVCs) are created.
To view the state of the newly created OSDs:
- Click Workloads → Pods from the OpenShift Web Console.
Select
openshift-storage
from the Project drop-down list.NoteIf the Show default projects option is disabled, use the toggle button to list all the default projects.
To view the state of the PVCs:
- Click Storage → Persistent Volume Claims from the OpenShift Web Console.
Select
openshift-storage
from the Project drop-down list.NoteIf the Show default projects option is disabled, use the toggle button to list all the default projects.
Optional: If cluster-wide encryption is enabled on the cluster, verify that the new OSD devices are encrypted.
Identify the node(s) where the new OSD pod(s) are running.
$ oc get -o=custom-columns=NODE:.spec.nodeName pod/<OSD-pod-name>
<OSD-pod-name>
Is the name of the OSD pod.
For example:
oc get -o=custom-columns=NODE:.spec.nodeName pod/rook-ceph-osd-0-544db49d7f-qrgqm
For each of the nodes identified in the previous step, do the following:
Create a debug pod and open a chroot environment for the selected host(s).
$ oc debug node/<node-name>
<node-name>
Is the name of the node.
$ chroot /host
Check for the
crypt
keyword beside theocs-deviceset
names.$ lsblk
Cluster reduction is supported only with the Red Hat Support Team’s assistance.
2.4. Scaling up storage by adding capacity to your OpenShift Data Foundation nodes on IBM Z or LinuxONE infrastructure
You can add storage capacity and performance to your configured Red Hat OpenShift Data Foundation worker nodes.
Prerequisites
- A running OpenShift Data Foundation Platform.
- Administrative privileges on the OpenShift Web Console.
- To scale using a storage class other than the one provisioned during deployment, first define an additional storage class. See Creating a storage class for details.
Procedure
Add additional hardware resources with zFCP disks.
List all the disks.
$ lszdev
Example output:
TYPE ID ON PERS NAMES zfcp-host 0.0.8204 yes yes zfcp-lun 0.0.8204:0x102107630b1b5060:0x4001402900000000 yes no sda sg0 zfcp-lun 0.0.8204:0x500407630c0b50a4:0x3002b03000000000 yes yes sdb sg1 qeth 0.0.bdd0:0.0.bdd1:0.0.bdd2 yes no encbdd0 generic-ccw 0.0.0009 yes no
A SCSI disk is represented as a
zfcp-lun
with the structure<device-id>:<wwpn>:<lun-id>
in the ID section. The first disk is used for the operating system. The device id for the new disk can be the same.Append a new SCSI disk.
$ chzdev -e 0.0.8204:0x400506630b1b50a4:0x3001301a00000000
NoteThe device ID for the new disk must be the same as the disk to be replaced. The new disk is identified with its WWPN and LUN ID.
List all the FCP devices to verify the new disk is configured.
$ lszdev zfcp-lun TYPE ID ON PERS NAMES zfcp-lun 0.0.8204:0x102107630b1b5060:0x4001402900000000 yes no sda sg0 zfcp-lun 0.0.8204:0x500507630b1b50a4:0x4001302a00000000 yes yes sdb sg1 zfcp-lun 0.0.8204:0x400506630b1b50a4:0x3001301a00000000 yes yes sdc sg2
- Navigate to the OpenShift Web Console.
- Click Operators on the left navigation bar.
- Select Installed Operators.
- In the window, click OpenShift Data Foundation Operator.
In the top navigation bar, scroll right and click Storage Systems tab.
- Click the Action menu (⋮) next to the visible list to extend the options menu.
Select Add Capacity from the options menu.
The Raw Capacity field shows the size set during storage class creation. The total amount of storage consumed is three times this amount, because OpenShift Data Foundation uses a replica count of 3.
- Click Add.
- To check the status, navigate to Storage → OpenShift Data Foundation and verify that Storage System in the Status card has a green tick.
Verification steps
Verify the Raw Capacity card.
- In the OpenShift Web Console, click Storage → OpenShift Data Foundation.
- In the Status card of the Overview tab, click Storage System and then click the storage system link from the pop up that appears.
In the Block and File tab, check the Raw Capacity card.
Note that the capacity increases based on your selections.
NoteThe raw capacity does not take replication into account and shows the full capacity.
Verify that the new OSDs and their corresponding new Persistent Volume Claims (PVCs) are created.
To view the state of the newly created OSDs:
- Click Workloads → Pods from the OpenShift Web Console.
Select
openshift-storage
from the Project drop-down list.NoteIf the Show default projects option is disabled, use the toggle button to list all the default projects.
To view the state of the PVCs:
- Click Storage → Persistent Volume Claims from the OpenShift Web Console.
Select
openshift-storage
from the Project drop-down list.NoteIf the Show default projects option is disabled, use the toggle button to list all the default projects.
Optional: If cluster-wide encryption is enabled on the cluster, verify that the new OSD devices are encrypted.
Identify the node(s) where the new OSD pod(s) are running.
$ oc get -o=custom-columns=NODE:.spec.nodeName pod/<OSD-pod-name>
<OSD-pod-name>
Is the name of the OSD pod.
For example:
oc get -o=custom-columns=NODE:.spec.nodeName pod/rook-ceph-osd-0-544db49d7f-qrgqm
For each of the nodes identified in the previous step, do the following:
Create a debug pod and open a chroot environment for the selected host(s).
$ oc debug node/<node-name>
<node-name>
Is the name of the node.
$ chroot /host
Check for the
crypt
keyword beside theocs-deviceset
names.$ lsblk
Cluster reduction is supported only with the Red Hat Support Team’s assistance..
2.5. Scaling up storage by adding capacity to your OpenShift Data Foundation nodes on IBM Power infrastructure using local storage devices
You can add storage capacity (additional storage devices) to your configured local storage based OpenShift Data Foundation worker nodes on IBM Power infrastructures.
Prerequisites
- You must be logged into OpenShift Container Platform cluster.
You must have installed the local storage operator. Use the following procedure:
- You must have three OpenShift Container Platform worker nodes with the same storage type and size attached to each node (for example, 0.5TB SSD) as the original OpenShift Data Foundation StorageCluster was created with.
Procedure
To add storage capacity to OpenShift Container Platform nodes with OpenShift Data Foundation installed, you need to
Find the available devices that you want to add, that is, a minimum of one device per worker node. You can follow the procedure for finding available storage devices in the respective deployment guide.
NoteMake sure you perform this process for all the existing nodes (minimum of 3) for which you want to add storage.
Add the additional disks to the
LocalVolume
custom resource (CR).$ oc edit -n openshift-local-storage localvolume localblock
Example output:
spec: logLevel: Normal managementState: Managed nodeSelector: nodeSelectorTerms: - matchExpressions: - key: kubernetes.io/hostname operator: In values: - worker-0 - worker-1 - worker-2 storageClassDevices: - devicePaths: - /dev/sda - /dev/sdx # newly added device storageClassName: localblock volumeMode: Block
Make sure to save the changes after editing the CR.
Example output:
localvolume.local.storage.openshift.io/localblock edited
You can see in this CR that new devices are added.
-
sdx
-
Display the newly created Persistent Volumes (PVs) with the
storageclass
name used in thelocalVolume
CR.$ oc get pv | grep localblock | grep Available
Example output:
local-pv-a04ffd8 500Gi RWO Delete Available localblock 24s local-pv-a0ca996b 500Gi RWO Delete Available localblock 23s local-pv-c171754a 500Gi RWO Delete Available localblock 23s
- Navigate to the OpenShift Web Console.
- Click Operators on the left navigation bar.
- Select Installed Operators.
- In the window, click OpenShift Data Foundation Operator.
In the top navigation bar, scroll right and click Storage System tab.
- Click the Action menu (⋮) next to the visible list to extend the options menu.
Select Add Capacity from the options menu.
From this dialog box, set the Storage Class name to the name used in the
localVolume
CR. Available Capacity displayed is based on the local disks available in storage class.- Click Add.
- To check the status, navigate to Storage → OpenShift Data Foundation and verify that Storage System in the Status card has a green tick.
Verification steps
Verify the available Capacity.
- In the OpenShift Web Console, click Storage → OpenShift Data Foundation.
-
Click the Storage Systems tab and then click on
ocs-storagecluster-storagesystem
. Navigate to Overview → Block and File tab, then check the Raw Capacity card.
Note that the capacity increases based on your selections.
NoteThe raw capacity does not take replication into account and shows the full capacity.
Verify that the new OSDs and their corresponding new Persistent Volume Claims (PVCs) are created.
To view the state of the newly created OSDs:
- Click Workloads → Pods from the OpenShift Web Console.
Select
openshift-storage
from the Project drop-down list.NoteIf the Show default projects option is disabled, use the toggle button to list all the default projects.
To view the state of the PVCs:
- Click Storage → Persistent Volume Claims from the OpenShift Web Console.
Select
openshift-storage
from the Project drop-down list.NoteIf the Show default projects option is disabled, use the toggle button to list all the default projects.
Optional: If cluster-wide encryption is enabled on the cluster, verify that the new OSD devices are encrypted.
Identify the node(s) where the new OSD pod(s) are running.
$ oc get -o=custom-columns=NODE:.spec.nodeName pod/<OSD-pod-name>
<OSD-pod-name>
Is the name of the OSD pod.
For example:
oc get -o=custom-columns=NODE:.spec.nodeName pod/rook-ceph-osd-0-544db49d7f-qrgqm
For each of the nodes identified in the previous step, do the following:
Create a debug pod and open a chroot environment for the selected host(s).
$ oc debug node/<node-name>
<node-name>
Is the name of the node.
$ chroot /host
Check for the
crypt
keyword beside theocs-deviceset
names.$ lsblk
Cluster reduction is supported only with the Red Hat Support Team’s assistance.