Chapter 9. Reclaiming space on target volumes
You can access the actual available space using the reclaim space operation on a target volume. The reclaim operation removes the ambiguity with the available storage space when a file or data is deleted on the persistent volume and this operation is available for the Ceph RBD. Since the object still remains on the RBD device and the storage is not released to the Ceph cluster, this gives an incorrect information of the actual available storage space.
Reclaim space operation triggers rbd sparsify on the RBD device, which in turn reclaims the space for zeroed image extents. For filesystem mode, fstrim is also performed on the filesystem mounted on the block device. When fstrim is enabled on the filesystem then you can create a PVC with ReadWriteOnce (RWO) access that is based on the Ceph RBD with the following mode:
- Volume mode Block
- Volume mode Filesystem
For newly installed clusters the reclaim space operation is enabled by default.
To enable the reclaim space operation for upgraded clusters, set CSI_ENABLE_CSIADDONS: "true" in the rook-ceph-operator-override configmap in openshift-storage namespace.
$ oc patch cm rook-ceph-operator-config -n openshift-storage -p $'data:\n "CSI_ENABLE_CSIADDONS": "true"'
You can use any one of the following three methods to reclaim the space:
- Enabling reclaim space operation using ReclaimSpaceJob
- Enabling reclaim space operation using ReclaimSpaceCronJob
- Enabling reclaim space operation using Annotating PersistentVolumeClaims (Red Hat recommends using this method for enabling reclaim space operation)
9.1. Enabling reclaim space operation using ReclaimSpaceJob
ReclaimSpaceJob is a namespaced custom resource designed to invoke reclaim space operation on the target volume. This is a one time method that immediately starts the reclaim space operation. You have to repeat the creation of ReclaimSpaceJob CR to repeat the reclaim space operation when required.
Procedure
Create and apply the following custom resource for reclaim space operation:
apiVersion: csiaddons.openshift.io/v1alpha1 kind: ReclaimSpaceJob metadata: name: sample-1 spec: target: persistentVolumeClaim: pvc-1The variable used in the above yaml are as follows:
targetindicates the volume target on which the operation is going to perform.-
persistentVolumeClaimcontains a string indicating the name ofPersistentVolumeClaim.
-
-
backOfflimitspecifies the maximum number of retries before the reclaim space operation fails. If not specified, default value sets to 6. The maximum and minimum allowed value are 60 and 0. -
retryDeadlineSecondsspecifies the duration in which the operation might retire relative to the start time. Its value must be a positive integer and unit of time is in seconds. If not specified, the default is set to 600 seconds. The maximum allowed value is 1800.
- Delete the customer resource after completion of the operation.
9.2. Enabling reclaim space operation using ReclaimSpaceCronJob
ReclaimSpaceCronJob invokes the reclaim space operation based on the given schedule(daily, weekly, etc). You have to create ReclaimSpaceCronJob one time only for a persistent volume claim. The CSI-addons controller creates a ReclaimSpaceJob at the requested time and interval with the schedule attribute.
Procedure
Create and apply the following custom resource for reclaim space operation
apiVersion: csiaddons.openshift.io/v1alpha1 kind: ReclaimSpaceCronJob metadata: name: reclaimspacecronjob-sample spec: jobTemplate: spec: target: persistentVolumeClaim: data-pvc schedule: '@weekly'The variable used in the above yaml are as follows:
-
failedJobsHistoryLimitkeeps the number of failedReclaimSpaceJobs. -
jobTemplatecontains ReclaimSpaceJob.spec structure which includes the requestedReclaimSpaceJoboperation. -
schedulesets the and/or interval of the recurring operation, see Kubernetes CronJobs for the schedule format. -
successfulJobsHistoryLimitkeeps the number of successfulReclaimSpaceJoboperations.
-
- Delete the customer resource after completion of the operation.
9.3. Enabling reclaim space operation using Annotating PersistentVolumeClaims
Use this procedure to annotate PersistentVolumeClaims so that it can invoke the reclaim space operation automatically based on a given schedule.
reclaimspace.csiaddons.openshift.io/schedule: "@midnight"
Procedure
Get the persistent volume claim (pvc) details.
$ oc get pvc data-pvc
for example:
NAME STATUS VOLUME CAPACITY ACCESS MODES STORAGECLASS AGE data-pvc Bound pvc-f37b8582-4b04-4676-88dd-e1b95c6abf74 1Gi RWO ocs-storagecluster-ceph-rbd 20h
Add annotation
reclaimspace.csiaddons.openshift.io/schedule: "@midnight"`to the PVC to createreclaimspacecronjob.$ oc annotate pvc data-pvc "reclaimspace.csiaddons.openshift.io/schedule=@midnight"
for example:
persistentvolumeclaim/data-pvc annotated
Verify that
reclaimspacecronjobis created in the following format "<pvc-name>-xxxxxxx".$ oc get reclaimspacecronjobs.csiaddons.openshift.io
for example:
NAME SCHEDULE SUSPEND ACTIVE LASTSCHEDULE AGE data-pvc-1642663516 @midnight 3s
Modify the schedule to run this job automatically.
$ oc annotate pvc data-pvc "reclaimspace.csiaddons.openshift.io/schedule=*/1 * * * *" --overwrite=true
for example:
persistentvolumeclaim/data-pvc annotated
Verify that the schedule for
reclaimspacecronjobhas been modified.$ oc get reclaimspacecronjobs.csiaddons.openshift.io
for example:
NAME SCHEDULE SUSPEND ACTIVE LASTSCHEDULE AGE data-pvc-1642664617 */1 * * * * 3s
The example provides the following details:
-
Name of newly created
ReclaimSpaceCronJobas data-pvc-1642664617. -
schedulesets the and/or interval of the recurring operation, see Kubernetes CronJobs for the schedule format. - If the schedule value is empty or in an invalid format, then the default schedule value is set "@weekly".
ReclaimSpaceCronJob is recreated when the schedule is modified and deleted when the annotation is removed.