Chapter 9. Reclaiming space on target volumes

You can access the actual available space using the reclaim space operation on a target volume. The reclaim operation removes the ambiguity with the available storage space when a file or data is deleted on the persistent volume and this operation is available for the Ceph RBD. Since the object still remains on the RBD device and the storage is not released to the Ceph cluster, this gives an incorrect information of the actual available storage space.

Reclaim space operation triggers rbd sparsify on the RBD device, which in turn reclaims the space for zeroed image extents. For filesystem mode, fstrim is also performed on the filesystem mounted on the block device. When fstrim is enabled on the filesystem then you can create a PVC with ReadWriteOnce (RWO) access that is based on the Ceph RBD with the following mode:

  • Volume mode Block
  • Volume mode Filesystem
Note

For newly installed clusters the reclaim space operation is enabled by default.

To enable the reclaim space operation for upgraded clusters, set CSI_ENABLE_CSIADDONS: "true" in the rook-ceph-operator-override configmap in openshift-storage namespace. 

$ oc patch cm rook-ceph-operator-config -n openshift-storage -p $'data:\n "CSI_ENABLE_CSIADDONS": "true"'

You can use any one of the following three methods to reclaim the space:

  • Enabling reclaim space operation using ReclaimSpaceJob
  • Enabling reclaim space operation using ReclaimSpaceCronJob
  • Enabling reclaim space operation using Annotating PersistentVolumeClaims (Red Hat recommends using this method for enabling reclaim space operation)

9.1. Enabling reclaim space operation using ReclaimSpaceJob

ReclaimSpaceJob is a namespaced custom resource designed to invoke reclaim space operation on the target volume. This is a one time method that immediately starts the reclaim space operation. You have to repeat the creation of ReclaimSpaceJob CR to repeat the reclaim space operation when required.

Procedure

  1. Create and apply the following custom resource for reclaim space operation: 

    apiVersion: csiaddons.openshift.io/v1alpha1
kind: ReclaimSpaceJob
metadata:
  name: sample-1
spec:
  target:
    persistentVolumeClaim: pvc-1

    The variable used in the above yaml are as follows:

    • target indicates the volume target on which the operation is going to perform.

      • persistentVolumeClaim contains a string indicating the name of PersistentVolumeClaim.
    • backOfflimit specifies the maximum number of retries before the reclaim space operation fails. If not specified, default value sets to 6. The maximum and minimum allowed value are 60 and 0.
    • retryDeadlineSeconds specifies the duration in which the operation might retire relative to the start time. Its value must be a positive integer and unit of time is in seconds. If not specified, the default is set to 600 seconds. The maximum allowed value is 1800.
  2. Delete the customer resource after completion of the operation.

9.2. Enabling reclaim space operation using ReclaimSpaceCronJob

ReclaimSpaceCronJob invokes the reclaim space operation based on the given schedule(daily, weekly, etc). You have to create ReclaimSpaceCronJob one time only for a persistent volume claim. The CSI-addons controller creates a ReclaimSpaceJob at the requested time and interval with the schedule attribute.

Procedure

  1. Create and apply the following custom resource for reclaim space operation 

    apiVersion: csiaddons.openshift.io/v1alpha1
kind: ReclaimSpaceCronJob
metadata:
  name: reclaimspacecronjob-sample
spec:
  jobTemplate:
    spec:
      target:
        persistentVolumeClaim: data-pvc
  schedule: '@weekly'

    The variable used in the above yaml are as follows:

    • failedJobsHistoryLimit keeps the number of failed ReclaimSpaceJobs.
    • jobTemplate contains ReclaimSpaceJob.spec structure which includes the requested ReclaimSpaceJob operation.
    • schedule sets the and/or interval of the recurring operation, see Kubernetes CronJobs for the schedule format.
    • successfulJobsHistoryLimit keeps the number of successful ReclaimSpaceJob operations.
  2. Delete the customer resource after completion of the operation.

9.3. Enabling reclaim space operation using Annotating PersistentVolumeClaims

Use this procedure to annotate PersistentVolumeClaims so that it can invoke the reclaim space operation automatically based on a given schedule. 

reclaimspace.csiaddons.openshift.io/schedule: "@midnight"

Procedure

  1. Get the persistent volume claim (pvc) details. 

    $ oc get pvc data-pvc

    for example: 

    NAME      STATUS   VOLUME                                     CAPACITY   ACCESS MODES   STORAGECLASS                          AGE
data-pvc  Bound    pvc-f37b8582-4b04-4676-88dd-e1b95c6abf74   1Gi        RWO            ocs-storagecluster-ceph-rbd           20h

  2. Add annotation reclaimspace.csiaddons.openshift.io/schedule: "@midnight"` to the PVC to create reclaimspacecronjob. 

    $ oc annotate pvc data-pvc "reclaimspace.csiaddons.openshift.io/schedule=@midnight"

    for example: 

    persistentvolumeclaim/data-pvc annotated

  3. Verify that reclaimspacecronjob is created in the following format "<pvc-name>-xxxxxxx". 

    $ oc get reclaimspacecronjobs.csiaddons.openshift.io

    for example: 

    NAME                    SCHEDULE    SUSPEND   ACTIVE   LASTSCHEDULE   AGE
data-pvc-1642663516   @midnight                                     3s

  4. Modify the schedule to run this job automatically. 

    $ oc annotate pvc data-pvc "reclaimspace.csiaddons.openshift.io/schedule=*/1 * * * *" --overwrite=true

    for example: 

    persistentvolumeclaim/data-pvc annotated

  5. Verify that the schedule for reclaimspacecronjob has been modified. 

    $ oc get reclaimspacecronjobs.csiaddons.openshift.io

    for example: 

    NAME                  SCHEDULE    SUSPEND   ACTIVE   LASTSCHEDULE   AGE
data-pvc-1642664617   */1 * * * *                                   3s

The example provides the following details:

  • Name of newly created ReclaimSpaceCronJob as data-pvc-1642664617.
  • schedule sets the and/or interval of the recurring operation, see Kubernetes CronJobs for the schedule format.
  • If the schedule value is empty or in an invalid format, then the default schedule value is set "@weekly".
Note

ReclaimSpaceCronJob is recreated when the schedule is modified and deleted when the annotation is removed.