Chapter 3. Creating a deduplicated and compressed logical volume

Procedure

1. Pick a name for your VDO LV, such as vdo1. You must use a different name and device for each VDO LV on the system.

   In all the following steps, replace vdo-name with the name.

2. Create the VDO LV:

   # lvcreate --type vdo \
              --name vdo-name
              --size physical-size
              --virtualsize logical-size \
              vg-name

   • Replace vg-name with the name of an existing LVM volume group where you want to place the VDO LV.
   • Replace logical-size with the amount of logical storage that the VDO LV will present.

   • If the physical size is larger than 16TiB, add the following option to increase the slab size on the volume to 32GiB:

     --config 'allocation/vdo_slab_size_mb=32768'

     If you use the default slab size of 2GiB on a physical size larger than 16TiB, the lvcreate command fails with the following error:

     ERROR - vdoformat: formatVDO failed on '/dev/device': VDO Status: Exceeds maximum number of slabs supported

     Example 3.1. Creating a VDO LV for container storage

     For example, to create a VDO LV for container storage on a 1TB VDO pool LV, you can use:

     # lvcreate --type vdo \
                --name vdo1
                --size 1T
                --virtualsize 10T \
                vg-name

     Important

     If a failure occurs when creating the VDO volume, remove the volume to clean up.

3. Create a file system on the VDO LV:

   • For the XFS file system:

     # mkfs.xfs -K /dev/vg-name/vdo-name

   • For the ext4 file system:

     # mkfs.ext4 -E nodiscard /dev/vg-name/vdo-name

Procedure

1. To find out if the compression and deduplication is enabled or disabled on your logical volumes:

   # lvs -o+vdo_compression,vdo_deduplication

2. Find out status of the compression and status of the deduplication index of your running active VDOPoolLV:

   # lvs -o+vdo_compression_state,vdo_index_state

   The vdo_index_state can show as error, close, opening, closing, online, and offline.

3. To enable or disable the compression for VDOPoolLV:

   # lvchange --compression y|n  vg-name/vdopoolname

4. To enable or disable the deduplication for VDOPoolLV:

   # lvchange --deduplication y|n vg-name/vdopoolname

Prerequisites

1. Install the VDO software. For more information, see Installing VDO.
2. An LVM volume group with free storage capacity exists on your system.
3. A thin-provisioned VDO volume using the lvcreate --type vdo --name myvdo myvg -L logical-size-of-pool --virtualsize virtual-size-of-vdo command. For more information, see Creating an LVM-VDO volume.

Procedure

1. Determine the optimal logical size for a thin-provisioned VDO volume

   # vdostats myvg-vpool0-vpool
   
   Device               1K-blocks Used     Available  Use% Space saving%
   myvg-vpool0-vpool   104856576  29664088 75192488   28%   69%

   To calculate the space savings ratio, use the following formula:

   Savings ratio = 1 / (1 - Space saving%)

   In this example,

   • there is approximately a 3.22:1 space savings ratio on a data set of about 80 GB.
   • Multiplying the data set size by the ratio would yield a potential logical size of 256 GB if more data with the same space savings were written to the VDO volume.
   • Adjusting this number downward to 200 GB yields a logical size with a safe margin of free physical space, given the same space savings ratio.

2. Monitor the free physical space in a VDO volume:

   # vdostats myvg-vpool0-vpool

   This command can be executed periodically to provide monitoring of the used and free physical space of the VDO volume.

3. Optional: View the warnings on physical space usage on a VDO volume by using the available /usr/share/doc/vdo/examples/monitor/monitor_check_vdostats_physicalSpace.pl script:

   # /usr/share/doc/vdo/examples/monitor/monitor_check_vdostats_physicalSpace.pl myvg-vpool0-vpool

4. When creating a VDO volume, the dmeventd monitoring service monitors the usage of physical space in a VDO volume. This is enabled by default when a VDO volume is created or started.

   Use the journalctl command to view the output of dmeventd in the logs while monitoring a VDO volume:

   lvm[8331]: Monitoring VDO pool myvg-vpool0-vpool.
   ...
   
   lvm[8331]: WARNING: VDO pool myvg-vpool0-vpool is now 84.63% full.
   lvm[8331]: WARNING: VDO pool myvg-vpool0-vpool is now 91.01% full.
   lvm[8331]: WARNING: VDO pool myvg-vpool0-vpool is now 97.34% full.

5. Remediate VDO volumes that are almost out of available physical space. When it is possible to add a physical space to a VDO volume, but the volume space is full before it can be grown, it may be necessary to temporarily stop I/O to the volume.

   To temporarily stop I/O to the volume, execute the following steps, where VDO volume myvdo contains a file system mounted on the /users/homeDir path:

   1. Freeze the filesystem:

      # xfs_freeze -f /users/homeDir
      
      # vgextend myvg /dev/vdc2
      
      # lvextend -l new_size myvg/vpool0-name
      
      # xfs_freeze -u /users/homeDir

   2. Unmount the filesystem:

      # umount /users/homeDir
      
      # vgextend myvg /dev/vdc2
      
      # lvextend -l new_size myvg/vpool0-name
      
      # mount -o discard /dev/myvg/myvdo /users/homeDir

      Note

      Unmounting or freezing a filesystem with cached data will incur a write of the cached data, which may fill the physical space of the VDO volume. Consider the maximum amount of cached filesystem data when setting a monitoring threshold for free physical space on a VDO volume.

6. Blocks that are no longer used by a file system can be cleaned up by using the fstrim utility. Executing fstrim against a mounted file system on top of a VDO volume may result in increased free physical space for that volume. The fstrim utility will send discards to the VDO volume, which are then used to remove references to the previously used blocks. If any of those blocks are single-referenced, the physical space will be available to use.

   1. Check VDO stats to see what the current amount of free space is:

      # vdostats --human-readable myvg-vpool0-vpool
      
       Device             Size  Used  Available Use%  Space saving%
      myvg-vpool0-vpool  100.0G 95.0G 5.0G      95%   73%

   2. Discard unused blocks:

      # fstrim /users/homeDir

   3. View the free physical space of the VDO volume:

      # vdostats --human-readable myvg-vpool0-vpool
      
      
       Device             Size    Used   Available Use%  Space saving%
      myvg-vpool0-vpool  100.0G   30.0G  70.0G     30%    43%

      In this example, after executing fstrim on the file system, the discards were able to return 65G of physical space to use in the VDO volume.

      Note

      Discarding volumes with lower levels of deduplication and compression will have a possibility of reclaiming physical space than discarding volumes with higher levels of deduplication and compression. A volume that has high levels of deduplication and compression may potentially require a more extensive cleanup to reclaim physical space than just simply discarding already unused blocks.