Chapter 3. Creating a deduplicated and compressed logical volume

You can create an LVM logical volume that uses the VDO feature to deduplicate and compress data.

3.1. LVM-VDO deployment scenarios

You can deploy VDO on LVM (LVM-VDO) in a variety of ways to provide deduplicated storage for:

  • block access
  • file access
  • local storage
  • remote storage

Because LVM-VDO exposes its deduplicated storage as a regular logical volume (LV), you can use it with standard file systems, iSCSI and FC target drivers, or as unified storage.

Note

Deploying Ceph Storage on LVM-VDO is currently not supported.

KVM

You can deploy LVM-VDO on a KVM server configured with Direct Attached Storage.

LVM-VDO deployment with KVM
File systems

You can create file systems on top of a VDO LV and expose them to NFS or CIFS users with the NFS server or Samba.

LVM-VDO deployment with NAS
iSCSI target

You can export the entirety of the VDO LV as an iSCSI target to remote iSCSI initiators.

LVM-VDO deployment with block storage target
Encryption

Device Mapper (DM) mechanisms such as DM Crypt are compatible with VDO. Encrypting a VDO LV volumes helps ensure data security, and any file systems above the VDO LV are still deduplicated.

LVM-VDO deployment with encryption
Important

Applying the encryption layer above the VDO LV results in little if any data deduplication. Encryption makes duplicate blocks different before VDO can deduplicate them.

Always place the encryption layer below the VDO LV.

3.2. The physical and logical size of an LVM-VDO volume

This section describes the physical size, available physical size, and logical size that VDO can utilize.

Physical size

This is the same size as the physical extents allocated to the VDO pool LV. VDO uses this storage for:

  • User data, which might be deduplicated and compressed
  • VDO metadata, such as the UDS index
Available physical size

This is the portion of the physical size that VDO is able to use for user data.

It is equivalent to the physical size minus the size of the metadata, rounded down to a multiple of the slab size.

Logical Size

This is the provisioned size that the VDO LV presents to applications. It is usually larger than the available physical size. VDO currently supports any logical size up to 254 times the size of the physical volume with an absolute maximum logical size of 4 PB.

When you set up a VDO logical volume (LV), you specify the amount of logical storage that the VDO LV presents. When hosting active VMs or containers, Red Hat recommends provisioning storage at a 10:1 logical to physical ratio, that is, if you are utilizing 1 TB of physical storage, you would present it as 10 TB of logical storage.

If you do not specify the --virtualsize option, VDO provisions the volume to a 1:1 ratio. For example, if you put a VDO LV on top of a 20 GB VDO pool LV, VDO reserves 2.5 GB for the UDS index, if the default index size is used. The remaining 17.5 GB is provided for the VDO metadata and user data. As a result, the available storage to consume is not more than 17.5 GB, and can be less due to metadata that makes up the actual VDO volume.

3.3. Slab size in VDO

The physical storage of the VDO volume is divided into a number of slabs. Each slab is a contiguous region of the physical space. All of the slabs for a given volume have the same size, which can be any power of 2 multiple of 128 MB up to 32 GB.

The default slab size is 2 GB to facilitate evaluating VDO on smaller test systems. A single VDO volume can have up to 8192 slabs. Therefore, in the default configuration with 2 GB slabs, the maximum allowed physical storage is 16 TB. When using 32 GB slabs, the maximum allowed physical storage is 256 TB. VDO always reserves at least one entire slab for metadata, and therefore, the reserved slab cannot be used for storing user data.

Slab size has no effect on the performance of the VDO volume.

Table 3.1. Recommended VDO slab sizes by physical volume size

Physical volume sizeRecommended slab size

10–99 GB

1 GB

100 GB – 1 TB

2 GB

2–256 TB

32 GB

Note

The minimal disk usage for a VDO volume using default settings of 2 GB slab size and 0.25 dense index, requires approx 4.7 GB. This provides slightly less than 2 GB of physical data to write at 0% deduplication or compression.

Here, the minimal disk usage is the sum of the default slab size and dense index.

You can control the slab size by providing the --config 'allocation/vdo_slab_size_mb=size-in-megabytes' option to the lvcreate command.

3.4. Installing VDO

This procedure installs software necessary to create, mount, and manage VDO volumes.

Procedure

  • Install the VDO software:

    # yum install lvm2 kmod-kvdo vdo

3.5. Creating an LVM-VDO volume

This procedure creates an VDO logical volume (LV) on a VDO pool LV.

Prerequisites

  • Install the VDO software. For more information, see Installing VDO.
  • An LVM volume group with free storage capacity exists on your system.

Procedure

  1. Pick a name for your VDO LV, such as vdo1. You must use a different name and device for each VDO LV on the system.

    In all the following steps, replace vdo-name with the name.

  2. Create the VDO LV:

    # lvcreate --type vdo \
               --name vdo-name
               --size physical-size
               --virtualsize logical-size \
               vg-name
    • Replace vg-name with the name of an existing LVM volume group where you want to place the VDO LV.
    • Replace logical-size with the amount of logical storage that the VDO LV will present.
    • If the physical size is larger than 16TiB, add the following option to increase the slab size on the volume to 32GiB:

      --config 'allocation/vdo_slab_size_mb=32768'

      If you use the default slab size of 2GiB on a physical size larger than 16TiB, the lvcreate command fails with the following error:

      ERROR - vdoformat: formatVDO failed on '/dev/device': VDO Status: Exceeds maximum number of slabs supported

      Example 3.1. Creating a VDO LV for container storage

      For example, to create a VDO LV for container storage on a 1TB VDO pool LV, you can use:

      # lvcreate --type vdo \
                 --name vdo1
                 --size 1T
                 --virtualsize 10T \
                 vg-name
      Important

      If a failure occurs when creating the VDO volume, remove the volume to clean up.

  3. Create a file system on the VDO LV:

    • For the XFS file system:

      # mkfs.xfs -K /dev/vg-name/vdo-name
    • For the ext4 file system:

      # mkfs.ext4 -E nodiscard /dev/vg-name/vdo-name

Additional resources

  • lvmvdo(7) man page

3.6. Mounting an LVM-VDO volume

This procedure mounts a file system on an LVM-VDO volume, either manually or persistently.

Prerequisites

Procedure

  • To mount the file system on the LVM-VDO volume manually, use:

    # mount /dev/vg-name/vdo-name mount-point
  • To configure the file system to mount automatically at boot, add a line to the /etc/fstab file:

    • For the XFS file system:

      /dev/vg-name/vdo-name mount-point xfs defaults 0 0
    • For the ext4 file system:

      /dev/vg-name/vdo-name mount-point ext4 defaults 0 0

    If the LVM-VDO volume is located on a block device that requires network, such as iSCSI, add the _netdev mount option. For iSCSI and other block devices requiring network, see the systemd.mount(5) man page for information about the _netdev mount option.

Additional resources

  • systemd.mount(5) man page

3.7. Changing the compression and deduplication settings on an LVM-VDO volume

This procedure enables or disables the compression and deduplication of a VDO pool logical volume (LV).

Note

Compression and deduplication are enabled by default.

Prerequisites

  • An LVM-VDO volume exists on your system.

Procedure

  1. To find out if the compression and deduplication is enabled or disabled on your logical volumes:

    # lvs -o+vdo_compression,vdo_deduplication
  2. Find out status of the compression and status of the deduplication index of your running active VDOPoolLV:

    # lvs -o+vdo_compression_state,vdo_index_state

    The vdo_index_state can show as error, close, opening, closing, online, and offline.

  3. To enable or disable the compression for VDOPoolLV:

    # lvchange --compression y|n  vg-name/vdopoolname
  4. To enable or disable the deduplication for VDOPoolLV:

    # lvchange --deduplication y|n vg-name/vdopoolname

Additional resources

  • lvmvdo(7) man page

3.8. Managing thin provisioning with Virtual Data Optimizer

It is possible to configure a thin-provisioned VDO volume to prepare for future expansion of the physical space, in order to address a condition where the physical space usage of the VDO volume is approaching 100%. Instead of using -l 100%FREE in the lvcreate operation, for example, use '95%FREE' to ensure that there is some reserved space for recovery later on if needed. This procedure describes how to resolve the following issues:

  • The volume runs out of space
  • The file system enters read-only mode
  • ENOSPC reported by the volume
Note

The best way to address high physical space usage on a VDO volume is to delete unused files, and discard the blocks used by these unused files either by using online discard or the fstrim program. The physical space of a VDO volume can only be grown to 8192 slabs that is 16 TB for a VDO volume with the default slab size of 2 GB, or 256 TB for a VDO volume with the maximal slab size of 32 GB.

In all of the following steps, replace myvg and myvdo with the volume group and VDO name respectively.

Prerequisites

  1. Install the VDO software. For more information, see Installing VDO.
  2. An LVM volume group with free storage capacity exists on your system.
  3. A thin-provisioned VDO volume using the lvcreate --type vdo --name myvdo myvg -L logical-size-of-pool --virtualsize virtual-size-of-vdo command. For more information, see Creating an LVM-VDO volume.

Procedure

  1. Determine the optimal logical size for a thin-provisioned VDO volume

    # vdostats myvg-vpool0-vpool
    
    Device               1K-blocks Used     Available  Use% Space saving%
    myvg-vpool0-vpool   104856576  29664088 75192488   28%   69%

    To calculate the space savings ratio, use the following formula:

    Savings ratio = 1 / (1 - Space saving%)

    In this example,

    • there is approximately a 3.22:1 space savings ratio on a data set of about 80 GB.
    • Multiplying the data set size by the ratio would yield a potential logical size of 256 GB if more data with the same space savings were written to the VDO volume.
    • Adjusting this number downward to 200 GB yields a logical size with a safe margin of free physical space, given the same space savings ratio.
  2. Monitor the free physical space in a VDO volume:

    # vdostats myvg-vpool0-vpool

    This command can be executed periodically to provide monitoring of the used and free physical space of the VDO volume.

  3. Optional: View the warnings on physical space usage on a VDO volume by using the available /usr/share/doc/vdo/examples/monitor/monitor_check_vdostats_physicalSpace.pl script:

    # /usr/share/doc/vdo/examples/monitor/monitor_check_vdostats_physicalSpace.pl myvg-vpool0-vpool
  4. When creating a VDO volume, the dmeventd monitoring service monitors the usage of physical space in a VDO volume. This is enabled by default when a VDO volume is created or started.

    Use the journalctl command to view the output of dmeventd in the logs while monitoring a VDO volume:

    lvm[8331]: Monitoring VDO pool myvg-vpool0-vpool.
    ...
    
    lvm[8331]: WARNING: VDO pool myvg-vpool0-vpool is now 84.63% full.
    lvm[8331]: WARNING: VDO pool myvg-vpool0-vpool is now 91.01% full.
    lvm[8331]: WARNING: VDO pool myvg-vpool0-vpool is now 97.34% full.
  5. Remediate VDO volumes that are almost out of available physical space. When it is possible to add a physical space to a VDO volume, but the volume space is full before it can be grown, it may be necessary to temporarily stop I/O to the volume.

    To temporarily stop I/O to the volume, execute the following steps, where VDO volume myvdo contains a file system mounted on the /users/homeDir path:

    1. Freeze the filesystem:

      # xfs_freeze -f /users/homeDir
      
      # vgextend myvg /dev/vdc2
      
      # lvextend -l new_size myvg/vpool0-name
      
      # xfs_freeze -u /users/homeDir
    2. Unmount the filesystem:

      # umount /users/homeDir
      
      # vgextend myvg /dev/vdc2
      
      # lvextend -l new_size myvg/vpool0-name
      
      # mount -o discard /dev/myvg/myvdo /users/homeDir
      Note

      Unmounting or freezing a filesystem with cached data will incur a write of the cached data, which may fill the physical space of the VDO volume. Consider the maximum amount of cached filesystem data when setting a monitoring threshold for free physical space on a VDO volume.

  6. Blocks that are no longer used by a file system can be cleaned up by using the fstrim utility. Executing fstrim against a mounted file system on top of a VDO volume may result in increased free physical space for that volume. The fstrim utility will send discards to the VDO volume, which are then used to remove references to the previously used blocks. If any of those blocks are single-referenced, the physical space will be available to use.

    1. Check VDO stats to see what the current amount of free space is:

      # vdostats --human-readable myvg-vpool0-vpool
      
       Device             Size  Used  Available Use%  Space saving%
      myvg-vpool0-vpool  100.0G 95.0G 5.0G      95%   73%
    2. Discard unused blocks:

      # fstrim /users/homeDir
    3. View the free physical space of the VDO volume:

      # vdostats --human-readable myvg-vpool0-vpool
      
      
       Device             Size    Used   Available Use%  Space saving%
      myvg-vpool0-vpool  100.0G   30.0G  70.0G     30%    43%

      In this example, after executing fstrim on the file system, the discards were able to return 65G of physical space to use in the VDO volume.

      Note

      Discarding volumes with lower levels of deduplication and compression will have a possibility of reclaiming physical space than discarding volumes with higher levels of deduplication and compression. A volume that has high levels of deduplication and compression may potentially require a more extensive cleanup to reclaim physical space than just simply discarding already unused blocks.