Chapter 3. Testing VDO space savings

You can perform a series of tests to determine how much storage space you can save by using VDO.

Prerequisites

  • One or more physical block devices are available.
  • The target block device is larger than 512 GiB.
  • VDO is installed.

3.1. The purpose and outcomes of testing VDO

VDO tests provided by Red Hat help produce an assessment of the integration of VDO into existing storage devices. They are intended to augment, not replace, your internal evaluation efforts.

The test results help Red Hat engineers to assist you in understanding VDO behavior in specific storage environments. Original equipment manufacturers (OEMs) can learn how to design their deduplication and compression capable devices, and how their customers can tune their applications for those devices.

Goals

  • Identify configuration settings that elicit optimal responses from the test device.
  • Explain basic tuning parameters to help avoid product misconfigurations.
  • Create a reference of performance results to compare with real use cases.
  • Identify how different workloads affect performance and data efficiency.
  • Shorten the time to market with VDO implementations.

The test plan and test conditions

The VDO tests provide conditions under which VDO can be most realistically evaluated. Altering test procedures or parameters might invalidate results. Red Hat sales engineers can guide you when modifying test plans.

For an effective test plan, you must study the VDO architecture and explore these items:

  • The performance in high-load environments
  • The configurable properties of VDO for performance tuning end-user applications
  • The impact of VDO being a native 4 KiB block device
  • The response to access patterns and distributions of deduplication and compression
  • The value of cost versus capacity versus performance for a given application

3.2. Thin provisioning in VDO

VDO is a thinly provisioned block storage target. The amount of physical space that a VDO volume uses might differ from the size of the volume that is presented to users of the storage. You can make use of this disparity to save on storage costs.

Out-of-space conditions

Take care to avoid unexpectedly running out of storage space, if the data written does not achieve the expected rate of optimization.

Whenever the number of logical blocks (virtual storage) exceeds the number of physical blocks (actual storage), it becomes possible for file systems and applications to unexpectedly run out of space. For that reason, storage systems using VDO must provide you with a way of monitoring the size of the free pool on the VDO volume.

You can determine the size of this free pool by using the vdostats utility. The default output of this utility lists information for all running VDO volumes in a format similar to the Linux df utility. For example:

Device                1K-blocks   Used        Available   Use%
/dev/mapper/vdo-name  211812352   105906176   105906176   50%

When the physical storage capacity of a VDO volume is almost full, VDO reports a warning in the system log, similar to the following:

Oct  2 17:13:39 system lvm[13863]: Monitoring VDO pool vdo-name.
Oct  2 17:27:39 system lvm[13863]: WARNING: VDO pool vdo-name is now 80.69% full.
Oct  2 17:28:19 system lvm[13863]: WARNING: VDO pool vdo-name is now 85.25% full.
Oct  2 17:29:39 system lvm[13863]: WARNING: VDO pool vdo-name is now 90.64% full.
Oct  2 17:30:29 system lvm[13863]: WARNING: VDO pool vdo-name is now 96.07% full.
Note

These warning messages appear only when the lvm2-monitor service is running. It is enabled by default.

How to prevent out-of-space conditions

If the size of free pool drops below a certain level, you can take action by:

  • Deleting data. This reclaims space whenever the deleted data is not duplicated. Deleting data frees the space only after discards are issued.
  • Adding physical storage
Important

Monitor physical space on your VDO volumes to prevent out-of-space situations. Running out of physical blocks might result in losing recently written, unacknowledged data on the VDO volume.

Thin provisioning and the TRIM and DISCARD commands

To benefit from the storage savings of thin provisioning, the physical storage layer needs to know when data is deleted. File systems that work with thinly provisioned storage send TRIM or DISCARD commands to inform the storage system when a logical block is no longer required.

Several methods of sending the TRIM or DISCARD commands are available:

  • With the discard mount option, the file systems can send these commands whenever a block is deleted.
  • You can send the commands in a controlled manner by using utilities such as fstrim. These utilities tell the file system to detect which logical blocks are unused and send the information to the storage system in the form of a TRIM or DISCARD command.

The need to use TRIM or DISCARD on unused blocks is not unique to VDO. Any thinly provisioned storage system has the same challenge.

3.3. Information to record before each VDO test

You must record the following information at the start of each test to ensure that the test environment is fully understood. You can capture much of the required information by using the sosreport utility.

Required information

  • The used Linux build, including the kernel build number
  • The complete list of installed packages, as obtained from the rpm -qa command
  • Complete system specifications

    • CPU type and quantity; available in the /proc/cpuinfo file
    • Installed memory and the amount available after the rase OS is running; available in the /proc/meminfo file
    • Types of used drive controllers
    • Types and quantity of used disks
  • A complete list of running processes; available from the ps aux command or a similar listing
  • Name of the physical volume and the volume group created for use with VDO; available from the pvs and vgs commands
  • File system used when formatting the VDO volume, if any
  • Permissions on the mounted directory
  • Content of the /etc/vdoconf.yaml file
  • Location of the VDO files

3.4. Creating a VDO test volume

This procedure creates a VDO volume with a logical size of 1 TiB on a 512 GiB physical volume for testing purposes.

Procedure

  1. Create a VDO volume:

    # vdo create --name=vdo-test \
                 --device=/dev/sdb \
                 --vdoLogicalSize=1T \
                 --writePolicy=policy \
                 --verbose
    • Replace /dev/sdb with the path to a block device.
    • To test the VDO async mode on top of asynchronous storage, create an asynchronous volume using the --writePolicy=async option.
    • To test the VDO sync mode on top of synchronous storage, create a synchronous volume using the --writePolicy=sync option.
  2. Format the new volume with an XFS or ext4 file system.

    • For XFS:

      # mkfs.xfs -K /dev/mapper/vdo-test
    • For ext4:

      # mkfs.ext4 -E nodiscard /dev/mapper/vdo-test
  3. Mount the formatted volume:

    # mkdir /mnt/vdo-test
    
    # mount /dev/mapper/vdo-test /mnt/vdo-test && \
      chmod a+rwx /mnt/vdo-test

3.5. Testing the VDO test volume

This procedure tests whether reading and writing to the VDO test volume works.

Prerequisites

Procedure

  1. Write 32 GiB of random data to the VDO volume:

    $ dd if=/dev/urandom of=/mnt/vdo-test/testfile bs=4096 count=8388608
  2. Read the data from the VDO volume and write it to another volume:

    $ dd if=/mnt/vdo-test/testfile of=another-location/testfile bs=4096
    • Replace another-location with any directory where you have write access that is not on the VDO test volume. For example, you can use your home directory.
  3. Compare the two files:

    $ diff --report-identical-files /mnt/vdo-test/testfile another-location/testfile

    The command should report that the files are the same.

  4. Copy the file back to a new location on the VDO volume:

    $ dd if=another-location/testfile of=/mnt/vdo-test/testfile2 bs=4096
  5. Compare the third file to the second file:

    $ diff --report-identical-files /mnt/vdo-test/testfile2 another-location/testfile

    The command should report that the files are the same.

Cleanup steps

3.6. Cleaning up the VDO test volume

This procedure removes the VDO volume used for testing VDO efficiency from the system.

Prerequisites

  • A VDO test volume is mounted.

Procedure

  1. Unmount the file system created on the VDO volume:

    # umount /mnt/vdo-test
  2. Remove the VDO test volume from the system:

    # vdo remove --name=vdo-test

Verification steps

  • Verify that the volume has been removed:

    # vdo list --all | grep vdo-test

    The command should not list the VDO test partition.

3.7. Measuring VDO deduplication

This procedure tests the efficiency of VDO data deduplication on a VDO test volume.

Prerequisites

Procedure

  1. Prepare a table where you can record the test results:

    StatisticBare file systemAfter seedAfter 10 copies

    File system used size

       

    VDO data used

       

    VDO logical used

       
  2. Create 10 directories on the VDO volume to hold 10 copies of the test data set:

    $ mkdir /mnt/vdo-test/vdo{01..10}
  3. Examine the disk usage reported by the file system:

    $ df --human-readable /mnt/vdo-test

    Example 3.1. Disk usage

    Filesystem            Size  Used Avail Use% Mounted on
    /dev/mapper/vdo-test  1.5T  198M  1.4T   1% /mnt/vdo-test
  4. Record the following values:

    # vdostats --verbose | grep "blocks used"

    Example 3.2. Used blocks

    data blocks used                : 1090
    overhead blocks used            : 538846
    logical blocks used             : 6059434
    • The data blocks used value is the number of blocks used by user data after optimization on the physical device running under VDO.
    • The logical blocks used value is the number of blocks used before optimization. It will be used as the starting point for measurements.
  5. Create a data source file on the VDO volume:

    $ dd if=/dev/urandom of=/mnt/vdo-test/sourcefile bs=4096 count=1048576
    
    4294967296 bytes (4.3 GB) copied, 540.538 s, 7.9 MB/s
  6. Re-examine the amount of used physical disk space:

    $ df --human-readable /mnt/vdo-test

    Example 3.3. Disk usage with the data source file

    Filesystem            Size  Used Avail Use% Mounted on
    /dev/mapper/vdo-test  1.5T  4.2G  1.4T   1% /mnt/vdo-test
    # vdostats --verbose | grep "blocks used"

    Example 3.4. Used blocks with the data source file

    data blocks used                : 1050093  # Increased by 4GiB
    overhead blocks used            : 538846   # Did not significantly change
    logical blocks used             : 7108036  # Increased by 4GiB

    This command should show an increase in the number of blocks used, corresponding to the size of the written file.

  7. Copy the file to each of the 10 subdirectories:

    $ for i in {01..10}; do
      cp /mnt/vdo-test/sourcefile /mnt/vdo-test/vdo$i
      done
  8. Re-examine the amount of used physical disk space:

    $ df -h /mnt/vdo-test

    Example 3.5. Disk usage after copying the file

    Filesystem            Size  Used Avail Use% Mounted on
    /dev/mapper/vdo-test  1.5T   45G  1.3T   4% /mnt/vdo-test
    # vdostats --verbose | grep "blocks used"

    Example 3.6. Used blocks after copying the file

    data blocks used                : 1050836   # Increased by 3 MiB
    overhead blocks used            : 538846
    logical blocks used             : 17594127  # Increased by 41 GiB

    The data blocks used value should be similar to the result of the earlier listing, with only a slight increase due to file system journaling and metadata.

  9. Subtract this new value of the space used by the file system from the value found before writing the test data. This is the amount of space consumed by this test from the perspective of the file system.
  10. Observe the space savings in your recorded statistics:

    Example 3.7. Recorded values

    StatisticBare file systemAfter seedAfter 10 copies

    File system used size

    198 MiB

    4.2 GiB

    45 GiB

    VDO data used

    4 MiB

    4.1 GiB

    4.1 GiB

    VDO logical used

    23.6 GiB (file system overhead for 1.6 TiB formatted drive)

    27.8 GiB

    68.7 GiB

    Note

    In the table, values have been converted to MiB or GiB. Blocks in the vdostats output are 4,096 B in size.

Cleanup steps

3.8. Measuring VDO compression

This procedure tests the efficiency of VDO data compression on a VDO test volume.

Prerequisites

Procedure

  1. Disable deduplication and enable compression on the VDO test volume:

    # vdo disableDeduplication --name=vdo-test
    # vdo enableCompression --name=vdo-test
  2. Synchronize the VDO volume to complete any unfinished compression:

    # sync && dmsetup message vdo-test 0 sync-dedupe
  3. Inspect VDO statistics before the transfer:

    # vdostats --verbose | grep "blocks used"

    Make note of the data blocks used and logical blocks used values.

  4. VDO optimizes file system overhead as well as actual user data. Calculate the number of 4 KiB blocks saved by compression for the empty file system as logical blocks used minus data blocks used.
  5. Copy the content of the /lib directory to the VDO volume:

    # cp --verbose --recursive /lib /mnt/vdo-test
    
    ...
    sent 152508960 bytes  received 60448 bytes  61027763.20 bytes/sec
    total size is 152293104  speedup is 1.00

    Record the total size of the copied data.

  6. Synchronize Linux caches and the VDO volume:

    # sync && dmsetup message vdo-test 0 sync-dedupe
  7. Inspect VDO statistics again:

    # vdostats --verbose | grep "blocks used"

    Observe the logical blocks used and data blocks used values.

  8. Calculate the amount of bytes saved by compression using the following formula:

    saved_bytes = (logical_blocks_used - data_blocks_used) * 4096

Cleanup steps

3.9. Measuring total VDO space savings

This procedure tests the combined efficiency of VDO data deduplication and compression on a VDO test volume.

Procedure

  1. Create and mount a VDO volume as described in Section 3.4, “Creating a VDO test volume”.
  2. Perform the tests described in ] and xref:measuring-vdo-compression_testing-vdo-space-savings[ on the same volume without removing it. Observe changes to space savings in the vdostats output.
  3. Experiment with your own datasets.

3.10. Testing the effect of TRIM and DISCARD on VDO

This procedure tests whether the TRIM and DISCARD commands properly free up blocks from deleted files on a VDO test volume. It demonstrates that discards inform VDO that the space is no longer used.

Prerequisites

Procedure

  1. Prepare a table where you can record the test results:

    StepFile space used (MB)Data blocks usedLogical blocks used

    Initial

       

    Add 1 GiB file

       

    Run fstrim

       

    Delete 1 GiB file

       

    Run fstrim

       
  2. Trim the file system to remove unneeded blocks:

    # fstrim /mnt/vdo-test

    The command might take a long time.

  3. Record the initial space usage in the file system:

    $ df -m /mnt/vdo-test
  4. See how many physical and logical data blocks the VDO volume uses:

    # vdostats --verbose | grep "blocks used"
  5. Create a 1 GiB file with non-duplicate data on the VDO volume:

    $ dd if=/dev/urandom of=/mnt/vdo-test/file bs=1M count=1K
  6. Record the space usage again:

    $ df -m /mnt/vdo-test
    
    # vdostats --verbose | grep "blocks used"

    The file system should use an additional 1 GiB. The data blocks used and logical blocks used values should increase similarly.

  7. Trim the file system again:

    # fstrim /mnt/vdo-test
  8. Inspect the space usage again to confirm that the trim had no impact on the physical volume usage:

    $ df -m /mnt/vdo-test
    
    # vdostats --verbose | grep "blocks used"
  9. Delete the 1 GiB file:

    $ rm /mnt/vdo-test/file
  10. Check and record the space usage again:

    $ df -m /mnt/vdo-test
    
    # vdostats --verbose | grep "blocks used"

    The file system is aware that a file has been deleted, but there is no change to the number of physical or logical blocks because the file deletion has not been communicated to the underlying storage.

  11. Trim the file system again:

    # fstrim /mnt/vdo-test
  12. Check and record the space usage again:

    $ df -m /mnt/vdo-test
    
    # vdostats --verbose | grep "blocks used"

    The fstrim utility looks for free blocks in the file system and sends a TRIM command to the VDO volume for unused addresses, which releases the associated logical blocks. VDO processes the TRIM command to release the underlying physical blocks.

Additional resources