Chapter 12. Creating and managing thinly-provisioned logical volumes (thin volumes)

Logical volumes can be thinly provisioned. This allows you to create logical volumes that are larger than the available extents.

12.1. Thinly-provisioned logical volumes (thin volumes)

Logical volumes can be thinly provisioned. This allows you to create logical volumes that are larger than the available extents. Using thin provisioning, you can manage a storage pool of free space, known as a thin pool, which can be allocated to an arbitrary number of devices when needed by applications. You can then create devices that can be bound to the thin pool for later allocation when an application actually writes to the logical volume. The thin pool can be expanded dynamically when needed for cost-effective allocation of storage space.

Note

Thin volumes are not supported across the nodes in a cluster. The thin pool and all its thin volumes must be exclusively activated on only one cluster node.

By using thin provisioning, a storage administrator can overcommit the physical storage, often avoiding the need to purchase additional storage. For example, if ten users each request a 100GB file system for their application, the storage administrator can create what appears to be a 100GB file system for each user but which is backed by less actual storage that is used only when needed. When using thin provisioning, it is important that the storage administrator monitor the storage pool and add more capacity if it starts to become full.

To make sure that all available space can be used, LVM supports data discard. This allows for re-use of the space that was formerly used by a discarded file or other block range.

Thin volumes provide support for a new implementation of copy-on-write (COW) snapshot logical volumes, which allow many virtual devices to share the same data in the thin pool.

12.2. Creating thinly-provisioned logical volumes

This procedure provides an overview of the basic commands you use to create and grow thinly-provisioned logical volumes. For detailed information on LVM thin provisioning as well as information on using the LVM commands and utilities with thinly-provisioned logical volumes, see the lvmthin(7) man page.

To create a thin volume, perform the following tasks:

  1. Create a volume group with the vgcreate command.
  2. Create a thin pool with the lvcreate command.
  3. Create a thin volume in the thin pool with the lvcreate command.

You can use the -T (or --thin) option of the lvcreate command to create either a thin pool or a thin volume. You can also use -T option of the lvcreate command to create both a thin pool and a thin volume in that pool at the same time with a single command.

The following command uses the -T option of the lvcreate command to create a thin pool named mythinpool in the volume group vg001 and that is 100M in size. Note that since you are creating a pool of physical space, you must specify the size of the pool. The -T option of the lvcreate command does not take an argument; it deduces what type of device is to be created from the other options the command specifies.

# lvcreate -L 100M -T vg001/mythinpool
  Thin pool volume with chunk size 64.00 KiB can address at most 15.81 TiB of data.
  Logical volume "mythinpool" created.
# lvs
  LV         VG    Attr       LSize   Pool Origin Data%  Meta%  Move Log Cpy%Sync Convert
  mythinpool vg001 twi-a-tz-- 100.00m             0.00   10.84

The following command uses the -T option of the lvcreate command to create a thin volume named thinvolume in the thin pool vg001/mythinpool. Note that in this case you are specifying virtual size, and that you are specifying a virtual size for the volume that is greater than the pool that contains it.

# lvcreate -V 1G -T vg001/mythinpool -n thinvolume
  WARNING: Sum of all thin volume sizes (1.00 GiB) exceeds the size of thin pool vg001/mythinpool (100.00 MiB).
  WARNING: You have not turned on protection against thin pools running out of space.
  WARNING: Set activation/thin_pool_autoextend_threshold below 100 to trigger automatic extension of thin pools before they get full.
  Logical volume "thinvolume" created.
# lvs
  LV          VG       Attr     LSize   Pool       Origin Data%  Move Log Copy%  Convert
  mythinpool  vg001    twi-a-tz 100.00m                     0.00
  thinvolume  vg001    Vwi-a-tz   1.00g mythinpool          0.00

The following command uses the -T option of the lvcreate command to create a thin pool and a thin volume in that pool by specifying both a size and a virtual size argument for the lvcreate command. This command creates a thin pool named mythinpool in the volume group vg001 and it also creates a thin volume named thinvolume in that pool.

# lvcreate -L 100M -T vg001/mythinpool -V 1G -n thinvolume
  Thin pool volume with chunk size 64.00 KiB can address at most 15.81 TiB of data.
  WARNING: Sum of all thin volume sizes (1.00 GiB) exceeds the size of thin pool vg001/mythinpool (100.00 MiB).
  WARNING: You have not turned on protection against thin pools running out of space.
  WARNING: Set activation/thin_pool_autoextend_threshold below 100 to trigger automatic extension of thin pools before they get full.
  Logical volume "thinvolume" created.
# lvs
  LV         VG    Attr       LSize   Pool       Origin Data%  Meta%  Move Log Cpy%Sync Convert
  mythinpool vg001 twi-aotz-- 100.00m                   0.00   10.94
  thinvolume vg001 Vwi-a-tz--   1.00g mythinpool        0.00

You can also create a thin pool by specifying the --thinpool parameter of the lvcreate command. Unlike the -T option, the --thinpool parameter requires an argument, which is the name of the thin pool logical volume that you are creating. The following example specifies the --thinpool parameter of the lvcreate command to create a thin pool named mythinpool in the volume group vg001 and that is 100M in size:

# lvcreate -L 100M --thinpool mythinpool vg001
  Thin pool volume with chunk size 64.00 KiB can address at most 15.81 TiB of data.
  Logical volume "mythinpool" created.
# lvs
  LV         VG    Attr       LSize   Pool Origin Data%  Meta%  Move Log Cpy%Sync Convert
  mythinpool vg001 twi-a-tz-- 100.00m             0.00   10.84

Use the following criteria for using the chunk size:

  • A smaller chunk size requires more metadata and hinders performance, but provides better space utilization with snapshots.
  • A bigger chunk size requires less metadata manipulation, but makes the snapshot less space efficient.

By default, lvm2 starts with a 64KiB chunk size and increases its value when the resulting size of the thin pool metadata device grows above 128MiB, this keeps the metadata size compact. However, this may result in some big chunk size values, which are less space efficient for snapshot usage. In such cases, a smaller chunk size and a bigger metadata size is a better option.

If the volume data size is in the range of TiB, use ~15.8GiB as the metadata size, which is the maximum supported size, and set the chunk size as per your requirement. But, note that it is not possible to increase the metadata size if you need to extend the volume’s data size and have a small chunk size.

Warning

Red Hat does not recommend setting a chunk size smaller than the default value. If the chunk size is too small and your volume runs out of space for metadata, the volume is unable to create data. Monitor your logical volumes to ensure that they are expanded, or create more storage before the metadata volumes become completely full. Ensure that you set up your thin pool with a large enough chunk size so that they do not run out of room for the metadata.

Striping is supported for pool creation. The following command creates a 100M thin pool named pool in volume group vg001 with two 64 kB stripes and a chunk size of 256 kB. It also creates a 1T thin volume, vg00/thin_lv.

# lvcreate -i 2 -I 64 -c 256 -L 100M -T vg00/pool -V 1T --name thin_lv

You can extend the size of a thin volume with the lvextend command. You cannot, however, reduce the size of a thin pool.

The following command resizes an existing thin pool that is 100M in size by extending it another 100M.

# lvextend -L+100M vg001/mythinpool
  Extending logical volume mythinpool to 200.00 MiB
  Logical volume mythinpool successfully resized
# lvs
  LV           VG       Attr     LSize   Pool     Origin Data%  Move Log Copy%  Convert
  mythinpool   vg001    twi-a-tz 200.00m                   0.00
  thinvolume   vg001    Vwi-a-tz   1.00g mythinpool          0.00

As with other types of logical volumes, you can rename the volume with the lvrename, you can remove the volume with the lvremove, and you can display information about the volume with the lvs and lvdisplay commands.

By default, the lvcreate command sets the size of the thin pool’s metadata logical volume according to the formula (Pool_LV_size / Pool_LV_chunk_size * 64). If you will have large numbers of snapshots or if you have have small chunk sizes for your thin pool and thus expect significant growth of the size of the thin pool at a later time, you may need to increase the default value of the thin pool’s metadata volume with the --poolmetadatasize parameter of the lvcreate command. The supported value for the thin pool’s metadata logical volume is in the range between 2MiB and 16GiB.

You can use the --thinpool parameter of the lvconvert command to convert an existing logical volume to a thin pool volume. When you convert an existing logical volume to a thin pool volume, you must use the --poolmetadata parameter in conjunction with the --thinpool parameter of the lvconvert to convert an existing logical volume to the thin pool volume’s metadata volume.

Note

Converting a logical volume to a thin pool volume or a thin pool metadata volume destroys the content of the logical volume, since in this case the lvconvert does not preserve the content of the devices but instead overwrites the content.

The following example converts the existing logical volume lv1 in volume group vg001 to a thin pool volume and converts the existing logical volume lv2 in volume group vg001 to the metadata volume for that thin pool volume.

# lvconvert --thinpool vg001/lv1 --poolmetadata vg001/lv2
  Converted vg001/lv1 to thin pool.

12.3. Thinly-provisioned snapshot volumes

Red Hat Enterprise Linux provides support for thinly-provisioned snapshot volumes. Thin snapshot volumes allow many virtual devices to be stored on the same data volume. This simplifies administration and allows for the sharing of data between snapshot volumes.

As for all LVM snapshot volumes, as well as all thin volumes, thin snapshot volumes are not supported across the nodes in a cluster. The snapshot volume must be exclusively activated on only one cluster node.

Thin snapshot volumes provide the following benefits:

  • A thin snapshot volume can reduce disk usage when there are multiple snapshots of the same origin volume.
  • If there are multiple snapshots of the same origin, then a write to the origin will cause one COW operation to preserve the data. Increasing the number of snapshots of the origin should yield no major slowdown.
  • Thin snapshot volumes can be used as a logical volume origin for another snapshot. This allows for an arbitrary depth of recursive snapshots (snapshots of snapshots of snapshots…​).
  • A snapshot of a thin logical volume also creates a thin logical volume. This consumes no data space until a COW operation is required, or until the snapshot itself is written.
  • A thin snapshot volume does not need to be activated with its origin, so a user may have only the origin active while there are many inactive snapshot volumes of the origin.
  • When you delete the origin of a thinly-provisioned snapshot volume, each snapshot of that origin volume becomes an independent thinly-provisioned volume. This means that instead of merging a snapshot with its origin volume, you may choose to delete the origin volume and then create a new thinly-provisioned snapshot using that independent volume as the origin volume for the new snapshot.

Although there are many advantages to using thin snapshot volumes, there are some use cases for which the older LVM snapshot volume feature may be more appropriate to your needs:

  • You cannot change the chunk size of a thin pool. If the thin pool has a large chunk size (for example, 1MB) and you require a short-living snapshot for which a chunk size that large is not efficient, you may elect to use the older snapshot feature.
  • You cannot limit the size of a thin snapshot volume; the snapshot will use all of the space in the thin pool, if necessary. This may not be appropriate for your needs.

In general, you should consider the specific requirements of your site when deciding which snapshot format to use.

12.4. Creating thinly-provisioned snapshot volumes

Red Hat Enterprise Linux provides support for thinly-provisioned snapshot volumes.

Note

This section provides an overview of the basic commands you use to create and grow thinly-provisioned snapshot volumes. For detailed information on LVM thin provisioning as well as information on using the LVM commands and utilities with thinly-provisioned logical volumes, see the lvmthin(7) man page.

Important

When creating a thin snapshot volume, you do not specify the size of the volume. If you specify a size parameter, the snapshot that will be created will not be a thin snapshot volume and will not use the thin pool for storing data. For example, the command lvcreate -s vg/thinvolume -L10M will not create a thin snapshot, even though the origin volume is a thin volume.

Thin snapshots can be created for thinly-provisioned origin volumes, or for origin volumes that are not thinly-provisioned.

You can specify a name for the snapshot volume with the --name option of the lvcreate command. The following command creates a thinly-provisioned snapshot volume of the thinly-provisioned logical volume vg001/thinvolume that is named mysnapshot1.

# lvcreate -s --name mysnapshot1 vg001/thinvolume
  Logical volume "mysnapshot1" created
# lvs
  LV          VG       Attr     LSize   Pool       Origin     Data%  Move Log Copy%  Convert
  mysnapshot1 vg001    Vwi-a-tz   1.00g mythinpool thinvolume   0.00
  mythinpool  vg001    twi-a-tz 100.00m                         0.00
  thinvolume  vg001    Vwi-a-tz   1.00g mythinpool              0.00

A thin snapshot volume has the same characteristics as any other thin volume. You can independently activate the volume, extend the volume, rename the volume, remove the volume, and even snapshot the volume.

By default, a snapshot volume is skipped during normal activation commands. For information on controlling the activation of a logical volume, see Logical volume activation.

You can also create a thinly-provisioned snapshot of a non-thinly-provisioned logical volume. Since the non-thinly-provisioned logical volume is not contained within a thin pool, it is referred to as an external origin. External origin volumes can be used and shared by many thinly-provisioned snapshot volumes, even from different thin pools. The external origin must be inactive and read-only at the time the thinly-provisioned snapshot is created.

To create a thinly-provisioned snapshot of an external origin, you must specify the --thinpool option. The following command creates a thin snapshot volume of the read-only inactive volume origin_volume. The thin snapshot volume is named mythinsnap. The logical volume origin_volume then becomes the thin external origin for the thin snapshot volume mythinsnap in volume group vg001 that will use the existing thin pool vg001/pool. Because the origin volume must be in the same volume group as the snapshot volume, you do not need to specify the volume group when specifying the origin logical volume.

# lvcreate -s --thinpool vg001/pool origin_volume --name mythinsnap

You can create a second thinly-provisioned snapshot volume of the first snapshot volume, as in the following command.

# lvcreate -s vg001/mythinsnap --name my2ndthinsnap

You can display a list of all ancestors and descendants of a thin snapshot logical volume by specifying the lv_ancestors and lv_descendants reporting fields of the lvs command.

In the following example:

  • stack1 is an origin volume in volume group vg001.
  • stack2 is a snapshot of stack1
  • stack3 is a snapshot of stack2
  • stack4 is a snapshot of stack3

Additionally:

  • stack5 is also a snapshot of stack2
  • stack6 is a snapshot of stack5
$ lvs -o name,lv_ancestors,lv_descendants vg001
  LV      Ancestors              Descendants
  stack1                         stack2,stack3,stack4,stack5,stack6
  stack2  stack1                 stack3,stack4,stack5,stack6
  stack3  stack2,stack1          stack4
  stack4  stack3,stack2,stack1
  stack5  stack2,stack1          stack6
  stack6  stack5,stack2,stack1
  pool
Note

The lv_ancestors and lv_descendants fields display existing dependencies but do not track removed entries which can break a dependency chain if the entry was removed from the middle of the chain. For example, if you remove the logical volume stack3 from this sample configuration, the display is as follows.

$ lvs -o name,lv_ancestors,lv_descendants vg001
  LV      Ancestors              Descendants
  stack1                         stack2,stack5,stack6
  stack2  stack1                 stack5,stack6
  stack4
  stack5  stack2,stack1          stack6
  stack6  stack5,stack2,stack1
  pool

You can configure your system to track and display logical volumes that have been removed, and you can display the full dependency chain that includes those volumes by specifying the lv_ancestors_full and lv_descendants_full fields.

12.5. Tracking and displaying thin snapshot volumes that have been removed

You can configure your system to track thin snapshot and thin logical volumes that have been removed by enabling the record_lvs_history metadata option in the lvm.conf configuration file. This allows you to display a full thin snapshot dependency chain that includes logical volumes that have been removed from the original dependency chain and have become historical logical volumes.

You can configure your system to retain historical volumes for a defined period of time by specifying the retention time, in seconds, with the lvs_history_retention_time metadata option in the lvm.conf configuration file.

A historical logical volume retains a simplified representation of the logical volume that has been removed, including the following reporting fields for the volume:

  • lv_time_removed: the removal time of the logical volume
  • lv_time: the creation time of the logical volume
  • lv_name: the name of the logical volume
  • lv_uuid: the UUID of the logical volume
  • vg_name: the volume group that contains the logical volume.

When a volume is removed, the historical logical volume name acquires a hypen as a prefix. For example, when you remove the logical volume lvol1, the name of the historical volume is -lvol1. A historical logical volume cannot be reactivated.

Even when the record_lvs_history metadata option enabled, you can prevent the retention of historical logical volumes on an individual basis when you remove a logical volume by specifying the --nohistory option of the lvremove command.

To include historical logical volumes in volume display, you specify the -H|--history option of an LVM display command. You can display a full thin snapshot dependency chain that includes historical volumes by specifying the lv_full_ancestors and lv_full_descendants reporting fields along with the -H option.

The following series of commands provides examples of how you can display and manage historical logical volumes.

  1. Ensure that historical logical volumes are retained by setting record_lvs_history=1 in the lvm.conf file. This metadata option is not enabled by default.
  2. Enter the following command to display a thin provisioned snapshot chain.

    In this example:

    • lvol1 is an origin volume, the first volume in the chain.
    • lvol2 is a snapshot of lvol1.
    • lvol3 is a snapshot of lvol2.
    • lvol4 is a snapshot of lvol3.
    • lvol5 is also a snapshot of lvol3.

      Note that even though the example lvs display command includes the -H option, no thin snapshot volume has yet been removed and there are no historical logical volumes to display.

      # lvs -H -o name,full_ancestors,full_descendants
        LV    FAncestors        FDescendants
        lvol1                   lvol2,lvol3,lvol4,lvol5
        lvol2 lvol1             lvol3,lvol4,lvol5
        lvol3 lvol2,lvol1       lvol4,lvol5
        lvol4 lvol3,lvol2,lvol1
        lvol5 lvol3,lvol2,lvol1
        pool
  3. Remove logical volume lvol3 from the snapshot chain, then run the following lvs command again to see how historical logical volumes are displayed, along with their ancestors and descendants.

    # lvremove -f vg/lvol3
      Logical volume "lvol3" successfully removed
    # lvs -H -o name,full_ancestors,full_descendants
      LV     FAncestors         FDescendants
      lvol1                     lvol2,-lvol3,lvol4,lvol5
      lvol2  lvol1              -lvol3,lvol4,lvol5
      -lvol3 lvol2,lvol1        lvol4,lvol5
      lvol4  -lvol3,lvol2,lvol1
      lvol5  -lvol3,lvol2,lvol1
      pool
  4. You can use the lv_time_removed reporting field to display the time a historical volume was removed.

    # lvs -H -o name,full_ancestors,full_descendants,time_removed
      LV     FAncestors         FDescendants              RTime
      lvol1                     lvol2,-lvol3,lvol4,lvol5
      lvol2  lvol1              -lvol3,lvol4,lvol5
      -lvol3 lvol2,lvol1        lvol4,lvol5               2016-03-14 14:14:32 +0100
      lvol4  -lvol3,lvol2,lvol1
      lvol5  -lvol3,lvol2,lvol1
      pool
  5. You can reference historical logical volumes individually in a display command by specifying the vgname/lvname format, as in the following example. Note that the fifth bit in the lv_attr field is set to h to indicate the volume is a historical volume.

    # lvs -H vg/-lvol3
      LV     VG   Attr       LSize
      -lvol3 vg   ----h-----    0
  6. LVM does not keep historical logical volumes if the volume has no live descendant. This means that if you remove a logical volume at the end of a snapshot chain, the logical volume is not retained as a historical logical volume.

    # lvremove -f vg/lvol5
      Automatically removing historical logical volume vg/-lvol5.
      Logical volume "lvol5" successfully removed
    # lvs -H -o name,full_ancestors,full_descendants
      LV     FAncestors         FDescendants
      lvol1                     lvol2,-lvol3,lvol4
      lvol2  lvol1              -lvol3,lvol4
      -lvol3 lvol2,lvol1        lvol4
      lvol4  -lvol3,lvol2,lvol1
      pool
  7. Run the following commands to remove the volume lvol1 and lvol2 and to see how the lvs command displays the volumes once they have been removed.

    # lvremove -f vg/lvol1 vg/lvol2
      Logical volume "lvol1" successfully removed
      Logical volume "lvol2" successfully removed
    # lvs -H -o name,full_ancestors,full_descendants
      LV     FAncestors           FDescendants
      -lvol1                      -lvol2,-lvol3,lvol4
      -lvol2 -lvol1               -lvol3,lvol4
      -lvol3 -lvol2,-lvol1        lvol4
      lvol4  -lvol3,-lvol2,-lvol1
      pool
  8. To remove a historical logical volume completely, you can run the lvremove command again, specifying the name of the historical volume that now includes the hyphen, as in the following example.

    # lvremove -f vg/-lvol3
    Historical logical volume "lvol3" successfully removed
    # lvs -H -o name,full_ancestors,full_descendants
      LV     FAncestors    FDescendants
      -lvol1               -lvol2,lvol4
      -lvol2 -lvol1        lvol4
      lvol4  -lvol2,-lvol1
      pool
  9. A historical logical volumes is retained as long as there is a chain that includes live volumes in its descendants. This means that removing a historical logical volume also removes all of the logical volumes in the chain if no existing descendant is linked to them, as shown in the following example.

    # lvremove -f vg/lvol4
      Automatically removing historical logical volume vg/-lvol1.
      Automatically removing historical logical volume vg/-lvol2.
      Automatically removing historical logical volume vg/-lvol4.
      Logical volume "lvol4" successfully removed