Menu Close
Settings Close

Language and Page Formatting Options

Chapter 7. Creating and managing thin provisioned volumes (thin volumes)

Red Hat Enterprise Linux supports thin provisioned snapshot volumes and logical volumes.

Logical volumes and snapshot volumes can be thinly provisioned:

  • Using thin-provisioned logical volumes, you can create logical volumes that are larger than the available physical storage.
  • Using thin-provisioned snapshot volumes, you can store more virtual devices on the same data volume.

7.1. Overview of thin provisioning

Many modern storage stacks now provide the ability to choose between thick provisioning and thin provisioning:

  • Thick provisioning provides the traditional behavior of block storage where blocks are allocated regardless of their actual usage.
  • Thin provisioning grants the ability to provision a larger pool of block storage that may be larger in size than the physical device storing the data, resulting in over-provisioning. Over-provisioning is possible because individual blocks are not allocated until they are actually used. If you have multiple thin-provisioned devices that share the same pool, then these devices can be over-provisioned.

By using thin provisioning, you can over-commit the physical storage, and instead can manage a pool of free space known as a thin pool. You can allocate this thin pool to an arbitrary number of devices when needed by applications. You can expand the thin pool dynamically when needed for cost-effective allocation of storage space.

For example, if ten users each request a 100GB file system for their application, then you can create what appears to be a 100GB file system for each user but which is backed by less actual storage that is used only when needed.

Note

When using thin provisioning, it is important that you monitor the storage pool and add more capacity as the available physical space runs out.

The following are a few advantages of using thin-provisioned devices:

  • You can create logical volumes that are larger than the available physical storage.
  • You can have more virtual devices to be stored on the same data volume.
  • You can create file systems that can grow logically and automatically to support the data requirements and the unused blocks are returned to the pool for use by any file system in the pool

The following are the potential drawbacks of using thin-provisioned devices:

  • Thin-provisioned volumes have an inherent risk of running out of available physical storage. If you have over-provisioned your underlying storage, it could possibly result in an outage due to the lack of available physical storage. For example, if you create 10T of thinly provisioned storage with only 1T physical storage for backing, the volumes will become unavailable or unwritable after the 1T is exhausted.
  • If volumes are not sending discards to the layers after thin-provisioned devices, then the accounting for usage will not be accurate. For example, placing a file system without the -o discard mount option and not running fstrim periodically on top of thin-provisioned devices will never unallocate previously used storage. In such cases, you end up using the full provisioned amount over time even if you are not really using it.
  • You must monitor the logical and physical usage so as to not run out of available physical space.
  • Copy on Write (CoW) operation can be slower on file systems with snapshots.
  • Data blocks can be intermixed between multiple file systems leading to random access limitations of the underlying storage even when it does not appear that way to the end user.

7.2. Creating thinly-provisioned logical volumes

Using thin-provisioned logical volumes, you can create logical volumes that are larger than the available physical storage. Creating a thinly provisioned set of volumes allows the system to allocate what you use instead of allocating the full amount of storage that is requested.

Using the -T or --thin option of the lvcreate command, you can create either a thin pool or a thin volume. You can also use the -T option of the lvcreate command to create both a thin pool and a thin volume at the same time with a single command. This procedure describes how to create and grow thinly-provisioned logical volumes.

Prerequisites

Procedure

  1. Create a thin pool:

    # lvcreate -L 100M -T vg001/mythinpool
      Thin pool volume with chunk size 64.00 KiB can address at most 15.81 TiB of data.
      Logical volume "mythinpool" created.

    Note that since you are creating a pool of physical space, you must specify the size of the pool. The -T option of the lvcreate command does not take an argument; it determines what type of device is to be created from the other options that are added with the command. You can also create thin pool using additional parameters as shown in the following examples:

    • You can also create a thin pool using the --thinpool parameter of the lvcreate command. Unlike the -T option, the --thinpool parameter requires that you specify the name of the thin pool logical volume you are creating. The following example uses the --thinpool parameter to create a thin pool named mythinpool in the volume group vg001 that is 100M in size:

      # lvcreate -L 100M --thinpool mythinpool vg001
        Thin pool volume with chunk size 64.00 KiB can address at most 15.81 TiB of data.
        Logical volume "mythinpool" created.
    • As striping is supported for pool creation, you can use the -i and -I options to create stripes.The following command creates a 100M thin pool named as thinpool in volume group vg001 with two 64 kB stripes and a chunk size of 256 kB. It also creates a 1T thin volume named vg001/thinvolume.

      Note

      Ensure that there are two physical volumes with sufficient free space in the volume group or you cannot create the thin pool.

      # lvcreate -i 2 -I 64 -c 256 -L 100M -T vg001/thinpool -V 1T --name thinvolume
  2. Create a thin volume:

    # lvcreate -V 1G -T vg001/mythinpool -n thinvolume
      WARNING: Sum of all thin volume sizes (1.00 GiB) exceeds the size of thin pool vg001/mythinpool (100.00 MiB).
      WARNING: You have not turned on protection against thin pools running out of space.
      WARNING: Set activation/thin_pool_autoextend_threshold below 100 to trigger automatic extension of thin pools before they get full.
      Logical volume "thinvolume" created.

    In this case, you are specifying virtual size for the volume that is greater than the pool that contains it. You can also create thin volumes using additional parameters as shown in the following examples:

    • To create both a thin volume and a thin pool, use the -T option of the lvcreate command and specify both the size and virtual size argument:

      # lvcreate -L 100M -T vg001/mythinpool -V 1G -n thinvolume
        Thin pool volume with chunk size 64.00 KiB can address at most 15.81 TiB of data.
        WARNING: Sum of all thin volume sizes (1.00 GiB) exceeds the size of thin pool vg001/mythinpool (100.00 MiB).
        WARNING: You have not turned on protection against thin pools running out of space.
        WARNING: Set activation/thin_pool_autoextend_threshold below 100 to trigger automatic extension of thin pools before they get full.
        Logical volume "thinvolume" created.
    • To use the remaining free space to create a thin volume and thin pool, use the 100%FREE option:

      # lvcreate -V 1G -l 100%FREE -T vg001/mythinpool -n thinvolume
      Thin pool volume with chunk size 64.00 KiB can address at most <15.88 TiB of data.
        Logical volume "thinvolume" created.
    • To convert an existing logical volume to a thin pool volume, use the --thinpool parameter of the lvconvert command. You must also use the --poolmetadata parameter in conjunction with the --thinpool parameter to convert an existing logical volume to a thin pool volume’s metadata volume.

      The following example converts the existing logical volume lv1 in volume group vg001 to a thin pool volume and converts the existing logical volume lv2 in volume group vg001 to the metadata volume for that thin pool volume:

      # lvconvert --thinpool vg001/lv1 --poolmetadata vg001/lv2
        Converted vg001/lv1 to thin pool.
      Note

      Converting a logical volume to a thin pool volume or a thin pool metadata volume destroys the content of the logical volume, as lvconvert does not preserve the content of the devices but instead overwrites the content.

    • By default, the lvcreate command sets the size of the thin pool’s metadata logical volume according to the following formula:

      Pool_LV_size / Pool_LV_chunk_size * 64

      If you have large numbers of snapshots or if you have have small chunk sizes for your thin pool and thus expect significant growth of the size of the thin pool at a later time, you may need to increase the default value of the thin pool’s metadata volume using the --poolmetadatasize parameter of the lvcreate command. The supported value for the thin pool’s metadata logical volume is in the range between 2MiB and 16GiB.

      The following example illustrates how to increase the default value of the thin pools’ metadata volume:

      # lvcreate -V 1G -l 100%FREE -T vg001/mythinpool --poolmetadatasize 16M -n thinvolume
      Thin pool volume with chunk size 64.00 KiB can address at most 15.81 TiB of data.
         Logical volume "thinvolume" created.
  3. View the created thin pool and thin volume:

    # lvs -a -o +devices
      LV                 VG    Attr       LSize   Pool       Origin Data%  Meta%  Move Log Cpy%Sync Convert Devices
      [lvol0_pmspare]    vg001 ewi-------   4.00m                                                           /dev/sda(0)
      mythinpool         vg001 twi-aotz-- 100.00m                   0.00   10.94                            mythinpool_tdata(0)
      [mythinpool_tdata] vg001 Twi-ao---- 100.00m                                                           /dev/sda(1)
      [mythinpool_tmeta] vg001 ewi-ao----   4.00m                                                           /dev/sda(26)
      thinvolume         vg001 Vwi-a-tz--   1.00g mythinpool        0.00
  4. Optional: Extend the size of a thin pool with the lvextend command. You cannot, however, reduce the size of a thin pool.

    Note

    This command fails if you use -l 100%FREE argument while creating a thin pool and thin volume.

    The following command resizes an existing thin pool that is 100M in size by extending it another 100M:

    # lvextend -L+100M vg001/mythinpool
      Size of logical volume vg001/mythinpool_tdata changed from 100.00 MiB (25 extents) to 200.00 MiB (50 extents).
      WARNING: Sum of all thin volume sizes (1.00 GiB) exceeds the size of thin pool vg001/mythinpool (200.00 MiB).
      WARNING: You have not turned on protection against thin pools running out of space.
      WARNING: Set activation/thin_pool_autoextend_threshold below 100 to trigger automatic extension of thin pools before they get full.
    
      Logical volume vg001/mythinpool successfully resized
    # lvs -a -o +devices
      LV                 VG    Attr       LSize   Pool       Origin Data%  Meta%  Move Log Cpy%Sync Convert Devices
      [lvol0_pmspare]    vg001 ewi-------   4.00m                                                           /dev/sda(0)
      mythinpool         vg001 twi-aotz-- 200.00m                   0.00   10.94                            mythinpool_tdata(0)
      [mythinpool_tdata] vg001 Twi-ao---- 200.00m                                                           /dev/sda(1)
      [mythinpool_tdata] vg001 Twi-ao---- 200.00m                                                           /dev/sda(27)
      [mythinpool_tmeta] vg001 ewi-ao----   4.00m                                                           /dev/sda(26)
      thinvolume         vg001 Vwi-a-tz--   1.00g mythinpool        0.00
  5. Optional: To rename the thin pool and thin volume, use the following command:

    # lvrename vg001/mythinpool vg001/mythinpool1
      Renamed "mythinpool" to "mythinpool1" in volume group "vg001"
    
    # lvrename vg001/thinvolume vg001/thinvolume1
      Renamed "thinvolume" to "thinvolume1" in volume group "vg001"

    View the thin pool and thin volume after renaming:

    # lvs
      LV          VG       Attr     LSize   Pool       Origin Data%  Move Log Copy%  Convert
    mythinpool1 vg001   twi-a-tz 100.00m                     0.00
    thinvolume1 vg001   Vwi-a-tz   1.00g mythinpool1         0.00
  6. Optional: To remove the thin pool, use the following command:

    # lvremove -f vg001/mythinpool1
      Logical volume "thinvolume1" successfully removed.
      Logical volume "mythinpool1" successfully removed.

Additional resources

  • lvcreate(8), lvrename(8), lvs(8), and lvconvert(8) man pages

7.3. Thinly-provisioned snapshot volumes

Red Hat Enterprise Linux supports thinly-provisioned snapshot volumes. A snapshot of a thin logical volume also creates a thin logical volume (LV). A thin snapshot volume has the same characteristics as any other thin volume. You can independently activate the volume, extend the volume, rename the volume, remove the volume, and even snapshot the volume.

Note

Similarly to all LVM snapshot volumes, and all thin volumes, thin snapshot volumes are not supported across the nodes in a cluster. The snapshot volume must be exclusively activated on only one cluster node.

Traditional snapshots must allocate new space for each snapshot created, where data is preserved as changes are made to the origin. But thin-provisioning snapshots share the same space with the origin. Snapshots of thin LVs are efficient because the data blocks common to a thin LV and any of its snapshots are shared. You can create snapshots of thin LVs or from the other thin snapshots. Blocks common to recursive snapshots are also shared in the thin pool.

Thin snapshot volumes provide the following benefits:

  • Increasing the number of snapshots of the origin has a negligible impact on performance.
  • A thin snapshot volume can reduce disk usage because only the new data is written and is not copied to each snapshot.
  • There is no need to simultaneously activate the thin snapshot volume with the origin, which is a requirement of traditional snapshots.
  • When restoring an origin from a snapshot, it is not required to merge the thin snapshot. You can remove the origin and instead use the snapshot. Traditional snapshots have a separate volume where they store changes that must be copied back, that is, merged to the origin to reset it.
  • There is a significantly higher limit on the number of allowed snapshots as compared to the traditional snapshots.

Although there are many advantages for using thin snapshot volumes, there are some use cases for which the traditional LVM snapshot volume feature might be more appropriate to your needs. You can use traditional snapshots with all types of volumes. However, to use thin-snapshots requires you to use thin-provisioning.

Note

You cannot limit the size of a thin snapshot volume; the snapshot uses all of the space in the thin pool, if necessary. In general, you should consider the specific requirements of your site when deciding which snapshot format to use.

By default, a thin snapshot volume is skipped during normal activation commands.

7.4. Creating thinly-provisioned snapshot volumes

Using thin-provisioned snapshot volumes, you can have more virtual devices stored on the same data volume.

Important

When creating a thin snapshot volume, do not specify the size of the volume. If you specify a size parameter, the snapshot that will be created will not be a thin snapshot volume and will not use the thin pool for storing data. For example, the command lvcreate -s vg/thinvolume -L10M will not create a thin snapshot, even though the origin volume is a thin volume.

Thin snapshots can be created for thinly-provisioned origin volumes, or for origin volumes that are not thinly-provisioned. The following procedure describes different ways to create a thinly-provisioned snapshot volume.

Prerequisites

Procedure

  • Create a thinly-provisioned snapshot volume. The following command creates a thinly-provisioned snapshot volume named as mysnapshot1 of the thinly-provisioned logical volume vg001/thinvolume:

    # lvcreate -s --name mysnapshot1 vg001/thinvolume
      Logical volume "mysnapshot1" created
    # lvs
      LV          VG       Attr     LSize   Pool       Origin     Data%  Move Log Copy%  Convert
      mysnapshot1 vg001    Vwi-a-tz   1.00g mythinpool thinvolume   0.00
      mythinpool  vg001    twi-a-tz 100.00m                         0.00
      thinvolume  vg001    Vwi-a-tz   1.00g mythinpool              0.00
    Note

    When using thin provisioning, it is important that the storage administrator monitor the storage pool and add more capacity if it starts to become full. For information on extending the size of a thin volume, see Creating thinly-provisioned logical volumes .

  • You can also create a thinly-provisioned snapshot of a non-thinly-provisioned logical volume. Since the non-thinly-provisioned logical volume is not contained within a thin pool, it is referred to as an external origin. External origin volumes can be used and shared by many thinly-provisioned snapshot volumes, even from different thin pools. The external origin must be inactive and read-only at the time the thinly-provisioned snapshot is created.

    The following example creates a thin snapshot volume of the read-only, inactive logical volume named origin_volume. The thin snapshot volume is named mythinsnap. The logical volume origin_volume then becomes the thin external origin for the thin snapshot volume mythinsnap in volume group vg001 that uses the existing thin pool vg001/pool. The origin volume must be in the same volume group as the snapshot volume. Do not specify the volume group when specifying the origin logical volume.

    # lvcreate -s --thinpool vg001/pool origin_volume --name mythinsnap
  • You can create a second thinly-provisioned snapshot volume of the first snapshot volume by executing the following command.

    # lvcreate -s vg001/mysnapshot1 --name mysnapshot2
    Logical volume "mysnapshot2" created.

    To create a third thinly-provisioned snapshot volume, use the following command:

    # lvcreate -s vg001/mysnapshot2 --name mysnapshot3
    Logical volume "mysnapshot3" created.

Verification

  • Display a list of all ancestors and descendants of a thin snapshot logical volume:

    $ lvs -o name,lv_ancestors,lv_descendants vg001
      LV           Ancestors                           Descendants
      mysnapshot2  mysnapshot1,thinvolume              mysnapshot3
      mysnapshot1  thinvolume              mysnapshot2,mysnapshot3
      mysnapshot3  mysnapshot2,mysnapshot1,thinvolume
      mythinpool
      thinvolume           	           	     mysnapshot1,mysnapshot2,mysnapshot3

    Here,

  • thinvolume is an origin volume in volume group vg001.
  • mysnapshot1 is a snapshot of thinvolume
  • mysnapshot2 is a snapshot of mysnapshot1
  • mysnapshot3 is a snapshot of mysnapshot2

    Note

    The lv_ancestors and lv_descendants fields display existing dependencies. However, they do not track removed entries which can break a dependency chain if the entry was removed from the middle of the chain.

Additional resources

  • lvcreate(8) man page

7.5. Historical logical volumes

You can configure the system to track thin snapshot and thin logical volumes that have been removed by enabling the record_lvs_history metadata option in the lvm.conf configuration file. This allows you to display a full thin snapshot dependency chain that includes logical volumes that have been removed from the original dependency chain and have become historical logical volumes.

You can configure the system to retain historical volumes for a defined period of time by specifying the retention time, in seconds, with the lvs_history_retention_time metadata option in the lvm.conf configuration file.

A historical logical volume retains a simplified representation of the already removed logical volume, including the following reporting fields for the volume:

  • lv_time_removed: the removal time of the logical volume
  • lv_time: the creation time of the logical volume
  • lv_name: the name of the logical volume
  • lv_uuid: the UUID of the logical volume
  • vg_name: the volume group that contains the logical volume.
Important

A historical logical volume cannot be reactivated.

When you remove a volume, the historical logical volume name acquires a hyphen as a prefix. For example, if you remove the logical volume lvol1, the name of the historical volume becomes -lvol1.

Logical Volume Manager (LVM) does not keep historical logical volumes if the volume has no live descendant. This means that if you remove a logical volume at the end of a snapshot chain, the logical volume is not retained as a historical logical volume.

To include historical logical volumes in volume display, you specify the -H|--history option of an LVM display command. You can display a full thin snapshot dependency chain that includes historical volumes by specifying the lv_full_ancestors and lv_full_descendants reporting fields along with the -H option.

7.6. Tracking and displaying removed thin snapshot volumes

This procedure describes how to display and manage deleted historical logical volumes.

Procedure

  1. Ensure that historical logical volumes are retained by setting record_lvs_history=1 in the lvm.conf file. This metadata option is not enabled by default.
  2. Optional: Set the value of the lvs_history_retention_time option in seconds. This is the time interval after which a record about individual historical logical volume is automatically destroyed. The automatic default value is 0 and it disables this feature. If you do not set this option, you can also delete individual historical volumes manually.
  3. To display a thin-provisioned snapshot chain:

    In this example:

    • lvol1 is an origin volume, the first volume in the chain.
    • lvol2 is a snapshot of lvol1.
    • lvol3 is a snapshot of lvol2.
    • lvol4 is a snapshot of lvol3.
    • lvol5 is also a snapshot of lvol3.

      # lvs -H -o name,full_ancestors,full_descendants
        LV    FAncestors        FDescendants
        lvol1                   lvol2,lvol3,lvol4,lvol5
        lvol2 lvol1             lvol3,lvol4,lvol5
        lvol3 lvol2,lvol1       lvol4,lvol5
        lvol4 lvol3,lvol2,lvol1
        lvol5 lvol3,lvol2,lvol1
        pool

      Note that, despite using lvs utility with the -H option, no thin snapshot volume is removed and there are no historical logical volumes to display.

  4. Remove logical volume lvol3 from the snapshot chain:

    # lvremove -f vg/lvol3
      Logical volume "lvol3" successfully removed
  5. Run the lvs utility to see the details of historical logical volumes, along with their ancestors and descendants:

    # lvs -H -o name,full_ancestors,full_descendants
      LV     FAncestors         FDescendants
      lvol1                     lvol2,-lvol3,lvol4,lvol5
      lvol2  lvol1              -lvol3,lvol4,lvol5
      -lvol3 lvol2,lvol1        lvol4,lvol5
      lvol4  -lvol3,lvol2,lvol1
      lvol5  -lvol3,lvol2,lvol1
      pool
  6. Additionally, display the removal timestamp of a historical volume:

    # lvs -H -o name,full_ancestors,full_descendants,time_removed
      LV     FAncestors         FDescendants              RTime
      lvol1                     lvol2,-lvol3,lvol4,lvol5
      lvol2  lvol1              -lvol3,lvol4,lvol5
      -lvol3 lvol2,lvol1        lvol4,lvol5               2016-03-14 14:14:32 +0100
      lvol4  -lvol3,lvol2,lvol1
      lvol5  -lvol3,lvol2,lvol1
      pool
  7. You can reference historical logical volumes individually in a display command by specifying the vgname/lvname format:

    # lvs -H vg/-lvol3
      LV     VG   Attr       LSize
      -lvol3 vg   ----h-----    0

    Note that the fifth bit in the lv_attr field is set to h to indicate the volume is a historical one.

  8. LVM does not keep historical logical volumes if the volume has no live descendant. This means that if you remove a logical volume at the end of a snapshot chain, the logical volume is not retained as a historical logical volume.

    # lvremove -f vg/lvol5
      Automatically removing historical logical volume vg/-lvol5.
      Logical volume "lvol5" successfully removed
    # lvs -H -o name,full_ancestors,full_descendants
      LV     FAncestors         FDescendants
      lvol1                     lvol2,-lvol3,lvol4
      lvol2  lvol1              -lvol3,lvol4
      -lvol3 lvol2,lvol1        lvol4
      lvol4  -lvol3,lvol2,lvol1
      pool
  9. Remove the volume lvol1 and lvol2 and to see how the lvs command displays the volumes once they have been removed.

    # lvremove -f vg/lvol1 vg/lvol2
      Logical volume "lvol1" successfully removed
      Logical volume "lvol2" successfully removed
    # lvs -H -o name,full_ancestors,full_descendants
      LV     FAncestors           FDescendants
      -lvol1                      -lvol2,-lvol3,lvol4
      -lvol2 -lvol1               -lvol3,lvol4
      -lvol3 -lvol2,-lvol1        lvol4
      lvol4  -lvol3,-lvol2,-lvol1
      pool
  10. Remove a historical logical volume completely by specifying the name of the historical volume that now includes the hyphen, as in the following example

    # lvremove -f vg/-lvol3
    Historical logical volume "lvol3" successfully removed
    # lvs -H -o name,full_ancestors,full_descendants
      LV     FAncestors    FDescendants
      -lvol1               -lvol2,lvol4
      -lvol2 -lvol1        lvol4
      lvol4  -lvol2,-lvol1
      pool