Chapter 14. Controlling LVM allocation
By default, a volume group allocates physical extents according to common-sense rules such as not placing parallel stripes on the same physical volume. This is the
normal allocation policy. You can use the
--alloc argument of the
vgcreate command to specify an allocation policy of
cling. In general, allocation policies other than
normal are required only in special cases where you need to specify unusual or nonstandard extent allocation.
14.1. LVM allocation policies
When an LVM operation needs to allocate physical extents for one or more logical volumes, the allocation proceeds as follows:
- The complete set of unallocated physical extents in the volume group is generated for consideration. If you supply any ranges of physical extents at the end of the command line, only unallocated physical extents within those ranges on the specified physical volumes are considered.
Each allocation policy is tried in turn, starting with the strictest policy (
contiguous) and ending with the allocation policy specified using the
--allocoption or set as the default for the particular logical volume or volume group. For each policy, working from the lowest-numbered logical extent of the empty logical volume space that needs to be filled, as much space as possible is allocated, according to the restrictions imposed by the allocation policy. If more space is needed, LVM moves on to the next policy.
The allocation policy restrictions are as follows:
An allocation policy of
contiguousrequires that the physical location of any logical extent that is not the first logical extent of a logical volume is adjacent to the physical location of the logical extent immediately preceding it.
When a logical volume is striped or mirrored, the
contiguousallocation restriction is applied independently to each stripe or mirror image (leg) that needs space.
An allocation policy of
clingrequires that the physical volume used for any logical extent be added to an existing logical volume that is already in use by at least one logical extent earlier in that logical volume. If the configuration parameter
allocation/cling_tag_listis defined, then two physical volumes are considered to match if any of the listed tags is present on both physical volumes. This allows groups of physical volumes with similar properties (such as their physical location) to be tagged and treated as equivalent for allocation purposes.
When a Logical Volume is striped or mirrored, the
clingallocation restriction is applied independently to each stripe or mirror image (leg) that needs space.
An allocation policy of
normalwill not choose a physical extent that shares the same physical volume as a logical extent already allocated to a parallel logical volume (that is, a different stripe or mirror image/leg) at the same offset within that parallel logical volume.
When allocating a mirror log at the same time as logical volumes to hold the mirror data, an allocation policy of
normalwill first try to select different physical volumes for the log and the data. If that is not possible and the
allocation/mirror_logs_require_separate_pvsconfiguration parameter is set to 0, it will then allow the log to share physical volume(s) with part of the data.
Similarly, when allocating thin pool metadata, an allocation policy of
normalwill follow the same considerations as for allocation of a mirror log, based on the value of the
If there are sufficient free extents to satisfy an allocation request but a
normalallocation policy would not use them, the
anywhereallocation policy will, even if that reduces performance by placing two stripes on the same physical volume.
The allocation policies can be changed using the
Be aware that future updates can bring code changes in layout behaviour according to the defined allocation policies. For example, if you supply on the command line two empty physical volumes that have an identical number of free physical extents available for allocation, LVM currently considers using each of them in the order they are listed; there is no guarantee that future releases will maintain that property. If it is important to obtain a specific layout for a particular Logical Volume, then you should build it up through a sequence of
lvconvert steps such that the allocation policies applied to each step leave LVM no discretion over the layout.
To view the way the allocation process currently works in any specific case, you can read the debug logging output, for example by adding the
-vvvv option to a command.
14.2. Preventing allocation on a physical volume
You can prevent allocation of physical extents on the free space of one or more physical volumes with the
pvchange command. This may be necessary if there are disk errors, or if you will be removing the physical volume.
The following command disallows the allocation of physical extents on
pvchange -x n /dev/sdk1
You can also use the
-xy arguments of the
pvchange command to allow allocation where it had previously been disallowed.
14.3. Extending a logical volume with the
cling allocation policy
When extending an LVM volume, you can use the
--alloc cling option of the
lvextend command to specify the
cling allocation policy. This policy will choose space on the same physical volumes as the last segment of the existing logical volume. If there is insufficient space on the physical volumes and a list of tags is defined in the
/etc/lvm/lvm.conf file, LVM will check whether any of the tags are attached to the physical volumes and seek to match those physical volume tags between existing extents and new extents.
For example, if you have logical volumes that are mirrored between two sites within a single volume group, you can tag the physical volumes according to where they are situated by tagging the physical volumes with
@site2 tags. You can then specify the following line in the
cling_tag_list = [ "@site1", "@site2" ]
In the following example, the
lvm.conf file has been modified to contain the following line:
cling_tag_list = [ "@A", "@B" ]
Also in this example, a volume group
taft has been created that consists of the physical volumes
/dev/sdh1. These physical volumes have been tagged with tags
C. The example does not use the
C tag, but this will show that LVM uses the tags to select which physical volumes to use for the mirror legs.
# pvs -a -o +pv_tags /dev/sd[bcdefgh] PV VG Fmt Attr PSize PFree PV Tags /dev/sdb1 taft lvm2 a-- 15.00g 15.00g A /dev/sdc1 taft lvm2 a-- 15.00g 15.00g B /dev/sdd1 taft lvm2 a-- 15.00g 15.00g B /dev/sde1 taft lvm2 a-- 15.00g 15.00g C /dev/sdf1 taft lvm2 a-- 15.00g 15.00g C /dev/sdg1 taft lvm2 a-- 15.00g 15.00g A /dev/sdh1 taft lvm2 a-- 15.00g 15.00g A
The following command creates a 10 gigabyte mirrored volume from the volume group
lvcreate --type raid1 -m 1 -n mirror --nosync -L 10G taftWARNING: New raid1 won't be synchronised. Don't read what you didn't write! Logical volume "mirror" created
The following command shows which devices are used for the mirror legs and RAID metadata subvolumes.
lvs -a -o +devicesLV VG Attr LSize Log Cpy%Sync Devices mirror taft Rwi-a-r--- 10.00g 100.00 mirror_rimage_0(0),mirror_rimage_1(0) [mirror_rimage_0] taft iwi-aor--- 10.00g /dev/sdb1(1) [mirror_rimage_1] taft iwi-aor--- 10.00g /dev/sdc1(1) [mirror_rmeta_0] taft ewi-aor--- 4.00m /dev/sdb1(0) [mirror_rmeta_1] taft ewi-aor--- 4.00m /dev/sdc1(0)
The following command extends the size of the mirrored volume, using the
cling allocation policy to indicate that the mirror legs should be extended using physical volumes with the same tag.
lvextend --alloc cling -L +10G taft/mirrorExtending 2 mirror images. Extending logical volume mirror to 20.00 GiB Logical volume mirror successfully resized
The following display command shows that the mirror legs have been extended using physical volumes with the same tag as the leg. Note that the physical volumes with a tag of
C were ignored.
lvs -a -o +devicesLV VG Attr LSize Log Cpy%Sync Devices mirror taft Rwi-a-r--- 20.00g 100.00 mirror_rimage_0(0),mirror_rimage_1(0) [mirror_rimage_0] taft iwi-aor--- 20.00g /dev/sdb1(1) [mirror_rimage_0] taft iwi-aor--- 20.00g /dev/sdg1(0) [mirror_rimage_1] taft iwi-aor--- 20.00g /dev/sdc1(1) [mirror_rimage_1] taft iwi-aor--- 20.00g /dev/sdd1(0) [mirror_rmeta_0] taft ewi-aor--- 4.00m /dev/sdb1(0) [mirror_rmeta_1] taft ewi-aor--- 4.00m /dev/sdc1(0)
14.4. Differentiating between LVM RAID objects using tags
You can assign tags to LVM RAID objects to group them, so that you can automate the control of LVM RAID behavior, such as activation, by group.
The physical volume (PV) tags are responsible for the allocation control in the LVM raid, as opposed to logical volume (LV) or volume group (VG) tags, because allocation in lvm occurs at the PV level based on allocation policies. To distinguish storage types, such as NVMe, SSD, or HDD, by their different properties, tag them appropriately. Red Hat recommends that you tag each new PV appropriately after you add it to a VG.
This procedure adds object tags to your logical volumes, assuming
/dev/sda is an SSD, and
/dev/sd[b-f] are HDDs with one partition.
lvm2package is installed.
- Storage devices to use as PVs are available.
Create a volume group.
# vgcreate MyVG /dev/sd[a-f]1
Add tags to your physical volumes.
# pvchange --addtag ssds /dev/sda1 # pvchange --addtag hdds /dev/sd[b-f]1
Create a RAID6 logical volume.
# lvcreate --type raid6 --stripes 3 -L1G -nr6 MyVG @hdds
Create a linear cache pool volume.
# lvcreate -nr6pool -L512m MyVG @ssds
Convert the RAID6 volume to be cached.
# lvconvert --type cache --cachevol MyVG/r6pool MyVG/r6