6.5. Creating a Mirrored LVM Logical Volume in a Cluster
lvm.conffile must be set correctly to enable cluster locking, either directly or by means of the
lvmconfcommand as described in Section 4.1, “Creating LVM Volumes in a Cluster”.
- In order to create a mirrored logical volume that is shared by all of the nodes in a cluster, the locking type must be set correctly in the
lvm.conffile in every node of the cluster. By default, the locking type is set to local. To change this, execute the following command in each node of the cluster to enable clustered locking:
- To create a clustered logical volume, the cluster infrastructure must be up and running on every node in the cluster. The following example verifies that the
clvmddaemon is running on the node from which it was issued:
ps auxw | grep clvmdroot 17642 0.0 0.1 32164 1072 ? Ssl Apr06 0:00 clvmd -T20 -t 90The following command shows the local view of the cluster status:
cman_tool servicesService Name GID LID State Code ... DLM Lock Space: "clvmd" 7 3 run - [1 2 3] ...
- Ensure that the
cmirror-kmodpackages are installed. The
cmirror-kmodpackage that must be installed depends on the kernel that is running. For example, if the running kernel is
kernel-largesmp, it is necessary to have
cmirror-kmod-largesmpfor the corresponding kernel version.
- Start the
service cmirror startLoading clustered mirror log: [ OK ]
- Create the mirror. The first step is creating the physical volumes. The following commands create three physical volumes. Two of the physical volumes will be used for the legs of the mirror, and the third physical volume will contain the mirror log.
pvcreate /dev/xvdb1Physical volume "/dev/xvdb1" successfully created #
pvcreate /dev/xvdb2Physical volume "/dev/xvdb2" successfully created #
pvcreate /dev/xvdc1Physical volume "/dev/xvdc1" successfully created
- Create the volume group. This example creates a volume group
vg001that consists of the three physical volumes that were created in the previous step.
vgcreate vg001 /dev/xvdb1 /dev/xvdb2 /dev/xvdc1Clustered volume group "vg001" successfully createdNote that the output of the
vgcreatecommand indicates that the volume group is clustered. You can verify that a volume group is clustered with the
vgscommand, which will show the volume group's attributes. If a volume group is clustered, it will show a c attribute.
vgs vg001VG #PV #LV #SN Attr VSize VFree vg001 3 0 0 wz--nc 68.97G 68.97G
- Create the mirrored logical volume. This example creates the logical volume
mirrorlvfrom the volume group
vg001. This volume has one mirror leg. This example specifies which extents of the physical volume will be used for the logical volume.
lvcreate -l 1000 -m1 vg001 -n mirrorlv /dev/xvdb1:1-1000 /dev/xvdb2:1-1000 /dev/xvdc1:0Logical volume "mirrorlv" createdYou can use the
lvscommand to display the progress of the mirror creation. The following example shows that the mirror is 47% synced, then 91% synced, then 100% synced when the mirror is complete.
lvs vg001/mirrorlvLV VG Attr LSize Origin Snap% Move Log Copy% Convert mirrorlv vg001 mwi-a- 3.91G vg001_mlog 47.00 #
lvs vg001/mirrorlvLV VG Attr LSize Origin Snap% Move Log Copy% Convert mirrorlv vg001 mwi-a- 3.91G vg001_mlog 91.00 #
lvs vg001/mirrorlvLV VG Attr LSize Origin Snap% Move Log Copy% Convert mirrorlv vg001 mwi-a- 3.91G vg001_mlog 100.00The completion of the mirror is noted in the system log:
May 10 14:52:52 doc-07 : Monitoring mirror device vg001-mirrorlv for events May 10 14:55:00 doc-07 lvm: vg001-mirrorlv is now in-sync
- You can use the
-o +devicesoptions to display the configuration of the mirror, including which devices make up the mirror legs. You can see that the logical volume in this example is composed of two linear images and one log.
lvs -a -o +devicesLV VG Attr LSize Origin Snap% Move Log Copy% Convert Devices mirrorlv vg001 mwi-a- 3.91G mirrorlv_mlog 100.00 mirrorlv_mimage_0(0),mirrorlv_mimage_1(0) [mirrorlv_mimage_0] vg001 iwi-ao 3.91G /dev/xvdb1(1) [mirrorlv_mimage_1] vg001 iwi-ao 3.91G /dev/xvdb2(1) [mirrorlv_mlog] vg001 lwi-ao 4.00M /dev/xvdc1(0)For Red Hat Enterprise Linux 5.2 and later, you can use the
seg_pe_rangesoption of the
lvsto display the data layout. You can use this option to verify that your layout is properly redundant. The output of this command displays PE ranges in the same format that the
lvresizecommands take as input.
lvs -a -o +seg_pe_ranges --segmentsPE Ranges mirrorlv_mimage_0:0-999 mirrorlv_mimage_1:0-999 /dev/xvdb1:1-1000 /dev/xvdb2:1-1000 /dev/xvdc1:0-0When you create the mirrored volume, you create the
clustered_logdlm space, which will contain the dlm locks for all mirrors.
cman_tool servicesService Name GID LID State Code Fence Domain: "default" 4 2 run - [1 2 3] DLM Lock Space: "clvmd" 12 7 run - [1 2 3] DLM Lock Space: "clustered_log" 14 9 run - [1 2 3] User: "usrm::manager" 10 4 run - [1 2 3]