Chapter 5. Configuring a GFS2 File System in a Cluster
- On all nodes of the cluster, install the
gfs2-utilspackages, which are part of the Resilient Storage channel.
yum install lvm2-cluster gfs2-utils
- Set the global Pacemaker parameter
NoteBy default, the value of
no-quorum-policyis set to
stop, indicating that once quorum is lost, all the resources on the remaining partition will immediately be stopped. Typically this default is the safest and most optimal option, but unlike most resources, GFS2 requires quorum to function. When quorum is lost both the applications using the GFS2 mounts and the GFS2 mount itself cannot be correctly stopped. Any attempts to stop these resources without quorum will fail which will ultimately result in the entire cluster being fenced every time quorum is lost.To address this situation, set
freezewhen GFS2 is in use. This means that when quorum is lost, the remaining partition will do nothing until quorum is regained.
pcs property set no-quorum-policy=freeze
- Set up a
dlmresource. This is a required dependency for
pcs resource create dlm ocf:pacemaker:controld op monitor interval=30s on-fail=fence clone interleave=true ordered=true
- Execute the following command in each node of the cluster to enable clustered locking. This command sets the
locking_typeparameter in the
/etc/lvm/lvm.conffile to 3.
- Set up
clvmdas a cluster resource.
pcs resource create clvmd ocf:heartbeat:clvm op monitor interval=30s on-fail=fence clone interleave=true ordered=trueNote that the
cmirrorddeamons are started and managed by Pacemaker using the
ocf:heartbeat:clvmresource agent and do not need to be started during boot with
systemd. Additionally, the
ocf:heartbeat:clvmresource agent, as part of the start procedure, sets the
locking_typeparameter in the
/etc/lvm/lvm.conffile to 3 and disables the
- Set up
dlmdependency and start up order.
clvmdmust start after
dlmand must run on the same node as
pcs constraint order start dlm-clone then clvmd-clone#
pcs constraint colocation add clvmd-clone with dlm-clone
- Create the clustered logical volume.
vgcreate -Ay -cy sasbin_vg /dev/vdb#
lvcreate -L5G -n sasbin_lv sasbin_vg
WarningWhen you create volume groups with CLVM on shared storage, you must ensure that all nodes in the cluster have access to the physical volumes that constitute the volume group. Asymmetric cluster configurations in which some nodes have access to the storage and others do not are not supported.When managing volume groups using CLVMD to allow for concurrent activation of volumes across multiple nodes, the volume groups must have the clustered flag enabled. This flag allows CLVMD to identify the volumes it must manage, which is what enables CLVMD to maintain LVM metadata continuity. Failure to adhere to this configuration renders the configuration unsupported by Red Hat and may result in storage corruption and loss of data.
- Format the logical volume with a GFS2 file system. One journal is required for each node that mounts the file system. Ensure that you create enough journals for each of the nodes in your cluster.
mkfs.gfs2 -j2 -p lock_dlm -t rhel7-demo:sasbin /dev/sasbin_vg/sasbin_lv
WarningWhen you create the GFS2 filesystem, it is important to specify a correct value for the
-t LockTableNameoption. The correct format is ClusterName:FSName. Failure to specify a correct value will prevent the filesystem from mounting. Additionally, the file system name must be unique. For more information on the options for the
mkfs.gfs2command, see Section 3.1, “Creating a GFS2 File System”.
- Configure a
clusterfsresource.You should not add the file system to the
/etc/fstabfile because it will be managed as a Pacemaker cluster resource. Mount options can be specified as part of the resource configuration with
options=options. Run the
pcs resource describe Filesystemcommand for full configuration options.This cluster resource creation command specifies the
noatimemount option, which is recommended for GFS2 file systems where the application allows it.In this example, the file system has the same name as the mount point. This is not a requirement, but it is recommended that the file system name relate to its actual use or mount point to help with troubleshooting should the file system encounter a problem.
# pcs resource create clusterfs Filesystem device="/dev/sasbin_vg/sasbin_lv" directory="/usr/local/sasbin" fstype="gfs2" options="noatime" op monitor interval=10s on-fail=fence clone interleave=true
- Set up GFS2 and
clvmddependency and startup order. GFS2 must start after
clvmdand must run on the same node as
pcs constraint order start clvmd-clone then clusterfs-clone#
pcs constraint colocation add clusterfs-clone with clvmd-clone
- Verify that GFS2 is mounted as expected.
mount |grep sas/dev/mapper/sasbin_vg-sasbin_lv on /usr/local/sasbin type gfs2 (rw,noatime,seclabel)