I'm unable to grow a GFS2 filesystem

Solution Verified - Updated -

Environment

  • Red Hat Enterprise Linux 5
  • Red Hat Cluster Suite
  • Clustered LVM (clmvd)
  • GFS2 clustered filesystem

Issue

  • I'm trying to extend a GFS2 filesystem on Clustered LVM. The gfs2_grow command is returning the error:
# gfs2_grow /mountpoint
Error: The device has grown by less than one Resource Group (RG).
The device grew by 30MB.  One RG is 255MB for this file system.
gfs2_grow complete.

Resolution

  • Ensure all nodes in the cluster have the same LVM Locking Type:
grep locking /etc/lvm/lvm.conf | grep -v "#"

If the locking types do not match, then follow these points, one node at a time:

*   Edit the `/etc/lvm/lvm.conf` and change to `locking_type = 3`

*   Stop the clustered services as described at:
    *What services/daemons are available for Red Hat Enterprise Linux Cluster?*
    [https://access.redhat.com/knowledge/node/5898](https://access.redhat.com/knowledge/node/5898)

*   Reboot the cluster node

*   When that node is up again, repeat these steps on another node with the wrong locking type
  • Ensure all nodes in the cluster have the same version of the GFS2 filesystem tools:
rpm -qa | grep gfs2-utils

If any nodes have an older version, then follow these points, one node at a time:

*   Install the correct version of the `gfs2-utils` package with `yum` or `rpm`

*   Stop the clustered services as described at:
    *What services/daemons are available for Red Hat Enterprise Linux Cluster?*
    [https://access.redhat.com/knowledge/node/5898](https://access.redhat.com/knowledge/node/5898)

*   Reboot the cluster node

*   When that node is up again, repeat from Step 1 on another node with the wrong tools version
  • Refresh the Clustered LVM state:
clvmd -R
  • Try to extend the size of the filesystem again:
gfs2_grow /mountpoint

Diagnostic Steps

Has the filesystem's underlying block device been resized?

Investigate filesystem location in /etc/fstab

Investigate filesystem size in df -h

Investigate underlying LVM Logical Volume in lvs -av

Filesystem is 6.5Tb and LV has been correctly expanded to 8Tb

Are there any known issues with this error message?

There was a bug fixed with RHBA-2012-0269 which related to size calculations of gfs2_grow, however that is fixed in gfs2-utils-0.1.62-34.el5 and the node running the resize does have this gfs2-utils version.

In older RHEL5 it was necessary to manually refresh the clustered LVM state with clvmd -R so let's see if clvm has a consistent view of the LVM state:

Are all nodes using the same LVM locking type?

$ grep locking node-*/etc/lvm/lvm.conf | grep -v "#"
node-1/etc/lvm/lvm.conf:    locking_type = 3
node-2/etc/lvm/lvm.conf:    locking_type = 1
node-3/etc/lvm/lvm.conf:    locking_type = 1
node-4/etc/lvm/lvm.conf:    locking_type = 1

No. All nodes in the cluster need the same Clustered LVM locking type.

Are all nodes running the updated gfs2-utils version?

$ grep gfs2-utils node-*/installed-rpms
node-1/installed-rpms:gfs2-utils-0.1.62-34.el5.x86_64
node-2/installed-rpms:gfs2-utils-0.1.62-34.el5.x86_64
node-3/installed-rpms:gfs2-utils-0.1.62-28.el5_6.1.x86_64
node-4/installed-rpms:gfs2-utils-0.1.62-34.el5.x86_64

No. All nodes in the cluster need the same version of the cluster tools.

Resolving these inconsistencies allowed the filesystem to grow successfully.

This solution is part of Red Hat’s fast-track publication program, providing a huge library of solutions that Red Hat engineers have created while supporting our customers. To give you the knowledge you need the instant it becomes available, these articles may be presented in a raw and unedited form.