How can I run an experimental RHEL6 fsck.gfs2 on a RHEL5 GFS2 filesystem?

Solution Verified - Updated -

Environment

  • Red Hat Enterprise Linux Server 5 (with the Resilient Storage Add Ons)
  • GFS2 filesystem
    • Filesystem has encountered some corruption
    • Filesystem corruption cannot be repaired with fsck.gfs2 provided by RHEL5's gfs2-utils package

Issue

  • How can I run an experimental RHEL6 fsck.gfs2 on a RHEL5 GFS2 filesystem?

Resolution

  • It is possible to load up a RHEL6 rescue-mode CD/DVD on one of the affected RHEL5 cluster nodes and to run the RHEL6 fsck.gfs2 from rescue mode to repair the filesystem.
  • This particular instruction includes downloading the latest experimental fsck.gfs2 that contains the latest fixes and logic to attempt to repair the filesystem. If this version is not used, it is recommended that the latest RHEL6 version be used (ie. latest gfs2-utils package).

1. Download a RHEL6 rescue ISO from Red Hat's website. Rescue ISO's are available here.
* Configure the server to boot from the ISO. You may have to burn a CD/DVD, or you may be able to connect the ISO image to a virtual console. Consult your server manual or contact your server hardware support team.

2. Boot off of the CD/DVD, and from the menu, choose Rescue installed system from the menu.

3. Allow the rescue mode to load, and choose your language and keyboard.

4. When it asks "What type of media contains the rescue image?", choose Local CD/DVD.

5. When asked if you want to star the network interfaces on this system, select Yes.
* Select your network interface, and enter the network information so that the server can communicate with the network.

6. When the rescue-mode menu wants to try to find your Linux installation and mount it under the directory /mnt/sysimage1, choose Continue.
* This allows you to record a log of the fsck session to the root disk so you can review at it after booting back into RHEL5.

7. Rescue mode menu should not mention that the filesystem is mounted (if you chose that option), and present you with an "OK" option.
* Choose shell Start shell from the menu, and you should be presented with the rescue shell.

8. Now we need to activate the clustered volume group. The following command should activate the volume group (the --config section is required to activate the clustered volume group without having to bring up the cluster infrastructure):

# vgchange -ay vgstorage --config 'global {locking_type = 0}'

9. Now you can download the experimental fsck.gfs2 file to the rescue shell (or alternatively, use a version that you previously downloaded to the local disks, which could be found under /mnt/sysimage):

# wget http://people.redhat.com/rpeterso/Experimental/RHEL6.x/fsck.gfs2

or

# cd /
# cp /mnt/sysimage/root/fsck.gfs2 .

10. Now make the fsck.gfs2 binary executable:

# chmod u+x ./fsck.gfs2

11. Execute the fsck.gfs2 on the filesystem and capture the output log to the local disk (it will be at /gfs2_fsck.log when the system is booted):

# fsck.gfs2 /dev/vgstorage/lvdata -v | tee /mnt/sysimage/gfs2_fsck.log

12. To recover the system, exit from the shell, and choose "reboot" from the menu to reboot the system.
* Ensure you are booting from the local disk and not from the recovery CDROM.

Root Cause

  • The experimental fsck.gfs2 binary for RHEL6 is more advanced than the version that ships with RHEL5, however it remains compatible with RHEL5 GFS2 filesystems so it can fix more advanced corruption.
    • However, the RHEL6 fsck.gfs2 cannot be run on RHEL5, so the filesystem needs to be exported to a RHEL6 system or fsck.gfs2 can be run from a RHEL6 rescue environment.
  • fsck.gfs2 on RHEL6 is capable of fixing more problems than the RHEL5 version.

Diagnostic Steps

  • This solution may be appropriate if the latest fsck.gfs2 from gfs-utils on RHEL5 is unable to fix gfs2 corruption.

This solution is part of Red Hat’s fast-track publication program, providing a huge library of solutions that Red Hat engineers have created while supporting our customers. To give you the knowledge you need the instant it becomes available, these articles may be presented in a raw and unedited form.

Close

Welcome! Check out the Getting Started with Red Hat page for quick tours and guides for common tasks.