GFS2 filesystem corrupt and fsck fails with "bad seek: Invalid argument from gfs2_replay_read_block" on Red Hat Enterprise Linux 5 or 6

Solution Unverified - Updated -

Issue

  • There was a power failure and the servers were turned off. After turning on, some gfs2 filesystem cannot be mounted, and fsck.gfs2 doesn't work.
  • One gfs2 filesystem cannot be mounted, and fsck.gfs2 dies prematurely before it completes. Whenever the command is run, it returns:
Bad read: Success from gfs2_replay_read_block:34: block 142386755796992 (0x818000000000)

Environment

  • Red Hat Enterprise Linux (RHEL) 5 or 6 with the Resilient Storage Add On
  • 2 or more nodes configured in a Red Hat Enterprise Linux High Availability cluster
  • Several GFS2 filesystems
    • At least one GFS2 filesystem cannot be successfully fsck'd. fsck fails and the filesystem cannot be mounted afterwards.
  • A power outage caused failure of both cluster nodes and storage devices.
    • After rebooting nodes, several GFS2 filesystems could not be accessed.
  • SAN controller ownership switched from A->B at some point, and it was found to have a duplicate IP addresses
    • One of the controller boards was replaced which resolved this.

Subscriber exclusive content

A Red Hat subscription provides unlimited access to our knowledgebase of over 48,000 articles and solutions.

Current Customers and Partners

Log in for full access

Log In
Close

Welcome! Check out the Getting Started with Red Hat page for quick tours and guides for common tasks.