GFS2 filesystem corrupt and fsck fails with "bad seek: Invalid argument from gfs2_replay_read_block" on Red Hat Enterprise Linux 5 or 6
Issue
- There was a power failure and the servers were turned off. After turning on, some gfs2 filesystem cannot be mounted, and
fsck.gfs2doesn't work. - One gfs2 filesystem cannot be mounted, and
fsck.gfs2dies prematurely before it completes. Whenever the command is run, it returns:
Bad read: Success from gfs2_replay_read_block:34: block 142386755796992 (0x818000000000)
Environment
- Red Hat Enterprise Linux (RHEL) 5 or 6 with the Resilient Storage Add On
- 2 or more nodes configured in a Red Hat Enterprise Linux High Availability cluster
- Several GFS2 filesystems
- At least one GFS2 filesystem cannot be successfully fsck'd. fsck fails and the filesystem cannot be mounted afterwards.
- A power outage caused failure of both cluster nodes and storage devices.
- After rebooting nodes, several GFS2 filesystems could not be accessed.
- SAN controller ownership switched from A->B at some point, and it was found to have a duplicate IP addresses
- One of the controller boards was replaced which resolved this.
Subscriber exclusive content
A Red Hat subscription provides unlimited access to our knowledgebase, tools, and much more.
