ext4 file system reports ext4_ext_check_inode(), mb_free_blocks() and ext4_free_blocks() functions on a filesystem.

Solution In Progress - Updated -

Environment

  • Red Hat Enterprise Linux Server 5
  • Red Hat Enterprise Linux Server 6
  • e2fsprogs-1.39-37.el5.x86_64

Issue

  • ext4 file system corrupted with following messages reporting on logs .
 Jun 14 04:03:32 testserverxxx01 kernel: EXT4-fs error (device dm-4): ext4_ext_check_inode: bad header/extent in inode #1441944: invalid magic - magic 4f69, entries 0, max 28692(0), depth 17(17)
 Jun 14 04:03:32 testserverxxx01 kernel: EXT4-fs error (device dm-4): ext4_ext_check_inode: bad header/extent in inode #1448735: invalid magic - magic 7900, entries 91, max 28893(0), depth 17(17)
 ... 
 Jun 14 13:37:16 testserverxxx01 kernel: EXT4-fs error (device dm-4): ext4_mb_complex_scan_group: 2 free blocks as per group info. But bitmap says 0
 Jun 14 13:37:41 testserverxxx01 kernel: EXT4-fs error (device dm-4): ext4_mb_complex_scan_group: 1 free blocks as per group info. But bitmap says 0
 ... 
 Jun 21 15:01:26 testserverxxx01 kernel: EXT4-fs error (device dm-4): ext4_free_blocks: Freeing blocks in system zone - Block = 6619186, count = 1 
 Jun 21 15:01:26 testserverxxx01 kernel: EXT4-fs error (device dm-4): ext4_free_blocks: Freeing blocks in system zone - Block = 3604534, count = 1 
 Jun 21 15:05:55 testserverxxx01 kernel: EXT4-fs error (device dm-4): mb_free_blocks: double-free of inode 57886659's block 256(bit 256 in group 0)
 Jun 21 15:05:55 testserverxxx01 kernel: EXT4-fs error (device dm-4): mb_free_blocks: double-free of inode 57886659's block 106510(bit 8206 in group 3)
 Jun 21 15:05:55 testserverxxx01 kernel: EXT4-fs error (device dm-4): mb_free_blocks: double-free of inode 57886659's block 106510(bit 8206 in group 3)
 ... 
 Jun 22 02:08:18 testserverxxx01 kernel: EXT4-fs error (device dm-4): ext4_ext_check_inode: bad header/extent in inode #2239671: invalid magic - magic 0, entries 0, max 0(0), depth 0(0)
 Jun 22 02:08:18 testserverxxx01 kernel: EXT4-fs error (device dm-4): ext4_ext_check_inode: bad header/extent in inode #2239671: invalid magic - magic 0, entries 0, max 0(0), depth 0(0)

Resolution

  • There is no definite reason for the filesystem reporting the above errors indicated . Some possible cause of the issue maybe due to problems caused by storage hardware. Hardware diagnostics are recommended.

Troubleshooting

When troubleshooting this issue, it would be useful to collect the following data before repairing the filesystem:

  • The whole dmesg output
  • The whole /var/log/messages file to find from when the issue was commenced .
  • Diagnostic image of the file system in question, using a command like:
    e2image -r /dev/<device> - | bzip2 > <device>.e2i.bz2
  • Workload description (percentage of read/write, sequential/random, directories and files per directory, ways files are interacted with)
  • Stop all application using the filesystem, umount it and then run the following commands:
 # fsck.ext4 -cc -fyv /dev/dm-4  |tee /tmp/01464761_fsck_badblocks.out

Root Cause

  • The EXT4-fs error reported by "ext4_mb_generate_buddy" simply means that there is an inconsistency between the block allocation bitmap and the number of free blocks reported in a particular block group descriptor.

This solution is part of Red Hat’s fast-track publication program, providing a huge library of solutions that Red Hat engineers have created while supporting our customers. To give you the knowledge you need the instant it becomes available, these articles may be presented in a raw and unedited form.

Comments