[RHEL5.6] e2fsck hangs in "Pass 2: Checking directory structure" on 400GB volume containing many hard links

Solution Verified - Updated -

Issue

  • fsck hangs on 400 GB volume used for rsync backups and containing lots of hard links
  • Disk checks hanging. We have multi TB disks that are marked as clean with errors.
    When I run e2fsck -fptt it hangs on pass two.

  • strace of fsck shows hung in "Pass 2: Checking directory structure", after a 4k read:

    1963  20:50:27 write(1, "                                "..., 80) = 80
    1963  20:50:27 write(1, "Pass 2: Checking ", 17) = 17
    1963  20:50:27 write(1, "directory", 9) = 9
    1963  20:50:27 write(1, " structure\n", 11) = 11
    1963  20:50:27 brk(0x4a2b000)           = 0x4a2b000
    1963  20:50:27 brk(0x56ab000)           = 0x56ab000
    1963  20:50:27 mmap(NULL, 154857472, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x2ab37bef3000
    1963  20:50:27 mmap(NULL, 200712192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x2ab3852a2000
    1963  20:50:33 munmap(0x2ab3852a2000, 200712192) = 0
    1963  20:50:33 lseek(3, 6303744, SEEK_SET) = 6303744
    1963  20:50:33 read(3, "\2\0\0\0\f\0\1\2.\0\0\0\2\0\0\0\f\0\2\2..\0\0\v\0\0\0\24\0\n\2"..., 4096) = 4096
    ...
    1963  20:50:33 lseek(3, 147316736, SEEK_SET) = 147316736
    1963  20:50:33 read(3, "\347G\0\0\f\0\1\2.\0\0\0\231\307?\6\364\17\2\2..\0\0\0\0\0\0\0\0\0\0"..., 4096) = 4096
    1963  21:09:07 --- SIGINT (Interrupt) @ 0 (0) ---
    1963  21:09:07 rt_sigreturn(0x2)        = 104843161
    
  • A backtrace of fsck during execution shows the following:

    (gdb) bt
    0  get_icount_el (icount=0x19948d40, ino=104843161, create=1) at icount.c:257
    1  0x000000000041cd0b in ext2fs_icount_increment (icount=0x19948d40, ino=104843161, ret=0x7fff4e11e74e) at icount.c:340
    2  0x000000000040ada4 in check_dir_block (fs=0x1990ce20, db=0x2adcd60a13e8, priv_data=0x7fff4e11e7d0) at pass2.c:1014
    3  0x000000000041a673 in ext2fs_dblist_iterate (dblist=0x199349c0, func=0x40a720 , priv_data=0x7fff4e11e7d0) at dblist.c:235
    4  0x000000000040a2d7 in e2fsck_pass2 (ctx=0x1990cad0) at pass2.c:149
    5  0x00000000004030a6 in e2fsck_run (ctx=0x1990cad0) at e2fsck.c:203
    6  0x00000000004023af in main (argc=, argv=) at unix.c:1148 
    

Environment

  • Red Hat Enterprise Linux 5.6
    • e2fsprogs-1.39-23.el5_5.1
  • Certain type ext3 filesystem

Subscriber exclusive content

A Red Hat subscription provides unlimited access to our knowledgebase, tools, and much more.

Current Customers and Partners

Log in for full access

Log In

New to Red Hat?

Learn more about Red Hat subscriptions

Using a Red Hat product through a public cloud?

How to access this content