ext3_lookup が "unlinked inode" を検出し ext3_abort を呼び出すと読み取り専用のファイルシステムになる

Issue

• ext3 ファイルシステムが読み取り専用になりました。/var/log/messages で以下のエラーがログに記録されました。

EXT3-fs error (device dm-4): ext3_lookup: unlinked inode 91626760 in dir #19039379
Aborting journal on device dm-4.
ext3_abort called.
EXT3-fs error (device dm-4): ext3_journal_start_sb:Detected aborted journal
Remounting filesystem read-only

• (「解決策」に記載されている方法で復旧してから) 数日後:

EXT3-fs error (device dm-4): ext3_free_blocks_sb: bit already cleared for block 43951098 
Aborting journal on device dm-4. 
EXT3-fs error (device dm-4) in ext3_free_blocks_sb:Journal has aborted 
EXT3-fs error (device dm-4) in ext3_reserve_inode_write:Journal has aborted 
EXT3-fs error (device dm-4) in ext3_truncate:Journal has aborted 
EXT3-fs error (device dm-4) in ext3_reserve_inode_write:Journal has aborted 
EXT3-fs error (device dm-4) in ext3_orphan_del:Journal has aborted 
EXT3-fs error (device dm-4) in ext3_reserve_inode_write:Journal has aborted 
EXT3-fs error (device dm-4) in ext3_delete_inode:Journal has aborted 
ext3_abort called. 
EXT3-fs error (device dm-4): ext3_journal_start_sb:Detected aborted journal 
Remounting filesystem read-only 
__journal_remove_journal_head: freeing b_committed_data 

• nfs がクロスマウントされている 2 台のサーバーで nfs ext3 ファイルシステムが読み取り専用となっています。

Aug 16 11:57:17 server1 kernel:EXT3-fs error (device dm-10): ext3_lookup: unlinked inode 2339312 in dir #318520
Aug 16 11:57:17 server1 kernel:Aborting journal on device dm-10.
Aug 16 11:57:17 server1 kernel:EXT3-fs error (device dm-10): ext3_lookup: unlinked inode 2339315 in dir #318520
Aug 16 11:57:17 server1 kernel:EXT3-fs error (device dm-10): ext3_lookup: unlinked inode 2339293 in dir #318520
Aug 16 11:57:17 server1 kernel:EXT3-fs error (device dm-10): ext3_lookup: unlinked inode 2339321 in dir #318520
Aug 16 11:57:17 server1 kernel:EXT3-fs error (device dm-10): ext3_lookup: unlinked inode 2339323 in dir #318520
Aug 16 11:57:17 server1 kernel:EXT3-fs error (device dm-10): ext3_lookup: unlinked inode 2339276 in dir #318520
Aug 16 11:57:17 server1 kernel:EXT3-fs error (device dm-10): ext3_lookup: unlinked inode 2339322 in dir #318520
Aug 16 11:57:17 server1 kernel:EXT3-fs error (device dm-10): ext3_lookup: unlinked inode 2339291 in dir #318520
Aug 16 11:57:17 server1 kernel: ext3_abort called.
Aug 16 11:57:17 server1 kernel:EXT3-fs error (device dm-10): ext3_journal_start_sb:Detected aborted journal
Aug 16 11:57:17 server1 kernel:Remounting filesystem read-only
Aug 16 12:02:10 server1 multipathd: dm-10: umount map (uevent) 
Aug 16 12:02:11 server1 kernel: ext3_abort called.
Aug 16 12:02:11 server1 kernel:EXT3-fs error (device dm-10): ext3_put_super:Couldn't clean up the journal
Aug 16 12:03:55 server1 multipathd: dm-10: remove map (uevent) 
Aug 16 12:05:01 server1 multipathd: dm-10: add map (uevent) 
Aug 16 12:05:22 server1 kernel: kjournald starting.Commit interval 5 seconds
Aug 16 12:05:22 server1 kernel:EXT3-fs warning (device dm-10): ext3_clear_journal_err:Filesystem error recorded from previous mount:IO failure
Aug 16 12:05:22 server1 kernel:EXT3-fs warning (device dm-10): ext3_clear_journal_err:Marking fs in need of filesystem check.
Aug 16 12:05:22 server1 kernel:EXT3-fs warning: mounting fs with errors, running e2fsck is recommended