XFS filesystem shutdown following metadata corruption detected in xfs_inode_buf_verify

Issue

• Our XFS filesystem shutdown and reported the following

  XFS (dm-2): Metadata corruption detected at xfs_inode_buf_verify+0x14d/0x160 [xfs], xfs_inode block 0x23d5a940 xfs_inode_buf_verify
  XFS (dm-2): Unmount and run xfs_repair
  XFS (dm-2): First 128 bytes of corrupted metadata buffer:
  ffff9cd56ea3f000: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  ................
  ffff9cd56ea3f010: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  ................
  ffff9cd56ea3f020: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  ................
  ffff9cd56ea3f030: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  ................
  ffff9cd56ea3f040: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  ................
  ffff9cd56ea3f050: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  ................
  ffff9cd56ea3f060: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  ................
  ffff9cd56ea3f070: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  ................
  XFS (dm-2): metadata I/O error in "xfs_trans_read_buf_map" at daddr 0x23d5a940 len 32 error 117
  XFS (dm-2): xfs_imap_to_bp: xfs_trans_read_buf() returned error -117.
  XFS (dm-2): xfs_iunlink_remove: xfs_imap_to_bp returned error -117.
  XFS (dm-2): xfs_inactive_ifree: xfs_ifree returned error -117
  XFS (dm-2): xfs_do_force_shutdown(0x1) called from line 1753 of file fs/xfs/xfs_inode.c.  Return address = ffffffffc03d0343
  XFS (dm-2): I/O Error Detected. Shutting down filesystem
  XFS (dm-2): Please umount the filesystem and rectify the problem(s)
  

• Filesystem shutdown with XFS read verifier failure

  XFS (dm-1): Metadata CRC error detected at xfs_dir3_data_read_verify+0xcd/0xf0 [xfs], xfs_dir3_data block 0x351f7880
  XFS (dm-1): Unmount and run xfs_repair
  XFS (dm-1): First 128 bytes of corrupted metadata buffer:
  ffff9501473ba000: 49 4e 81 a4 03 02 00 00 00 00 00 00 00 00 00 00  IN..............
  ffff9501473ba010: 00 00 00 01 00 00 00 00 00 00 00 00 00 00 00 00  ................
  ffff9501473ba020: 62 8b 49 01 16 a8 aa 38 62 8b 49 01 16 a8 aa 38  b.I....8b.I....8
  ffff9501473ba030: 62 8b 49 01 16 a8 aa 38 00 00 00 00 00 00 00 00  b.I....8........
  ffff9501473ba040: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  ................
  ffff9501473ba050: 00 00 22 01 00 00 00 00 00 00 00 00 3f 51 f4 77  ..".........?Q.w
  ffff9501473ba060: ff ff ff ff d6 d6 05 71 00 00 00 00 00 00 00 04  .......q........
  ffff9501473ba070: 00 28 49 98 00 00 89 ae 00 00 00 00 00 00 00 00  .(I.............
  XFS (dm-1): metadata I/O error in "xfs_trans_read_buf_map" at daddr 0x351f7880 len 8 error 74
  XFS (dm-1): xfs_do_force_shutdown(0x1) called from line 316 of file fs/xfs/xfs_trans_buf.c.  Return address = ffffffffc0b1cf6e
  XFS (dm-1): I/O Error Detected. Shutting down filesystem
  XFS (dm-1): xfs_inactive_ifree: xfs_trans_commit returned error -5
  XFS (dm-1): xfs_do_force_shutdown(0x1) called from line 239 of file fs/xfs/libxfs/xfs_defer.c.  Return address = ffffffffc0acc2fb
  XFS (dm-1): Please umount the filesystem and rectify the problem(s)