How to recover data from a XFS filesystem hosted on a full VDO device

Issue

• When the VDO device is full, dmesg shows:

  kernel: Buffer I/O error on dev dm-4, logical block 1993267920, lost async page write
  kernel: XFS (dm-4): metadata I/O error: block 0x2000adb08 ("xlog_iodone") error 28 numblks 64
  kernel: XFS (dm-4): xfs_do_force_shutdown(0x2) called from line 1222 of file fs/xfs/xfs_log.c.  Return address = 0xffffffffc058ce20
  kernel: XFS (dm-4): Log I/O Error Detected.  Shutting down filesystem
  kernel: XFS (dm-4): Please umount the filesystem and rectify the problem(s)
  

  • Trying to remount this file system results in error:

    # mount /data
    mount: mount /dev/mapper/ssd_vdo on /ssd failed: No space left on device
    

  • And dmesg shows:

    kernel: XFS (dm-4): Mounting V5 Filesystem
    kernel: XFS (dm-4): xfs_do_force_shutdown(0x1) called from line 1238 of file fs/xfs/xfs_buf.c.  Return address = 0xffffffffc056c3cc
    kernel: XFS (dm-4): I/O Error Detected. Shutting down filesystem
    kernel: XFS (dm-4): Please umount the filesystem and rectify the problem(s)
    kernel: XFS (dm-4): metadata I/O error: block 0x2000adb08 ("xlog_bwrite") error 28 numblks 8200
    kernel: XFS (dm-4): failed to locate log tail
    kernel: XFS (dm-4): log mount/recovery failed: error -28
    kernel: XFS (dm-4): log mount failed