Low memory deadlock with md devices and external (imsm) metadata handling
Issue
-
The server got hung up. Ran sysrq-c to capture vmcore.
-
Lots of tasks are waiting on XFS I/O for page reclaiming with the backtrace like this:
PID: 36781 TASK: ffff96bff2f330c0 CPU: 13 COMMAND: "splunkd" #0 [ffff96c6dbab7578] __schedule at ffffffff8a567747 #1 [ffff96c6dbab7608] schedule at ffffffff8a567c49 #2 [ffff96c6dbab7618] _xfs_log_force_lsn at ffffffffc0839fe4 [xfs] #3 [ffff96c6dbab76a0] xfs_log_force_lsn at ffffffffc083a1a4 [xfs] #4 [ffff96c6dbab76d8] __xfs_iunpin_wait at ffffffffc0827c2c [xfs] #5 [ffff96c6dbab7750] xfs_iunpin_wait at ffffffffc082af89 [xfs] #6 [ffff96c6dbab7760] xfs_reclaim_inode at ffffffffc081e7d3 [xfs] #7 [ffff96c6dbab77b0] xfs_reclaim_inodes_ag at ffffffffc081ec57 [xfs] #8 [ffff96c6dbab7948] xfs_reclaim_inodes_nr at ffffffffc081fe33 [xfs] #9 [ffff96c6dbab7968] xfs_fs_free_cached_objects at ffffffffc0830445 [xfs] #10 [ffff96c6dbab7978] prune_super at ffffffff8a044286 #11 [ffff96c6dbab79b0] shrink_slab at ffffffff89fc9de5 #12 [ffff96c6dbab7a50] do_try_to_free_pages at ffffffff89fcd0a2 #13 [ffff96c6dbab7ac8] try_to_free_pages at ffffffff89fcd2bc #14 [ffff96c6dbab7b60] __alloc_pages_slowpath at ffffffff8a55d1ef #15 [ffff96c6dbab7c50] __alloc_pages_nodemask at ffffffff89fc1145 #16 [ffff96c6dbab7d00] alloc_pages_vma at ffffffff8a011535 #17 [ffff96c6dbab7d70] handle_pte_fault at ffffffff89fe9a77 #18 [ffff96c6dbab7e08] handle_mm_fault at ffffffff89fec01d #19 [ffff96c6dbab7eb0] __do_page_fault at ffffffff8a56f5e3 #20 [ffff96c6dbab7f20] do_page_fault at ffffffff8a56f915 #21 [ffff96c6dbab7f50] page_fault at ffffffff8a56b758 RIP: 0000557a7a3c7318 RSP: 00007f4714dfeb60 RFLAGS: 00010297 RAX: 00007f4c849a6000 RBX: 00007f4ff4616000 RCX: 0000000000116b71 RDX: 00007f4c84eb5b88 RSI: 00007f458ae00000 RDI: 000000000007ffff RBP: 0000000000116b71 R8: 00007f50011305a0 R9: 0000000000000268 R10: 00007f4ff460e000 R11: 00007f4ff460e000 R12: 00000000001975ed R13: 0000000000008098 R14: 0000000000040000 R15: 0000000000000000 ORIG_RAX: ffffffffffffffff CS: 0033 SS: 002b
Environment
- Red Hat Enterprise Linux 8
- Red Hat Enterprise Linux 7
- Red Hat Enterprise Linux 6
- Software RAID (mdraid)
Subscriber exclusive content
A Red Hat subscription provides unlimited access to our knowledgebase, tools, and much more.