RHEL8: Soft lockup at scan_swap_map_try_ssd_cluster()

Solution Verified - Updated -

Issue

  • Soft lockup occurred with logs:
[181421.172388] watchdog: BUG: soft lockup - CPU#11 stuck for 67s! [kswapd0:243]
[181421.173578] Modules linked in: vfat fat crct10dif_ce ghash_ce sha2_ce sha256_arm64 sha1_ce xfs libcrc32c nvme ena nvme_core dm_mirror dm_region_hash dm_log dm_mod ipmi_devintf ipmi_msghandler
[181421.176339] CPU: 11 PID: 243 Comm: kswapd0 Kdump: loaded Not tainted 4.18.0-372.32.1.el8_6.aarch64 #1
[181421.177868] Hardware name: Amazon EC2 m6g.4xlarge/, BIOS 1.0 11/1/2018
[181421.178955] pstate: 20c00005 (nzCv daif +PAN +UAO)
[181421.179759] pc : scan_swap_map_try_ssd_cluster+0x50/0x190
[181421.180678] lr : scan_swap_map_slots+0xa0/0x8d0
[181421.181439] sp : ffff80001b28f770
[181421.182011] x29: ffff80001b28f770 x28: ffffbce771605a10 
[181421.182898] x27: 0000000002000000 x26: ffff68a31b8a2000 
[181421.183779] x25: ffff68a31b8a2000 x24: fffffbdfffce72d8 
[181421.184669] x23: 0000000000000000 x22: ffff80001b28f860 
[181421.185561] x21: ffff80001b28f858 x20: 000000000001f2ba 
[181421.186464] x19: 000000000001f300 x18: ffffbce770e38788 
[181421.187357] x17: fffffffa2c7b1980 x16: ffffbce76f696138 
[181421.188270] x15: 0000000000000001 x14: fffffffa27a00000 
[181421.189156] x13: 0000000002000001 x12: ffffffdfffe00000 
[181421.190034] x11: 0000000000000000 x10: ffff68b26d7ee580 
[181421.190941] x9 : 000000000000001d x8 : 0000000000000001 
[181421.191864] x7 : ffffbce771605000 x6 : 00000000ffffffce 
[181421.192768] x5 : 0000000460c10000 x4 : ffff68a307ecc980 
[181421.193660] x3 : ffff68a31b071c00 x2 : 0000000000000002 
[181421.194560] x1 : ffffabcafcab0000 x0 : 00005015032372d8 
[181421.195447] Call trace:
[181421.195869]  scan_swap_map_try_ssd_cluster+0x50/0x190
[181421.196724]  scan_swap_map_slots+0xa0/0x8d0
[181421.197440]  get_swap_pages+0x2c8/0x388
[181421.198108]  get_swap_page+0x114/0x218
[181421.198760]  add_to_swap+0x24/0x90
[181421.199346]  shrink_page_list+0x8d0/0xc30
[181421.200036]  shrink_inactive_list+0x1dc/0x490
[181421.200780]  shrink_lruvec+0x3b0/0x5b8
[181421.201413]  shrink_node+0x3d0/0x700
[181421.202042]  balance_pgdat+0x2f4/0x5d0
[181421.202705]  kswapd+0x1bc/0x458
[181421.203259]  kthread+0x128/0x138
[181421.203814]  ret_from_fork+0x10/0x18
[181421.204432] Kernel panic - not syncing: softlockup: hung tasks
[181421.205414] CPU: 11 PID: 243 Comm: kswapd0 Kdump: loaded Tainted: G             L   --------- -  - 4.18.0-372.32.1.el8_6.aarch64 #1
[181421.207337] Hardware name: Amazon EC2 m6g.4xlarge/, BIOS 1.0 11/1/2018
[181421.208432] Call trace:
[181421.208868]  dump_backtrace+0x0/0x160
[181421.209496]  show_stack+0x28/0x38
[181421.210066]  dump_stack+0x5c/0x74
[181421.210628]  panic+0x140/0x30c
[181421.211164]  watchdog_timer_fn+0x264/0x290
[181421.211848]  __hrtimer_run_queues+0x114/0x358
[181421.212578]  hrtimer_interrupt+0x118/0x2d0
[181421.213267]  arch_timer_handler_virt+0x40/0x50
[181421.214027]  handle_percpu_devid_irq+0x94/0x240
[181421.214798]  generic_handle_irq+0x40/0x58
[181421.215463]  __handle_domain_irq+0x70/0xc8
[181421.216148]  gic_handle_irq+0xd8/0x17c
[181421.216788]  el1_irq+0xb8/0x140
[181421.217328]  scan_swap_map_try_ssd_cluster+0x50/0x190
[181421.218157]  scan_swap_map_slots+0xa0/0x8d0
[181421.218855]  get_swap_pages+0x2c8/0x388
[181421.219497]  get_swap_page+0x114/0x218
[181421.220120]  add_to_swap+0x24/0x90
[181421.220695]  shrink_page_list+0x8d0/0xc30
[181421.221346]  shrink_inactive_list+0x1dc/0x490
[181421.222080]  shrink_lruvec+0x3b0/0x5b8
[181421.222716]  shrink_node+0x3d0/0x700
[181421.223329]  balance_pgdat+0x2f4/0x5d0
[181421.223959]  kswapd+0x1bc/0x458
[181421.224520]  kthread+0x128/0x138
[181421.225092]  ret_from_fork+0x10/0x18
[181421.225726] SMP: stopping secondary CPUs
[181421.227913] Starting crashdump kernel...
[181421.228569] Bye!

Environment

  • Red Hat Enterprise Linux 8
  • kernel-4.18.0-372.32.1.el8_6.aarch64
  • aarch64

Subscriber exclusive content

A Red Hat subscription provides unlimited access to our knowledgebase, tools, and much more.

Current Customers and Partners

Log in for full access

Log In

New to Red Hat?

Learn more about Red Hat subscriptions

Using a Red Hat product through a public cloud?

How to access this content