RHEL8: Soft lockup occurred during __page_mapcount and split_huge_page_to_list
Issue
- soft lockup occurred with logs:
[241888.165144] watchdog: BUG: soft lockup - CPU#45 stuck for 22s! [kswapd1:514]
[241888.173282] Modules linked in: mptcp_diag xsk_diag raw_diag unix_diag af_packet_diag netlink_diag udp_diag tcp_diag inet_diag rpcsec_gss_krb5 nfsv4 dns_resolver nfs lockd grace fscache bonding tls intel_rapl_msr intel_rapl_common sb_edac x86_pkg_temp_thermal mgag200 intel_powerclamp coretemp irqbypass i2c_algo_bit crct10dif_pclmul crc32_pclmul drm_shmem_helper drm_kms_helper ghash_clmulni_intel ipmi_ssif syscopyarea rapl sysfillrect sysimgblt fb_sys_fops iTCO_wdt intel_cstate ioatdma iTCO_vendor_support usbnet mei_me mii i2c_i801 mxm_wmi drm lpc_ich intel_uncore mei pcspkr dca ipmi_si ipmi_devintf acpi_pad ipmi_msghandler wmi acpi_power_meter vfat fat binfmt_misc auth_rpcgss sunrpc xfs libcrc32c sd_mod t10_pi sg crc32c_intel tg3 megaraid_sas dm_mirror dm_region_hash dm_log dm_mod
[241888.251280] CPU: 45 PID: 514 Comm: kswapd1 Kdump: loaded Not tainted 4.18.0-425.10.1.el8_7.x86_64 #1
[241888.261696] Hardware name: LENOVO Lenovo NeXtScale nx360 M5: -[5465AC1]-/00YE752, BIOS -[THE130I-2.40]- 04/05/2017
[241888.273473] RIP: 0010:native_queued_spin_lock_slowpath+0x61/0x1c0
[241888.280502] Code: 0f ba 2f 08 0f 92 c0 0f b6 c0 c1 e0 08 89 c2 8b 07 30 e4 09 d0 a9 00 01 ff ff 75 4b 85 c0 74 0e 8b 07 84 c0 74 08 f3 90 8b 07 <84> c0 75 f8 b8 01 00 00 00 66 89 07 e9 1e b7 aa 00 8b 37 81 fe 00
[241888.301845] RSP: 0018:ffff9c810847f998 EFLAGS: 00000202 ORIG_RAX: ffffffffffffff13
[241888.310540] RAX: 0000000000000101 RBX: ffff9c810847f9d8 RCX: 000ffffffffff000
[241888.318752] RDX: 0000000000000000 RSI: 0000000000000000 RDI: ffffe0d80e488f28
[241888.326968] RBP: ffff8c2736d34e00 R08: 0000000000000067 R09: ffffe0d80e518000
[241888.335188] R10: ffffe0d80e518048 R11: ffffe0d80e518008 R12: ffffe0d80e518000
[241888.343421] R13: 03ffffffffffffff R14: 0000039fc6a00000 R15: 7c00000000000000
[241888.351638] FS: 0000000000000000(0000) GS:ffff8c2dffa40000(0000) knlGS:0000000000000000
[241888.360938] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[241888.367616] CR2: 000003dd36840080 CR3: 0000001712a10003 CR4: 00000000003706e0
[241888.375850] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[241888.384087] DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
[241888.392318] Call Trace:
[241888.395316] _raw_spin_lock+0x1e/0x30
[241888.399678] page_vma_mapped_walk+0x3b8/0x8b0
[241888.404817] remove_migration_pte+0x7f/0x420
[241888.409859] rmap_walk_anon+0xe7/0x2a0
[241888.414320] remove_migration_ptes+0x53/0x80
[241888.419362] ? migrate_vma_check_page+0xb0/0xb0
[241888.424698] remap_page+0x4a/0x70
[241888.428676] split_huge_page_to_list+0x894/0xa50
[241888.434113] ? free_unref_page_commit+0x91/0x110
[241888.439553] deferred_split_scan+0x1ca/0x310
[241888.444606] do_shrink_slab+0x11d/0x330
[241888.449176] shrink_slab+0x249/0x2f0
[241888.453454] shrink_node+0x246/0x700
[241888.457730] balance_pgdat+0x2d7/0x550
[241888.462206] kswapd+0x201/0x3c0
[241888.465999] ? finish_wait+0x80/0x80
[241888.470285] ? balance_pgdat+0x550/0x550
[241888.474958] kthread+0x10b/0x130
[241888.478857] ? set_kthread_struct+0x50/0x50
[241888.483822] ret_from_fork+0x35/0x40
- Another logs:
[73592.755861] watchdog: BUG: soft lockup - CPU#33 stuck for 22s! [kernel_server:2399822]
[73592.831735] CPU: 28 PID: 2329307 Comm: kernel_server Kdump: loaded Not tainted 4.18.0-425.10.1.el8_7.x86_64 #1
[73592.852000] Hardware name: LENOVO Lenovo NeXtScale nx360 M5: -[5465AC1]-/00YE752, BIOS -[THE130I-2.40]- 04/05/2017
[73592.855525] RIP: 0010:__page_mapcount+0x98/0xa0
[73593.024082] Call Trace:
[73592.855525] RIP: 0010:__page_mapcount+0x98/0xa0
[73593.036206] smaps_account+0xb9/0x230
[73593.042702] smaps_pte_range+0x343/0x600
[73593.050760] walk_pgd_range+0x46f/0xa20
[73593.058043] __walk_page_range+0x57/0x180
[73593.067342] walk_page_vma+0x66/0x90
[73593.077020] smap_gather_stats+0x42/0xa0
[73593.086289] show_smap+0x42/0x1c0
[73593.095065] seq_read+0x303/0x420
[73593.101299] vfs_read+0x91/0x150
[73593.107999] ksys_read+0x4f/0xb0
[73593.115663] do_syscall_64+0x5b/0x1b0
[73593.121939] entry_SYSCALL_64_after_hwframe+0x61/0xc6
Environment
- Red Hat Enterprise Linux 8.7
- Red Hat Enterprise Linux 8.6
Subscriber exclusive content
A Red Hat subscription provides unlimited access to our knowledgebase, tools, and much more.