NMI Watchdog detected hard lockup on the CPU running user-space code
Issue
- NMI Watchdog detected hard lockup on the CPU running user-space code, and the system crashed with the following call traces:
[Fri May 6 09:43:23 EDT 2022] NMI watchdog: Watchdog detected hard LOCKUP on cpu 21
[Fri May 6 09:43:23 EDT 2022] Modules linked in: iptable_filter ...
[Fri May 6 09:43:23 EDT 2022] libata tg3 megaraid_sas mdio mtd drm_panel_orientation_quirks ptp pps_core nfit libnvdimm dm_mirror dm_region_hash dm_log dm_mod
[Fri May 6 09:43:23 EDT 2022] CPU: 21 PID: 239601 Comm: kepler_data_bor Kdump: loaded Tainted: GOE ------------ 3.10.0-1160.21.1.el7.x86_64 #1
[Fri May 6 09:43:23 EDT 2022] Hardware name: Lenovo Thinksystem SR665/7D2VCTO1WW, BIOS D8E110K-1.21 12/16/2020
[Fri May 6 09:43:23 EDT 2022] task: ffff950894392100 ti: ffff9508fdf98000 task.ti: ffff9508fdf98000
[Fri May 6 09:43:23 EDT 2022] RIP: 0033:[<00007fdd0a532832>] [<00007fdd0a532832>] 0x7fdd0a532832
[Fri May 6 09:43:23 EDT 2022] RSP: 002b:00007fff7cc07750 EFLAGS: 00000246
[Fri May 6 09:43:23 EDT 2022] RAX: 00007fdd0a83fba0 RBX: 000056435e47b8b0 RCX: 7fffffffffffffff
[Fri May 6 09:43:23 EDT 2022] RDX: 0000000000000000 RSI: 00007fdcedcbb510 RDI: 00007fdcf57678f0
[Fri May 6 09:43:23 EDT 2022] RBP: 00007fdcf57678f0 R08: 00000000ffffffff R09: 00007fff7cc07480
[Fri May 6 09:43:23 EDT 2022] R10: 0000000000000000 R11: 0000000000000001 R12: 00007fdd0aa546c0
[Fri May 6 09:43:23 EDT 2022] R13: 000056435e47bad0 R14: 00007fdc7f18f680 R15: 00007fdcb7522e50
[Fri May 6 09:43:23 EDT 2022] FS: 00007fdd0aa8d740(0000) GS:ffff96024e540000(0000) knlGS:0000000000000000
[Fri May 6 09:43:23 EDT 2022] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[Fri May 6 09:43:23 EDT 2022] CR2: 00007fdc8de6d010 CR3: 000001079bbb2000 CR4: 0000000000340fe0
[Fri May 6 09:43:23 EDT 2022]
[Fri May 6 09:43:23 EDT 2022] Kernel panic - not syncing: Hard LOCKUP
[Fri May 6 09:43:23 EDT 2022] CPU: 21 PID: 239601 Comm: kepler_data_bor Kdump: loaded Tainted: GOE ------------ 3.10.0-1160.21.1.el7.x86_64 #1
[Fri May 6 09:43:23 EDT 2022] Hardware name: Lenovo Thinksystem SR665/7D2VCTO1WW, BIOS D8E110K-1.21 12/16/2020
[Fri May 6 09:43:23 EDT 2022] Call Trace:
[Fri May 6 09:43:23 EDT 2022] [<ffffffffb998305a>] dump_stack+0x19/0x1b
[Fri May 6 09:43:23 EDT 2022] [<ffffffffb997c5b2>] panic+0xe8/0x21f
[Fri May 6 09:43:23 EDT 2022] [<ffffffffb922f958>] ? show_regs+0x58/0x290
[Fri May 6 09:43:23 EDT 2022] [<ffffffffb929ae9f>] nmi_panic+0x3f/0x40
[Fri May 6 09:43:23 EDT 2022] [<ffffffffb934eeb9>] watchdog_overflow_callback+0x119/0x140
[Fri May 6 09:43:23 EDT 2022] [<ffffffffb93a8a47>] __perf_event_overflow+0x57/0x100
[Fri May 6 09:43:23 EDT 2022] [<ffffffffb93b2244>] perf_event_overflow+0x14/0x20
[Fri May 6 09:43:23 EDT 2022] [<ffffffffb9205595>] x86_pmu_handle_irq+0x125/0x180
[Fri May 6 09:43:23 EDT 2022] [<ffffffffb9337df1>] ? audit_copy_inode+0xa1/0xd0
[Fri May 6 09:43:23 EDT 2022] [<ffffffffb92e2ffb>] ? should_numa_migrate_memory+0x5b/0x150
[Fri May 6 09:43:23 EDT 2022] [<ffffffffb93f34be>] ? do_numa_page+0x1be/0x250
[Fri May 6 09:43:23 EDT 2022] [<ffffffffb93f5e0b>] ? handle_mm_fault+0xadb/0xfb0
[Fri May 6 09:43:23 EDT 2022] [<ffffffffb9297bfa>] ? get_task_exe_file+0x3a/0x50
[Fri May 6 09:43:23 EDT 2022] [<ffffffffb9340853>] ? audit_exe_compare+0x43/0x50
[Fri May 6 09:43:23 EDT 2022] [<ffffffffb92066f5>] amd_pmu_handle_irq+0x35/0x80
[Fri May 6 09:43:23 EDT 2022] [<ffffffffb998c031>] perf_event_nmi_handler+0x31/0x50
[Fri May 6 09:43:23 EDT 2022] [<ffffffffb998d93c>] nmi_handle.isra.0+0x8c/0x150
[Fri May 6 09:43:23 EDT 2022] [<ffffffffb998db5d>] do_nmi+0x15d/0x460
[Fri May 6 09:43:23 EDT 2022] [<ffffffffb998ccc9>] nmi+0x109/0x158
Environment
- Red Hat Enterprise Linux 7
- Lenovo Thinksystem SR665
- AMD EPYC 7542 32-Core Processor
Subscriber exclusive content
A Red Hat subscription provides unlimited access to our knowledgebase, tools, and much more.