A hard lockup happens on a CPU on which a task is stuck waiting on qwrlock spinlock
Issue
- A hard lockup happened on a CPU on which a task was stuck waiting on qwrlock spinlock:
[486163.609652] Kernel panic - not syncing: Hard LOCKUP
[486163.609790] CPU: 5 PID: 52922 Comm: awk Kdump: loaded Tainted: G OEL ------------ 3.10.0-1160.2.2.el7.x86_64 #1
[486163.610063] Hardware name: Dell Inc. PowerEdge R6525/0GK70M, BIOS 2.10.2 10/25/2022
[486163.610326] Call Trace:
[486163.610454] <NMI> [<ffffffff97f813c0>] dump_stack+0x19/0x1b
[486163.610598] [<ffffffff97f7a918>] panic+0xe8/0x21f
[486163.610734] [<ffffffff97f93f92>] ? system_call_fastpath+0x25/0x2a
[486163.610876] [<ffffffff9789af0f>] nmi_panic+0x3f/0x40
[486163.611014] [<ffffffff9794ede1>] watchdog_overflow_callback+0x121/0x140
[486163.611156] [<ffffffff979a88a7>] __perf_event_overflow+0x57/0x100
[486163.611298] [<ffffffff979b20a4>] perf_event_overflow+0x14/0x20
[486163.611439] [<ffffffff97805595>] x86_pmu_handle_irq+0x125/0x180
[486163.611579] [<ffffffff97b88358>] ? ioremap_page_range+0x2e8/0x480
[486163.611721] [<ffffffff97a051b4>] ? vunmap_page_range+0x234/0x470
[486163.611863] [<ffffffff97a05401>] ? unmap_kernel_range_noflush+0x11/0x20
[486163.612006] [<ffffffff97c4f78e>] ? ghes_copy_tofrom_phys+0x10e/0x210
[486163.612149] [<ffffffff97c4f930>] ? ghes_read_estatus+0xa0/0x190
[486163.612289] [<ffffffff978066f5>] amd_pmu_handle_irq+0x35/0x80
[486163.612430] [<ffffffff97f8a031>] perf_event_nmi_handler+0x31/0x50
[486163.612571] [<ffffffff97f8b93c>] nmi_handle.isra.0+0x8c/0x150
[486163.612712] [<ffffffff97f8bb5d>] do_nmi+0x15d/0x460
[486163.612850] [<ffffffff97f8ad9c>] end_repeat_nmi+0x1e/0x81
[486163.612996] [<ffffffff97917812>] ? queued_write_lock_slowpath+0x62/0x90
[486163.613138] [<ffffffff97917812>] ? queued_write_lock_slowpath+0x62/0x90
[486163.613281] [<ffffffff97917812>] ? queued_write_lock_slowpath+0x62/0x90
[486163.613424] <EOE> [<ffffffff97f89d11>] _raw_qwrite_lock+0x21/0x30
[486163.613569] [<ffffffff978976bb>] tasklist_write_lock_irq+0x3b/0x50
[486163.613711] [<ffffffff978a1958>] do_exit+0x328/0xa20
[486163.613849] [<ffffffff978a20cf>] do_group_exit+0x3f/0xa0
[486163.613988] [<ffffffff978a2144>] SyS_exit_group+0x14/0x20
[486163.614127] [<ffffffff97f93f92>] system_call_fastpath+0x25/0x2a
Environment
- Red Hat Enterprise Linux 7.9.z - kernel-3.10.0-1160.2.2.el7.x86_64
- Dell PowerEdge R6525
- Mellanox proprietary Infiniband drivers
Subscriber exclusive content
A Red Hat subscription provides unlimited access to our knowledgebase, tools, and much more.