A hard lockup happens on a CPU on which a task is stuck waiting on qwrlock spinlock

Solution Unverified - Updated -

Issue

  • A hard lockup happened on a CPU on which a task was stuck waiting on qwrlock spinlock:
[486163.609652] Kernel panic - not syncing: Hard LOCKUP
[486163.609790] CPU: 5 PID: 52922 Comm: awk Kdump: loaded Tainted: G           OEL ------------   3.10.0-1160.2.2.el7.x86_64 #1
[486163.610063] Hardware name: Dell Inc. PowerEdge R6525/0GK70M, BIOS 2.10.2 10/25/2022
[486163.610326] Call Trace:
[486163.610454]  <NMI>  [<ffffffff97f813c0>] dump_stack+0x19/0x1b
[486163.610598]  [<ffffffff97f7a918>] panic+0xe8/0x21f
[486163.610734]  [<ffffffff97f93f92>] ? system_call_fastpath+0x25/0x2a
[486163.610876]  [<ffffffff9789af0f>] nmi_panic+0x3f/0x40
[486163.611014]  [<ffffffff9794ede1>] watchdog_overflow_callback+0x121/0x140
[486163.611156]  [<ffffffff979a88a7>] __perf_event_overflow+0x57/0x100
[486163.611298]  [<ffffffff979b20a4>] perf_event_overflow+0x14/0x20
[486163.611439]  [<ffffffff97805595>] x86_pmu_handle_irq+0x125/0x180
[486163.611579]  [<ffffffff97b88358>] ? ioremap_page_range+0x2e8/0x480
[486163.611721]  [<ffffffff97a051b4>] ? vunmap_page_range+0x234/0x470
[486163.611863]  [<ffffffff97a05401>] ? unmap_kernel_range_noflush+0x11/0x20
[486163.612006]  [<ffffffff97c4f78e>] ? ghes_copy_tofrom_phys+0x10e/0x210
[486163.612149]  [<ffffffff97c4f930>] ? ghes_read_estatus+0xa0/0x190
[486163.612289]  [<ffffffff978066f5>] amd_pmu_handle_irq+0x35/0x80
[486163.612430]  [<ffffffff97f8a031>] perf_event_nmi_handler+0x31/0x50
[486163.612571]  [<ffffffff97f8b93c>] nmi_handle.isra.0+0x8c/0x150
[486163.612712]  [<ffffffff97f8bb5d>] do_nmi+0x15d/0x460
[486163.612850]  [<ffffffff97f8ad9c>] end_repeat_nmi+0x1e/0x81
[486163.612996]  [<ffffffff97917812>] ? queued_write_lock_slowpath+0x62/0x90
[486163.613138]  [<ffffffff97917812>] ? queued_write_lock_slowpath+0x62/0x90
[486163.613281]  [<ffffffff97917812>] ? queued_write_lock_slowpath+0x62/0x90
[486163.613424]  <EOE>  [<ffffffff97f89d11>] _raw_qwrite_lock+0x21/0x30
[486163.613569]  [<ffffffff978976bb>] tasklist_write_lock_irq+0x3b/0x50
[486163.613711]  [<ffffffff978a1958>] do_exit+0x328/0xa20
[486163.613849]  [<ffffffff978a20cf>] do_group_exit+0x3f/0xa0
[486163.613988]  [<ffffffff978a2144>] SyS_exit_group+0x14/0x20
[486163.614127]  [<ffffffff97f93f92>] system_call_fastpath+0x25/0x2a

Environment

  • Red Hat Enterprise Linux 7.9.z - kernel-3.10.0-1160.2.2.el7.x86_64
  • Dell PowerEdge R6525
  • Mellanox proprietary Infiniband drivers

Subscriber exclusive content

A Red Hat subscription provides unlimited access to our knowledgebase, tools, and much more.

Current Customers and Partners

Log in for full access

Log In

New to Red Hat?

Learn more about Red Hat subscriptions

Using a Red Hat product through a public cloud?

How to access this content