Hard lockup panic in lpfc_dmp_dbg

Solution Verified - Updated -

Issue

  • Hard lockup panic in lpfc_dmp_dbg
  • During SAN maintenance, the system kernel crashed. vmcores were captured with the following logs:
NMI watchdog: Watchdog detected hard LOCKUP on cpu 4
CPU: 4 PID: 752 Comm: lpfc_worker_0 Kdump: loaded Tainted: G           OE    --------- -  - 4.18.0-348.2.1.el8_5.x86_64 #1
Hardware name: HP ProLiant BL460c Gen9, BIOS I36 04/29/2021
RIP: 0010:native_queued_spin_lock_slowpath+0x59/0x1d0
...
RSP: 0018:ffffab6ccdc17c88 EFLAGS: 00000002
RAX: 0000000000000101 RBX: ffff98fa09740000 RCX: 0000000000000000
RDX: 0000000000000000 RSI: 0000000000000000 RDI: ffff98fa09741940
RBP: ffff98fa09741930 R08: ffffab6ccdc17c48 R09: ffff992266ddc800
R10: 0035a64b9f20310a R11: ffff98fa0974ca9c R12: ffff992266ddc800
R13: ffff98fa09741940 R14: ffffffffc05f9cc8 R15: 0000000000000000
FS:  0000000000000000(0000) GS:ffff9918ffb00000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007ffdb2f0996c CR3: 00000031a0410004 CR4: 00000000001706e0
Call Trace:
 _raw_spin_lock_irq+0x23/0x26
 lpfc_create_vport_work_array+0x53/0xf0 [lpfc]
 lpfc_dmp_dbg+0x24/0x210 [lpfc]
 lpfc_cmpl_els_flogi+0x15b/0x1200 [lpfc]
 lpfc_sli_cancel_iocbs+0xce/0xe0 [lpfc]
 lpfc_els_flush_cmd+0x13a/0x680 [lpfc]
 lpfc_els_flush_all_cmd+0x37/0x60 [lpfc]
 lpfc_sli4_async_event_proc+0x538/0xd40 [lpfc]
 ? lpfc_sli4_post_async_mbox+0x35f/0x450 [lpfc]
 lpfc_do_work+0x10ac/0x1850 [lpfc]
 ? finish_wait+0x80/0x80
 ? lpfc_unregister_unused_fcf+0x80/0x80 [lpfc]
 kthread+0x116/0x130
 ? kthread_flush_work_fn+0x10/0x10
 ret_from_fork+0x35/0x40
Kernel panic - not syncing: Hard LOCKUP
...
RIP: 0010:native_queued_spin_lock_slowpath+0x59/0x1d0
PID: 926      TASK: ffffa06189f02800  CPU: 6    COMMAND: "lpfc_worker_0"
...
    [exception RIP: native_queued_spin_lock_slowpath+0x5d]
...
--- <NMI exception stack> ---
#12 [ffffb6178e3cbcb0] native_queued_spin_lock_slowpath at ffffffffa874e46d
#13 [ffffb6178e3cbcb0] _raw_spin_lock_irqsave at ffffffffa8fa62e0
#14 [ffffb6178e3cbcc0] lpfc_dmp_dbg at ffffffffc08f05d9 [lpfc]
#15 [ffffb6178e3cbcf8] lpfc_cmpl_els_flogi at ffffffffc08e2723 [lpfc]
#16 [ffffb6178e3cbd50] lpfc_sli_cancel_iocbs at ffffffffc08b804e [lpfc]
#17 [ffffb6178e3cbd80] lpfc_els_flush_cmd at ffffffffc08dea5a [lpfc]
#18 [ffffb6178e3cbe18] lpfc_els_flush_all_cmd at ffffffffc08defd7 [lpfc]
#19 [ffffb6178e3cbe38] lpfc_sli4_async_event_proc at ffffffffc08fe011 [lpfc]
#20 [ffffb6178e3cbe88] lpfc_do_work at ffffffffc08ee491 [lpfc]
#21 [ffffb6178e3cbf10] kthread at ffffffffa8712a2a

Environment

  • Red Hat Enterprise Linux 8.5 (4.18.0-348.2.1.el8_5)
  • Red Hat Enterprise Linux 8.6 (4.18.0-372.13.1.el8_6)
  • Emulex SCSI HBA using lpfc driver

Subscriber exclusive content

A Red Hat subscription provides unlimited access to our knowledgebase, tools, and much more.

Current Customers and Partners

Log in for full access

Log In

New to Red Hat?

Learn more about Red Hat subscriptions

Using a Red Hat product through a public cloud?

How to access this content