[RHEL 7.9]Crash in lpfc_bsg_issue_mbox_ext_handle_job() when Broadcom/Emulex provided lpfc driver is present

Solution Unverified - Updated -

Issue

System crashes with panic when NULL pointer is dereferenced in lpfc_bsg_issue_mbox_ext_handle_job() function with the following kernel stack trace reported in the console messages:

[169326.958481] BUG: unable to handle kernel NULL pointer dereference at 0000000000000010
[169326.958501] IP: [<ffffffffc0c65909>] lpfc_bsg_issue_mbox_ext_handle_job+0xb9/0x280 [lpfc]
[169326.958504] PGD 80000001afb88067 PUD 1bd3a0067 PMD 0 
[169326.958505] Oops: 0000 [#1] SMP 
[169326.958558] Modules linked in: lin_tape(OE) pfo(OE) dm_service_time ch osst st mmfs26(OE) mmfslinux(OE) tracedev(OE) lpfc(OE) nvmet_fc nvmet bonding sunrpc skx_edac intel_powerclamp coretemp intel_rapl iosf_mbi kvm_intel kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd pcspkr ses enclosure scsi_transport_sas sg mei_me lpc_ich joydev mei wmi ipmi_si ipmi_devintf ipmi_msghandler tpm_crb acpi_power_meter acpi_pad dm_multipath binfmt_misc ip_tables ext4 mbcache jbd2 sd_mod crc_t10dif crct10dif_generic mgag200 i2c_algo_bit drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm crct10dif_pclmul crct10dif_common fnic(OE) crc32c_intel drm ixgbe nvme_fc ahci nvme_fabrics nvme_core libahci scsi_transport_fc libata megaraid_sas mdio scsi_tgt enic(OE)
[169326.958564]  ptp pps_core dca drm_panel_orientation_quirks nfit libnvdimm dm_mirror dm_region_hash dm_log dm_mod [last unloaded: pfo]
[169326.958567] CPU: 1 PID: 4137 Comm: lpfc_worker_2 Kdump: loaded Tainted: G           OE  ------------   3.10.0-1160.11.1.el7.x86_64 #1
[169326.958568] Hardware name: Cisco Systems Inc UCSC-C240-M5SX/UCSC-C240-M5SX, BIOS C240M5.4.1.3f.0.0502211336 05/02/2021
[169326.958569] task: ffff9c9a3078d280 ti: ffff9c3a08b8c000 task.ti: ffff9c3a08b8c000
[169326.958578] RIP: 0010:[<ffffffffc0c65909>]  [<ffffffffc0c65909>] lpfc_bsg_issue_mbox_ext_handle_job+0xb9/0x280 [lpfc]
[169326.958580] RSP: 0018:ffff9c3a08b8fc30  EFLAGS: 00010246
[169326.958581] RAX: 0000000000000000 RBX: ffff9c9a6ef8c000 RCX: 0000000000000000
[169326.958581] RDX: 0000000000000001 RSI: ffff9c3f45ae8110 RDI: ffff9cf4b8131100
[169326.958582] RBP: ffff9c3a08b8fc68 R08: 0000000000000000 R09: 00000000000000e8
[169326.958583] R10: ffff9cf546d71000 R11: 0000000000000200 R12: 0000000000000000
[169326.958584] R13: ffff9c3ec6ecde00 R14: ffff9c3f45ae8000 R15: ffff9cf4b8131000
[169326.958585] FS:  0000000000000000(0000) GS:ffff9c95ffe40000(0000) knlGS:0000000000000000
[169326.958587] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[169326.958587] CR2: 0000000000000010 CR3: 00000001b83ee000 CR4: 00000000007607e0
[169326.958589] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[169326.958590] DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
[169326.958593] Call Trace:
[169326.958602]  [<ffffffffc0be212d>] ? lpfc_sli_free_hbq+0x2d/0x70 [lpfc]
[169326.958609]  [<ffffffffc0c67d79>] lpfc_bsg_issue_read_mbox_ext_cmpl+0x19/0x110 [lpfc]
[169326.958617]  [<ffffffffc0be22de>] lpfc_sli_handle_mb_event+0x16e/0x4d0 [lpfc]
[169326.958625]  [<ffffffff854adaa3>] ? internal_add_timer+0x83/0xe0
[169326.958629]  [<ffffffff854aed9c>] ? mod_timer+0x10c/0x230
[169326.958636]  [<ffffffffc0be5558>] ? lpfc_sli4_post_async_mbox+0x128/0x420 [lpfc]
[169326.958645]  [<ffffffffc0c1b480>] lpfc_work_done+0xa50/0x1570 [lpfc]
[169326.958651]  [<ffffffff85b868ef>] ? __schedule+0x3af/0x860
[169326.958659]  [<ffffffffc0c1c010>] lpfc_do_work+0x70/0x1e0 [lpfc]
[169326.958664]  [<ffffffff854c6f60>] ? wake_up_atomic_t+0x30/0x30
[169326.958671]  [<ffffffffc0c1bfa0>] ? lpfc_work_done+0x1570/0x1570 [lpfc]
[169326.958673]  [<ffffffff854c5e71>] kthread+0xd1/0xe0
[169326.958676]  [<ffffffff854c5da0>] ? insert_kthread_work+0x40/0x40
[169326.958679]  [<ffffffff85b93ddd>] ret_from_fork_nospec_begin+0x7/0x21
[169326.958681]  [<ffffffff854c5da0>] ? insert_kthread_work+0x40/0x40
[169326.958704] Code: 16 b1 04 89 17 41 f6 c0 02 74 0c 0f b7 14 0e 66 89 14 0f 48 83 c1 02 41 83 e0 01 74 07 0f b6 14 0e 88 14 0f 48 8b 83 08 07 00 00 <48> 8b 40 10 f6 40 04 01 0f 84 e9 00 00 00 4d 85 e4 0f 84 fb 00 
[169326.958711] RIP  [<ffffffffc0c65909>] lpfc_bsg_issue_mbox_ext_handle_job+0xb9/0x280 [lpfc]
[169326.958711]  RSP <ffff9c3a08b8fc30>
[169326.958712] CR2: 0000000000000010

Environment

  • Red Hat Enterprise Linux 7.9.z
  • kernel 3.10.0-1160.11.1.el7.x86_64
  • tainted kernel with Broadcom / Emulex provided lpfc driver version 12.6.240.48-1

The lpfc related packages for the above version are:

elx-lpfc-extras-12.6.240.48-1.rhel7.noarch  Wed Sep 15 16:28:21 2021    1631716101  Broadcom Inc. or its subsidiaries   scmecagrh7l645.lvn.broadcom.net (none)  (none)
kmod-elx-lpfc-12.6.240.48-1.rhel7u9.x86_64  Wed Sep 15 16:18:45 2021    1631715525  Broadcom Inc. or its subsidiaries   scmecagrh7l645  (none)  (none)

Subscriber exclusive content

A Red Hat subscription provides unlimited access to our knowledgebase, tools, and much more.

Current Customers and Partners

Log in for full access

Log In

New to Red Hat?

Learn more about Red Hat subscriptions

Using a Red Hat product through a public cloud?

How to access this content