kernel panic at handle_cmd_completion+0x60a

Issue

• Panic log from RHEL 7:

[1028465.568365] BUG: unable to handle kernel NULL pointer dereference at 0000000000000008
[1028465.568411] IP: [<ffffffff9d5544fa>] handle_cmd_completion+0x60a/0xef0
[1028465.568445] PGD 0 
[1028465.568459] Oops: 0000 [#1] SMP 
[1028465.568479] Modules linked in: macvlan mpt2sas raid_class scsi_transport_sas mptctl mptbase xt_nat veth ipt_MASQUERADE nf_nat_masquerade_ipv4 xt_addrtype br_netfilter dm_thin_pool dm_persistent_data dm_bio_prison dm_bufio loop rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver nfs lockd grace fscache team_mode_roundrobin ip6t_rpfilter ip6t_REJECT nf_reject_ipv6 ipt_REJECT nf_reject_ipv4 xt_conntrack ebtable_nat ebtable_broute bridge 8021q garp mrp stp llc ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 team ip6table_mangle ip6table_security ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat iptable_mangle iptable_security iptable_raw nf_conntrack ip_set nfnetlink ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter dell_rbu skx_edac intel_powerclamp coretemp
[1028465.568907]  intel_rapl iosf_mbi kvm_intel iTCO_wdt iTCO_vendor_support kvm sunrpc irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd dell_smbios dcdbas pcspkr dell_wmi_descriptor vfat fat sg mei_me mei lpc_ich i2c_i801 ipmi_si acpi_power_meter acpi_pad ip_tables xfs libcrc32c nvidia_drm(POE) nvidia_modeset(POE) nvidia_uvm(OE) nvidia(POE) sd_mod sr_mod crc_t10dif cdrom crct10dif_generic mgag200 drm_kms_helper ttm syscopyarea sysfillrect sysimgblt fb_sys_fops ahci drm ixgbe crct10dif_pclmul crct10dif_common crc32c_intel libahci igb libata megaraid_sas mdio ptp pps_core i2c_algo_bit dca drm_panel_orientation_quirks wmi nfit ipmi_devintf ipmi_msghandler libnvdimm dm_mirror dm_region_hash dm_log dm_mod
[1028465.569321] CPU: 0 PID: 0 Comm: swapper/0 Kdump: loaded Tainted: P        W  OE  ------------   3.10.0-1127.18.2.el7.x86_64 #1
[1028465.569372] Hardware name: Dell Inc. PowerEdge R640/0RGP26, BIOS 2.3.10 08/15/2019
[1028465.569407] task: ffffffff9dc18480 ti: ffffffff9dc00000 task.ti: ffffffff9dc00000
[1028465.569442] RIP: 0010:[<ffffffff9d5544fa>]  [<ffffffff9d5544fa>] handle_cmd_completion+0x60a/0xef0
[1028465.569487] RSP: 0018:ffff980fee603d90  EFLAGS: 00010046
[1028465.569512] RAX: 0000000000000000 RBX: ffff980fd11707a0 RCX: 0000000000000006
[1028465.570886] RDX: 0000000000000000 RSI: 0000000000000000 RDI: ffff98107a6ac260
[1028465.572262] RBP: ffff980fee603e20 R08: 000000000000000a R09: 0000000000000000
[1028465.573636] R10: 000000000013e016 R11: ffff980fee603a2e R12: ffff980fec5a3420
[1028465.575013] R13: ffff980fec5a5130 R14: 0000000000000005 R15: ffff98107a6ac260
[1028465.576382] FS:  0000000000000000(0000) GS:ffff980fee600000(0000) knlGS:0000000000000000
[1028465.577754] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[1028465.579125] CR2: 0000000000000008 CR3: 000000029fa10000 CR4: 00000000007607f0
[1028465.580505] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[1028465.581884] DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
[1028465.583251] PKRU: 00000000
[1028465.584606] Call Trace:
[1028465.585946]  <IRQ> 
[1028465.585963]  [<ffffffff9d50ef49>] ? usb_hcd_poll_rh_status+0x129/0x180
[1028465.588626]  [<ffffffff9d5564cc>] xhci_irq+0x39c/0xa50
[1028465.589965]  [<ffffffff9d0ae150>] ? __internal_add_timer+0x130/0x130
[1028465.591310]  [<ffffffff9d556b91>] xhci_msi_irq+0x11/0x20
[1028465.592651]  [<ffffffff9d150764>] __handle_irq_event_percpu+0x44/0x1c0
[1028465.593993]  [<ffffffff9d150912>] handle_irq_event_percpu+0x32/0x80
[1028465.595339]  [<ffffffff9d15099c>] handle_irq_event+0x3c/0x60
[1028465.596671]  [<ffffffff9d15378f>] handle_edge_irq+0x7f/0x150
[1028465.597992]  [<ffffffff9d02f5f4>] handle_irq+0xe4/0x1a0
[1028465.599325]  [<ffffffff9d0ded2e>] ? sched_clock_idle_wakeup_event+0x1e/0x20
[1028465.600664]  [<ffffffff9d1118ad>] ? tick_check_idle+0xbd/0xd0
[1028465.602001]  [<ffffffff9d79786d>] do_IRQ+0x4d/0xf0
[1028465.603320]  [<ffffffff9d78936a>] common_interrupt+0x16a/0x16a
[1028465.604628]  <EOI> 
[1028465.604644]  [<ffffffff9d5c61e7>] ? cpuidle_enter_state+0x57/0xd0
[1028465.607241]  [<ffffffff9d5c61dd>] ? cpuidle_enter_state+0x4d/0xd0
[1028465.608525]  [<ffffffff9d5c633e>] cpuidle_idle_call+0xde/0x230
[1028465.609774]  [<ffffffff9d037c8e>] arch_cpu_idle+0xe/0xc0
[1028465.610993]  [<ffffffff9d101c7a>] cpu_startup_entry+0x14a/0x1e0
[1028465.612188]  [<ffffffff9d76e6a7>] rest_init+0x77/0x80
[1028465.613350]  [<ffffffff9dd8b1cf>] start_kernel+0x44b/0x46c
[1028465.614487]  [<ffffffff9dd8ab84>] ? repair_env_string+0x5c/0x5c
[1028465.615594]  [<ffffffff9dd8a120>] ? early_idt_handler_array+0x120/0x120
[1028465.616675]  [<ffffffff9dd8a738>] x86_64_start_reservations+0x24/0x26
[1028465.617726]  [<ffffffff9dd8a88e>] x86_64_start_kernel+0x154/0x177
[1028465.618744]  [<ffffffff9d0000d5>] start_cpu+0x5/0x14
[1028465.619725] Code: 08 00 00 89 c8 25 00 00 1f 00 4c 89 ff c1 e8 10 83 e8 01 89 c2 89 45 90 48 63 45 a8 49 8d 04 c7 48 89 45 98 48 8b 80 c8 01 00 00 <48> 8b 70 08 e8 ad 84 ff ff 49 89 c4 0f 1f 44 00 00 31 c0 44 89 
[1028465.621827] RIP  [<ffffffff9d5544fa>] handle_cmd_completion+0x60a/0xef0
[1028465.622800]  RSP <ffff980fee603d90>
[1028465.623737] CR2: 0000000000000008

• Panic log from RHEL 8:

[14228151.832142] usb 1-14.1: New USB device strings: Mfr=0, Product=0, SerialNumber=0
[14228151.832910] hub 1-14.1:1.0: USB hub found
[14228151.833081] hub 1-14.1:1.0: 4 ports detected
[14228151.898008] usb 1-14.4: new high-speed USB device number 101 using xhci_hcd
[14228151.983885] usb 1-14.4: Device not responding to setup address.
[14228151.983915] xhci_hcd 0000:00:14.0: WARN Event TRB for slot 1 ep 0 with no TDs queued?
[14228151.983934] usb 1-14: USB disconnect, device number 99
[14228151.983967] BUG: unable to handle kernel NULL pointer dereference at 0000000000000008
[14228151.992055] PGD 0 P4D 0 
[14228151.994853] Oops: 0000 [#1] SMP NOPTI
[14228151.998780] CPU: 36 PID: 0 Comm: swapper/36 Kdump: loaded Tainted: G          I      --------- -  - 4.18.0-305.76.1.el8
_4.x86_64 #1
[14228152.010851] Hardware name: Dell Inc. PowerEdge R640/0HG0J8, BIOS 2.15.1 06/15/2022
[14228152.018684] RIP: 0010:handle_cmd_completion+0x8d3/0x1130
[14228152.024259] Code: 24 08 0f 85 46 06 00 00 8b 44 24 08 41 c1 ed 10 48 89 df 41 83 e5 1f 41 8d 55 ff 4c 8d 2c c3 49 8b 85
 00 02 00 00 89 54 24 18 <48> 8b 70 08 e8 44 7f ff ff 48 89 c1 0f 1f 44 00 00 89 e9 48 c7 c2
[14228152.043264] RSP: 0000:ffffb9020d3c8cd8 EFLAGS: 00010002
[14228152.048748] RAX: 0000000000000000 RBX: ffff9450166842c0 RCX: 0000000000000001
[14228152.056143] RDX: 0000000000000000 RSI: 0000000000000000 RDI: ffff9450166842c0
[14228152.063533] RBP: 000000000000000b R08: ffff945020c9b2d0 R09: 0000000000000000
[14228152.070926] R10: 0000000000000000 R11: 00000000158d51f6 R12: ffff94501e2f2a70
[14228152.078320] R13: ffff9450166842c8 R14: ffff9450162fad70 R15: ffff944d8696b420
[14228152.085711] FS:  0000000000000000(0000) GS:ffff945020c80000(0000) knlGS:0000000000000000
[14228152.094057] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[14228152.100063] CR2: 0000000000000008 CR3: 0000002de1410003 CR4: 00000000007726e0
[14228152.107454] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[14228152.114846] DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
[14228152.122240] PKRU: 55555554
[14228152.125212] Call Trace:
[14228152.127926]  <IRQ>
[14228152.130211]  ? queue_command+0x102/0x220
[14228152.134405]  ? update_load_avg+0x7e/0x610
[14228152.138683]  ? account_entity_enqueue+0x9c/0xd0
[14228152.143476]  xhci_irq+0x1ec/0x430
[14228152.147058]  __handle_irq_event_percpu+0x40/0x180
[14228152.156733]  handle_irq_event+0x36/0x53
[14228152.160833]  handle_edge_irq+0x82/0x190
[14228152.164938]  handle_irq+0x1c/0x30
[14228152.168522]  do_IRQ+0x49/0xd0
[14228152.171752]  common_interrupt+0xf/0xf
[14228152.175678] RIP: 0010:__do_softirq+0x72/0x2d6
[14228152.180298] Code: 67 51 ff 65 81 05 b2 5b 41 79 00 01 00 00 c7 44 24 10 0a 00 00 00 48 c7 c0 00 97 02 00 65 66 c7 00 00 00 fb 66 0f 1f 44 00 00 <b8> ff ff ff ff 48 c7 c3 00 61 60 87 41 0f bc c7 89 c6 83 c6 01 89
[14228152.199300] RSP: 0000:ffffb9020d3c8f68 EFLAGS: 00000286 ORIG_RAX: ffffffffffffff2b
[14228152.207126] RAX: 0000000000029700 RBX: 00000000000000eb RCX: 000000000000001f
[14228152.214518] RDX: 0000000000000044 RSI: 000000003158af9d RDI: ffb96c767e86db3e
[14228152.221910] RBP: ffffb9020ca1fdb8 R08: 0000000000000002 R09: 0000000000029780
[14228152.229302] R10: 013a2fc1227b5cda R11: 0000000000000000 R12: ffff9450159c0600
[14228152.236695] R13: 00000000000000eb R14: 0000000000000000 R15: 0000000000000008
[14228152.244090]  ? common_interrupt+0xa/0xf
[14228152.248188]  ? __do_softirq+0x4b/0x2d6
[14228152.252203]  irq_exit+0xf7/0x100
[14228152.255696]  do_IRQ+0x7f/0xd0
[14228152.258926]  common_interrupt+0xf/0xf
[14228152.262853]  </IRQ>
[14228152.265222] RIP: 0010:cpuidle_enter_state+0xd9/0x3c0
[14228152.270443] Code: e8 6c 5a 9f ff 80 7c 24 07 00 74 17 9c 58 0f 1f 44 00 00 f6 c4 02 0f 85 97 02 00 00 31 ff e8 0e c6 a5 ff fb 66 0f 1f 44 00 00 <45> 85 ed 0f 88 23 01 00 00 49 63 d5 48 2b 5c 24 08 48 8d 04 d5 00
[14228152.289447] RSP: 0000:ffffb9020ca1fe68 EFLAGS: 00000246 ORIG_RAX: ffffffffffffff14
[14228152.297273] RAX: ffff945020ca9f40 RBX: 00328c6dc8d053b8 RCX: 000000000000001f
[14228152.304666] RDX: 00328c6dc8d053b8 RSI: 000000003158af9d RDI: 0000000000000000
[14228152.312058] RBP: ffffd8ea00c80390 R08: 0000000000000002 R09: 0000000000029780
[14228152.319452] R10: 013a2fc1227b54a2 R11: ffff945020ca8c64 R12: ffffffff87730ba0
[14228152.326842] R13: 0000000000000001 R14: 0000000000000001 R15: 0000000000000001
[14228152.334239]  ? cpuidle_enter_state+0xb4/0x3c0
[14228152.338854]  cpuidle_enter+0x2c/0x40
[14228152.342697]  do_idle+0x234/0x260
[14228152.346188]  cpu_startup_entry+0x6f/0x80
[14228152.350373]  start_secondary+0x198/0x1e0
[14228152.354562]  secondary_startup_64_no_verify+0xc2/0xcb
[14228152.359872] Modules linked in: joydev vhost_net vhost vhost_iotlb tap tun tcp_diag inet_diag xt_CT ipt_rpfilter ip_set_hash_ip ip_set_hash_net veth nf_conntrack_netlink xt_addrtype xt_set ip_set_hash_ipportip ip_set_hash_ipportnet ip_set_hash_ipport ip_set_bitmap_port ip_set dummy ip6t_MASQUERADE ipt_MASQUERADE xt_conntrack xt_comment xt_mark nft_chain_nat nft_counter xt_multiport nft_compat nf_tables overlay 8021q garp mrp bonding intel_rapl_msr iTCO_wdt iTCO_vendor_support wmi_bmof dell_smbios dell_wmi_descriptor dcdbas intel_rapl_common isst_if_common skx_edac nfit libnvdimm x86_pkg_temp_thermal intel_powerclamp coretemp kvm_intel kvm crct10dif_pclmul crc32_pclmul i40iw ghash_clmulni_intel rapl intel_cstate intel_uncore ib_uverbs snd_pcm snd_timer ib_core snd soundcore pcspkr ipmi_ssif lpc_ich i2c_i801 mei_me mei wmi acpi_ipmi ipmi_si ipmi_devintf ipmi_msghandler acpi_power_meter vxlan ip6_udp_tunnel udp_tunnel nfnetlink_cttimeout nfnetlink openvswitch nf_conncount nf_nat vfat fat ext4
[14228152.359900]  mbcache jbd2 auth_rpcgss sunrpc xfs sd_mod t10_pi sg mgag200 drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops i40e drm ahci libahci megaraid_sas i2c_algo_bit tg3 libata dm_mirror dm_region_hash dm_log dm_mod vfio_pci irqbypass vfio_virqfd vfio_iommu_type1 vfio scsi_transport_fc br_netfilter bridge stp llc ip_vs_sh ip_vs_wrr ip_vs_rr ip_vs nf_conntrack nf_defrag_ipv6 nf_defrag_ipv4 libcrc32c crc32c_intel fuse
[14228152.486061] Features: eBPF/xdp
[14228152.489380] CR2: 0000000000000008