System crash in blk_mq_free_request() following NVMe controller reset due to NULL mq_hctx

Solution Unverified - Updated -

Issue

  • System crash in blk_mq_free_request() following NVMe controller reset due to NULL mq_hctx. Example 1:
nvme nvme0: I/O 71 QID 1 timeout, aborting
nvme nvme0: Abort status: 0x0
nvme nvme0: I/O 71 QID 1 timeout, reset controller
nvme nvme0: 2/0/0 default/read/poll queues
BUG: unable to handle kernel NULL pointer dereference at 0000000000000108
PGD 0 P4D 0 
Oops: 0000 [#1] SMP PTI
CPU: 0 PID: 0 Comm: swapper/0 Kdump: loaded Not tainted 4.18.0-348.2.1.el8_5.x86_64 #1
Hardware name: VMware, Inc. VMware7,1/440BX Desktop Reference Platform, BIOS VMW71.00V.18227214.B64.2106252220 06/25/2021
RIP: 0010:blk_mq_free_request+0xbc/0x120
....
Call Trace:
<IRQ>
nvme_irq+0x127/0x240 [nvme]
__handle_irq_event_percpu+0x40/0x180
handle_irq_event_percpu+0x30/0x80
handle_irq_event+0x36/0x53
handle_edge_irq+0x82/0x190
handle_irq+0x1c/0x30
do_IRQ+0x49/0xd0
common_interrupt+0xf/0xf
....
  • Example 2:
nvme nvme0: I/O 197 QID 1 timeout, aborting
nvme nvme0: Abort status: 0x0
nvme nvme0: I/O 197 QID 1 timeout, reset controller
nvme nvme0: 4/0/0 default/read/poll queues
nvme nvme0: I/O 81 QID 2 timeout, aborting
nvme nvme0: Abort status: 0x0
nvme nvme0: I/O 81 QID 2 timeout, reset controller
nvme nvme0: 4/0/0 default/read/poll queues
BUG: unable to handle kernel NULL pointer dereference at 0000000000000000
PGD 0 P4D 0 
Oops: 0002 [#1] SMP NOPTI
CPU: 1 PID: 0 Comm: swapper/1 Kdump: loaded Tainted: G           OE    --------- -  - 4.18.0-193.el8.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 12/12/2018
RIP: 0010:dma_pool_free+0x92/0xb0
....
Call Trace:
<IRQ>
nvme_unmap_data+0x113/0x1f0 [nvme]
nvme_pci_complete_rq+0x30/0x40 [nvme]
blk_mq_complete_request+0x6c/0x100
nvme_irq+0xfc/0x240 [nvme]
__handle_irq_event_percpu+0x40/0x180
handle_irq_event_percpu+0x30/0x80
handle_irq_event+0x36/0x53
handle_edge_irq+0x82/0x190
handle_irq+0xbf/0x100
do_IRQ+0x49/0xe0
common_interrupt+0xf/0xf

Environment

  • Red Hat Enterprise Linux 8

Subscriber exclusive content

A Red Hat subscription provides unlimited access to our knowledgebase, tools, and much more.

Current Customers and Partners

Log in for full access

Log In

New to Red Hat?

Learn more about Red Hat subscriptions

Using a Red Hat product through a public cloud?

How to access this content