BUG_ON in blk_requeue_request(), race when aborting tasks in mptscsih

Solution Verified - Updated -

Issue

System crashes over BUG_ON(lk_queued_rq(rq)); in blk_requeue_request().

This is preceded with a frantic task abort activity shortly before the panic for the same device as seen in the message buffer:

...
[6556920.107225] mptscsih: ioc0: attempting task abort! (sc=ffff986341de4380)
[6556920.107227] sd 0:0:0:0: [sda] CDB: Write(10) 2a 00 03 87 e9 30 00 00 08 00
[6556920.107229] mptscsih: ioc0: task abort: SUCCESS (rv=2002) (sc=ffff986341de4380)
[6556920.107233] mptscsih: ioc0: attempting task abort! (sc=ffff9863832108c0)
[6556920.107235] sd 0:0:0:0: [sda] CDB: Read(10) 28 00 03 ce be 78 00 00 50 00
[6556920.107237] mptscsih: ioc0: task abort: SUCCESS (rv=2002) (sc=ffff9863832108c0)
[6556920.107242] mptscsih: ioc0: attempting task abort! (sc=ffff98634f9fc000)
[6556920.107244] sd 0:0:0:0: [sda] CDB: Read(10) 28 00 03 ce be 58 00 00 18 00
[6556920.107246] mptscsih: ioc0: task abort: SUCCESS (rv=2002) (sc=ffff98634f9fc000)
[6556920.107288] ------------[ cut here ]------------
[6556920.107319] kernel BUG at block/blk-core.c:1559!
[6556920.107871] invalid opcode: 0000 [#1] SMP 
[6556920.108291] Modules linked in: fuse btrfs raid6_pq xor vfat msdos fat xfs libcrc32c bridge stp llc iptable_filter nfsv3 nfs fscache vmw_vsock_vmci_transport vsock sb_edac iosf_mbi crc32_pclmul ghash_clmulni_intel aesni_intel ppdev lrw gf128mul glue_helper ablk_helper cryptd vmw_balloon joydev pcspkr sg vmw_vmci i2c_piix4 parport_pc parport nfsd auth_rpcgss nfs_acl lockd grace sunrpc binfmt_misc ip_tables ext4 mbcache jbd2 sr_mod sd_mod cdrom crc_t10dif crct10dif_generic ata_generic pata_acpi vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm drm ata_piix crct10dif_pclmul crct10dif_common crc32c_intel mptspi libata scsi_transport_spi serio_raw mptscsih mptbase pcnet32 mii drm_panel_orientation_quirks floppy dm_mirror dm_region_hash dm_log dm_mod
[6556920.111985] CPU: 1 PID: 2471 Comm: kworker/u64:2 Kdump: loaded Tainted: G             L ------------   3.10.0-1160.41.1.el7.x86_64 #1
[6556920.112983] Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 09/21/2015
[6556920.114045] Workqueue: scsi_tmf_0 scmd_eh_abort_handler
[6556920.114613] task: ffff98733db51080 ti: ffff98635ff30000 task.ti: ffff98635ff30000
[6556920.115190] RIP: 0010:[<ffffffff90f539d0>]  [<ffffffff90f539d0>] blk_requeue_request+0x90/0xa0
[6556920.115783] RSP: 0000:ffff98635ff33d90  EFLAGS: 00010083
[6556920.116371] RAX: ffff9863414482d0 RBX: ffff98634f9fc000 RCX: dead000000000200
[6556920.116981] RDX: ffff9863414482d0 RSI: ffff986341448180 RDI: ffff9863414482d0
[6556920.117579] RBP: ffff98635ff33da8 R08: ffff9863414482d0 R09: 0000000000000029
[6556920.118180] R10: 000000000000124d R11: 3063663966343336 R12: ffff986341448180
[6556920.118804] R13: ffff987339230000 R14: 0000000000000246 R15: 0000000000001057
[6556920.119424] FS:  0000000000000000(0000) GS:ffff98733ee40000(0000) knlGS:0000000000000000
[6556920.120074] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[6556920.120723] CR2: 00000000004001a0 CR3: 0000000ffd4c8000 CR4: 00000000000407e0
[6556920.121522] Call Trace:
[6556920.122210]  [<ffffffff910ed8f6>] __scsi_queue_insert+0xb6/0x100
[6556920.122949]  [<ffffffff910ed953>] scsi_queue_insert+0x13/0x20
[6556920.123686]  [<ffffffff910e9722>] scmd_eh_abort_handler+0x1b2/0x2a0
[6556920.124416]  [<ffffffff90cbde8f>] process_one_work+0x17f/0x440
[6556920.125163]  [<ffffffff90cbefa6>] worker_thread+0x126/0x3c0
[6556920.125859]  [<ffffffff90cbee80>] ? manage_workers.isra.26+0x2a0/0x2a0
[6556920.126563]  [<ffffffff90cc5e61>] kthread+0xd1/0xe0
[6556920.127279]  [<ffffffff90cc5d90>] ? insert_kthread_work+0x40/0x40
[6556920.128003]  [<ffffffff91395df7>] ret_from_fork_nospec_begin+0x21/0x21
[6556920.128705]  [<ffffffff90cc5d90>] ? insert_kthread_work+0x40/0x40
[6556920.129412] Code: 83 c3 10 4c 89 e2 4c 89 ee e8 7d 45 04 00 48 8b 03 48 85 c0 75 e5 41 f6 44 24 4a 10 74 ad 4c 89 e6 4c 89 ef e8 d2 43 00 00 eb a0 <0f> 0b 0f 1f 40 00 66 2e 0f 1f 84 00 00 00 00 00 0f 1f 44 00 00 
[6556920.131641] RIP  [<ffffffff90f539d0>] blk_requeue_request+0x90/0xa0
[6556920.132349]  RSP <ffff98635ff33d90>

Environment

  • Red Hat Enetrprise Linux 7.9.z
  • kernel-3.10.0-1160.41.1.el7
  • The OS instance is in a Virtual Machine on top of a VMware hypervisor

Subscriber exclusive content

A Red Hat subscription provides unlimited access to our knowledgebase, tools, and much more.

Current Customers and Partners

Log in for full access

Log In

New to Red Hat?

Learn more about Red Hat subscriptions

Using a Red Hat product through a public cloud?

How to access this content