BUG_ON in blk_requeue_request(), race when aborting tasks in mptscsih
Issue
System crashes over BUG_ON(lk_queued_rq(rq)); in blk_requeue_request().
This is preceded with a frantic task abort activity shortly before the panic for the same device as seen in the message buffer:
...
[6556920.107225] mptscsih: ioc0: attempting task abort! (sc=ffff986341de4380)
[6556920.107227] sd 0:0:0:0: [sda] CDB: Write(10) 2a 00 03 87 e9 30 00 00 08 00
[6556920.107229] mptscsih: ioc0: task abort: SUCCESS (rv=2002) (sc=ffff986341de4380)
[6556920.107233] mptscsih: ioc0: attempting task abort! (sc=ffff9863832108c0)
[6556920.107235] sd 0:0:0:0: [sda] CDB: Read(10) 28 00 03 ce be 78 00 00 50 00
[6556920.107237] mptscsih: ioc0: task abort: SUCCESS (rv=2002) (sc=ffff9863832108c0)
[6556920.107242] mptscsih: ioc0: attempting task abort! (sc=ffff98634f9fc000)
[6556920.107244] sd 0:0:0:0: [sda] CDB: Read(10) 28 00 03 ce be 58 00 00 18 00
[6556920.107246] mptscsih: ioc0: task abort: SUCCESS (rv=2002) (sc=ffff98634f9fc000)
[6556920.107288] ------------[ cut here ]------------
[6556920.107319] kernel BUG at block/blk-core.c:1559!
[6556920.107871] invalid opcode: 0000 [#1] SMP
[6556920.108291] Modules linked in: fuse btrfs raid6_pq xor vfat msdos fat xfs libcrc32c bridge stp llc iptable_filter nfsv3 nfs fscache vmw_vsock_vmci_transport vsock sb_edac iosf_mbi crc32_pclmul ghash_clmulni_intel aesni_intel ppdev lrw gf128mul glue_helper ablk_helper cryptd vmw_balloon joydev pcspkr sg vmw_vmci i2c_piix4 parport_pc parport nfsd auth_rpcgss nfs_acl lockd grace sunrpc binfmt_misc ip_tables ext4 mbcache jbd2 sr_mod sd_mod cdrom crc_t10dif crct10dif_generic ata_generic pata_acpi vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm drm ata_piix crct10dif_pclmul crct10dif_common crc32c_intel mptspi libata scsi_transport_spi serio_raw mptscsih mptbase pcnet32 mii drm_panel_orientation_quirks floppy dm_mirror dm_region_hash dm_log dm_mod
[6556920.111985] CPU: 1 PID: 2471 Comm: kworker/u64:2 Kdump: loaded Tainted: G L ------------ 3.10.0-1160.41.1.el7.x86_64 #1
[6556920.112983] Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 09/21/2015
[6556920.114045] Workqueue: scsi_tmf_0 scmd_eh_abort_handler
[6556920.114613] task: ffff98733db51080 ti: ffff98635ff30000 task.ti: ffff98635ff30000
[6556920.115190] RIP: 0010:[<ffffffff90f539d0>] [<ffffffff90f539d0>] blk_requeue_request+0x90/0xa0
[6556920.115783] RSP: 0000:ffff98635ff33d90 EFLAGS: 00010083
[6556920.116371] RAX: ffff9863414482d0 RBX: ffff98634f9fc000 RCX: dead000000000200
[6556920.116981] RDX: ffff9863414482d0 RSI: ffff986341448180 RDI: ffff9863414482d0
[6556920.117579] RBP: ffff98635ff33da8 R08: ffff9863414482d0 R09: 0000000000000029
[6556920.118180] R10: 000000000000124d R11: 3063663966343336 R12: ffff986341448180
[6556920.118804] R13: ffff987339230000 R14: 0000000000000246 R15: 0000000000001057
[6556920.119424] FS: 0000000000000000(0000) GS:ffff98733ee40000(0000) knlGS:0000000000000000
[6556920.120074] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[6556920.120723] CR2: 00000000004001a0 CR3: 0000000ffd4c8000 CR4: 00000000000407e0
[6556920.121522] Call Trace:
[6556920.122210] [<ffffffff910ed8f6>] __scsi_queue_insert+0xb6/0x100
[6556920.122949] [<ffffffff910ed953>] scsi_queue_insert+0x13/0x20
[6556920.123686] [<ffffffff910e9722>] scmd_eh_abort_handler+0x1b2/0x2a0
[6556920.124416] [<ffffffff90cbde8f>] process_one_work+0x17f/0x440
[6556920.125163] [<ffffffff90cbefa6>] worker_thread+0x126/0x3c0
[6556920.125859] [<ffffffff90cbee80>] ? manage_workers.isra.26+0x2a0/0x2a0
[6556920.126563] [<ffffffff90cc5e61>] kthread+0xd1/0xe0
[6556920.127279] [<ffffffff90cc5d90>] ? insert_kthread_work+0x40/0x40
[6556920.128003] [<ffffffff91395df7>] ret_from_fork_nospec_begin+0x21/0x21
[6556920.128705] [<ffffffff90cc5d90>] ? insert_kthread_work+0x40/0x40
[6556920.129412] Code: 83 c3 10 4c 89 e2 4c 89 ee e8 7d 45 04 00 48 8b 03 48 85 c0 75 e5 41 f6 44 24 4a 10 74 ad 4c 89 e6 4c 89 ef e8 d2 43 00 00 eb a0 <0f> 0b 0f 1f 40 00 66 2e 0f 1f 84 00 00 00 00 00 0f 1f 44 00 00
[6556920.131641] RIP [<ffffffff90f539d0>] blk_requeue_request+0x90/0xa0
[6556920.132349] RSP <ffff98635ff33d90>
Environment
- Red Hat Enetrprise Linux 7.9.z
- kernel-3.10.0-1160.41.1.el7
- The OS instance is in a Virtual Machine on top of a VMware hypervisor
Subscriber exclusive content
A Red Hat subscription provides unlimited access to our knowledgebase, tools, and much more.