Why does system panic with emcp module ?

Issue

• After upgrading to kernel 3.10.0-1127.el7.8.2.x86_64 or 3.10.0-1160.62.1.el7.x86_64 the server crashes with PowerPath:

[ 3641.477105] general protection fault: 0000 [#1] SMP 
[ 3641.477129] Modules linked in: team_mode_activebackup 8021q garp mrp stp llc team dell_rbu emcpdm(POE) emcpgpx(POE) emcpmpx(POE) emcp(POE) dell_smbios iTCO_wdt iTCO_vendor_support dell_wmi_descriptor dcdbas skx_edac intel_powerclamp coretemp intel_rapl iosf_mbi kvm_intel kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel ipmi_ssif pcspkr sg i2c_i801 lpc_ich mei_me mei wmi ipmi_si ipmi_devintf ipmi_msghandler acpi_power_meter acpi_pad binfmt_misc ip_tables xfs libcrc32c sd_mod crc_t10dif qla2xxx mgag200 i2c_algo_bit drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm ahci crct10dif_pclmul nvme_fc drm nvme_fabrics libahci nvme_core megaraid_sas i40e libata scsi_transport_fc scsi_tgt ptp pps_core drm_panel_orientation_quirks nfit libnvdimm dm_mirror dm_region_hash dm_log authenc
[ 3641.477364]  cmac zlib wp512 twofish_generic twofish_avx_x86_64 twofish_x86_64_3way twofish_x86_64 twofish_common tea sha512_ssse3 sha512_generic serpent_avx2 serpent_avx_x86_64 serpent_sse2_x86_64 serpent_generic seed salsa20_generic salsa20_x86_64 rmd320 rmd256 rmd160 rmd128 michael_mic md4 khazad ghash_generic gcm fcrypt drbg dm_crypt dm_mod des_generic deflate cts crypto_null crct10dif_generic crct10dif_common crc32c_intel ccm cast6_avx_x86_64 cast6_generic cast_common camellia_generic camellia_aesni_avx2 camellia_aesni_avx_x86_64 ablk_helper cryptd camellia_x86_64 xts lrw gf128mul glue_helper blowfish_generic blowfish_x86_64 blowfish_common arc4 ansi_cprng [last unloaded: emcpioc]
[ 3641.477556] CPU: 24 PID: 0 Comm: swapper/24 Kdump: loaded Tainted: P           OE  ------------   3.10.0-1127.8.2.el7.x86_64 #1
[ 3641.477583] Hardware name: Dell Inc. PowerEdge R740/00WGD1, BIOS 2.6.4 04/09/2020
[ 3641.477602] task: ffff990a1d6d62a0 ti: ffff990a1d6d8000 task.ti: ffff990a1d6d8000
[ 3641.477622] RIP: 0010:[<ffffffff9e37da10>]  [<ffffffff9e37da10>] __blk_add_trace+0x20/0x340
[ 3641.477649] RSP: 0018:ffff99c5feb03ba0  EFLAGS: 00010286
[ 3641.477663] RAX: 9000000000000080 RBX: ffff9a85f67814b8 RCX: 0000000000000000
[ 3641.477682] RDX: 00000000a368db00 RSI: ffff9a85d73a8060 RDI: 9000000000000080
[ 3641.477700] RBP: ffff99c5feb03c00 R08: 0000000001800008 R09: 000000000000000f
[ 3641.477718] R10: 9000000000000080 R11: ffff9a85d73a8060 R12: ffff99c2a368db00
[ 3641.477735] R13: ffff990a1d6d62a0 R14: 0000000000000000 R15: 0000000000000000
[ 3641.477753] FS:  0000000000000000(0000) GS:ffff99c5feb00000(0000) knlGS:0000000000000000
[ 3641.477772] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[ 3641.477787] CR2: 0000000000c90000 CR3: 000000b7d6df4000 CR4: 00000000007607e0
[ 3641.477805] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[ 3641.477823] DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
[ 3641.477840] PKRU: 00000000
[ 3641.477847] Call Trace:
[ 3641.477856]  <IRQ> 
[ 3641.477866]  [<ffffffff9e3c1517>] ? mempool_free_slab+0x17/0x20
[ 3641.477884]  [<ffffffff9e3c192f>] ? mempool_free+0x4f/0xa0
[ 3641.477900]  [<ffffffff9e37dd84>] blk_add_trace_rq.isra.10+0x54/0x90
[ 3641.477927]  [<ffffffffc11ead39>] PowerPlatformTopIodone+0x1b9/0x300 [emcp]
[ 3641.477949]  [<ffffffffc11eae9e>] PowerTopIodone+0x1e/0x130 [emcp]
[ 3641.477969]  [<ffffffffc11eb124>] PowerProcessTopIodonePirps+0x64/0xe0 [emcp]
[ 3641.477989]  [<ffffffffc11eb44c>] PowerBottomIodoneNew+0x2ac/0x5d0 [emcp]
[ 3641.478009]  [<ffffffffc11ebb83>] PowerPlatformBottomIodone+0xc3/0x1f0 [emcp]
[ 3641.478030]  [<ffffffff9e48cbec>] bio_endio+0x8c/0x130
[ 3641.478047]  [<ffffffff9e554b10>] blk_update_request+0x90/0x370
[ 3641.478065]  [<ffffffff9e6ec274>] scsi_end_request+0x34/0x1e0
[ 3641.478081]  [<ffffffff9e6ec5e8>] scsi_io_completion+0x168/0x720
[ 3641.478813]  [<ffffffff9e6e18dc>] scsi_finish_command+0xdc/0x140
[ 3641.479471]  [<ffffffff9e6ebb30>] scsi_softirq_done+0x130/0x160
[ 3641.480127]  [<ffffffff9e55c496>] blk_done_softirq+0x96/0xc0
[ 3641.480753]  [<ffffffff9e2a5695>] __do_softirq+0xf5/0x280
[ 3641.481370]  [<ffffffff9e99642c>] call_softirq+0x1c/0x30
[ 3641.481977]  [<ffffffff9e22f715>] do_softirq+0x65/0xa0
[ 3641.482544]  [<ffffffff9e2a5a15>] irq_exit+0x105/0x110
[ 3641.483094]  [<ffffffff9e2596b9>] smp_call_function_single_interrupt+0x39/0x40
[ 3641.483636]  [<ffffffff9e9951aa>] call_function_single_interrupt+0x16a/0x170
[ 3641.484184]  <EOI> 
[ 3641.484195]  [<ffffffff9e7c6177>] ? cpuidle_enter_state+0x57/0xd0
[ 3641.485233]  [<ffffffff9e7c62ce>] cpuidle_idle_call+0xde/0x230
[ 3641.485748]  [<ffffffff9e237c6e>] arch_cpu_idle+0xe/0xc0
[ 3641.486225]  [<ffffffff9e301c2a>] cpu_startup_entry+0x14a/0x1e0
[ 3641.486688]  [<ffffffff9e25a517>] start_secondary+0x1f7/0x270
[ 3641.487155]  [<ffffffff9e2000d5>] start_cpu+0x5/0x14
[ 3641.487598] Code: fe ff ff 0f 1f 84 00 00 00 00 00 55 49 89 fa 49 89 f3 48 89 e5 41 57 41 56 41 55 65 4c 8b 2c 25 c0 0e 01 00 41 54 53 48 83 ec 38 <83> 3f 02 89 55 d0 44 89 4d cc 44 0f b6 3d 36 2c be 00 0f 85 7d 
[ 3641.488553] RIP  [<ffffffff9e37da10>] __blk_add_trace+0x20/0x340
[ 3641.489002]  RSP <ffff99c5feb03ba0>

• On RHEL 6 system panic because of 'emcp' module with following traces:

PID: 8001   TASK: ffff880108bb6aa0  CPU: 0   COMMAND: "oracle"
 #0 [ffff880028207e90] crash_nmi_callback at ffffffff8102d3a6
 #1 [ffff880028207ea0] notifier_call_chain at ffffffff81513705
 #2 [ffff880028207ee0] atomic_notifier_call_chain at ffffffff8151376a
 #3 [ffff880028207ef0] notify_die at ffffffff8109cc1e
 #4 [ffff880028207f20] do_nmi at ffffffff815113cb
 #5 [ffff880028207f50] nmi at ffffffff81510c90
    [exception RIP: _spin_lock_irqsave+0x2f]
    RIP: ffffffff815103ef  RSP: ffff880028203b20  RFLAGS: 00000097
    RAX: 0000000000002c93  RBX: ffff88090b863620  RCX: 0000000000002c92
    RDX: 0000000000000202  RSI: 000000000145b708  RDI: ffff88090ec2fe70
    RBP: ffff880028203b20   R8: 0000000000000001   R9: 0000000000000001
    R10: ffff88090bfe7f80  R11: 0000000000000400  R12: 0000000007800030
    R13: 0000000000000078  R14: 0000000000000000  R15: ffff88090ec2fe70
    ORIG_RAX: ffffffffffffffff  CS: 0010  SS: 0018
--- <NMI exception stack> ---
 #6 [ffff880028203b20] _spin_lock_irqsave at ffffffff815103ef
 #7 [ffff880028203b28] PowerPlatformTopIodone at ffffffffa03876df [emcp]
 #8 [ffff880028203b88] PowerTopIodone at ffffffffa038f1a9 [emcp]
 #9 [ffff880028203bb8] PowerProcessTopIodonePirps at ffffffffa038f347 [emcp]
#10 [ffff880028203bf8] PowerBottomIodoneNew at ffffffffa038f700 [emcp]
#11 [ffff880028203c48] PowerPlatformBottomIodone at ffffffffa038fbf7 [emcp]
#12 [ffff880028203c98] bio_endio at ffffffff811b92cd
#13 [ffff880028203ca8] req_bio_endio at ffffffff8125ad2b
#14 [ffff880028203cd8] blk_update_request at ffffffff8125c7d7
#15 [ffff880028203d38] blk_update_bidi_request at ffffffff8125cb87
#16 [ffff880028203d68] blk_end_bidi_request at ffffffff8125dfcf
#17 [ffff880028203d98] blk_end_request at ffffffff8125e070
#18 [ffff880028203da8] scsi_io_completion at ffffffff8137aa1f
#19 [ffff880028203e28] scsi_finish_command at ffffffff81371a52
#20 [ffff880028203e68] scsi_softirq_done at ffffffff8137b195
#21 [ffff880028203e98] blk_done_softirq at ffffffff81263715
#22 [ffff880028203ec8] __do_softirq at ffffffff81076fb1
#23 [ffff880028203f38] call_softirq at ffffffff8100c1cc
#24 [ffff880028203f50] do_softirq at ffffffff8100de05
#25 [ffff880028203f70] irq_exit at ffffffff81076d95
#26 [ffff880028203f80] do_IRQ at ffffffff815171c5

or

PID: 21461  TASK: ffff880ffbaa2040  CPU: 13  COMMAND: "oracle"
 #0 [ffff88085c4a7e90] crash_nmi_callback at ffffffff810297e6
 #1 [ffff88085c4a7ea0] notifier_call_chain at ffffffff814efc75
 #2 [ffff88085c4a7ee0] atomic_notifier_call_chain at ffffffff814efcda
 #3 [ffff88085c4a7ef0] notify_die at ffffffff81096c3e
 #4 [ffff88085c4a7f20] do_nmi at ffffffff814ed8f3
 #5 [ffff88085c4a7f50] nmi at ffffffff814ed200
    [exception RIP: _spin_lock_irq+40]
    RIP: ffffffff814ec998  RSP: ffff880fa7a0d8f8  RFLAGS: 00000093
    RAX: 0000000000007bff  RBX: ffff881025c86f68  RCX: ffff880b8e68e4c0
    RDX: 0000000000007bfd  RSI: ffff880fa7a0d920  RDI: ffff881028a99c70
    RBP: ffff880fa7a0d8f8   R8: 0000000000000246   R9: 0000000000000000
    R10: 0000000000000000  R11: 0000000000000000  R12: 0000000000000000
    R13: 0000000000000005  R14: 0000000000004000  R15: 0000000007800060
    ORIG_RAX: ffffffffffffffff  CS: 0010  SS: 0018
--- <NMI exception stack> ---
 #6 [ffff880fa7a0d8f8] _spin_lock_irq at ffffffff814ec998
 #7 [ffff880fa7a0d900] __make_request at ffffffff81250390
 #8 [ffff880fa7a0d980] emcp_mrf at ffffffffa026947d [emcp]
 #9 [ffff880fa7a0d9a0] generic_make_request at ffffffff8124ea7e
#10 [ffff880fa7a0da70] submit_bio at ffffffff8124eddd
#11 [ffff880fa7a0dac0] dio_bio_submit at ffffffff811b0f3c
#12 [ffff880fa7a0daf0] __blockdev_direct_IO_newtrunc at ffffffff811b1921
#13 [ffff880fa7a0dbd0] __blockdev_direct_IO at ffffffff811b1e7e
#14 [ffff880fa7a0dc50] blkdev_direct_IO at ffffffff811af4c7
#15 [ffff880fa7a0dc90] generic_file_aio_read at ffffffff81111f4b
#16 [ffff880fa7a0dd70] blkdev_aio_read at ffffffff811aea13
#17 [ffff880fa7a0ddc0] do_sync_read at ffffffff811766da
#18 [ffff880fa7a0def0] vfs_read at ffffffff81176fc5
#19 [ffff880fa7a0df30] sys_pread64 at ffffffff811772f2
#20 [ffff880fa7a0df80] system_call_fastpath at ffffffff8100b072