Kernel panic due to Hard LOCKUP in the function blk_throtl_drain()
Issue
- Kernel panic due to Hard LOCKUP or the system hang with the following warning message. Issue has only been reported with third-party software, either VxDMP or PowerPath installed.
- VxDMP:
WARNING: at block/blk-throttle.c:1222 blk_throtl_drain+0xff/0x180() (Tainted: P ------------- )
Hardware name: PowerEdge R720
Modules linked in: vxodm(P)(U) vxgms(P)(U) amf(P)(U) vxglm(P)(U) vxfen(P)(U) gab(P)(U) llt(P)(U) autofs4 dmpjbod(P)(U) dmpap(P)(U) dmpaa(P)(U) vxspec(P)(U) vxio(P)(U) vxdmp(P)(U) ib_ipoib rdma_ucm ib_ucm ib_uverbs ib_umad rdma_cm ib_cm iw_cm vxcafs(P)(U) vxportal(P)(U) fdd(P)(U) vxfs(P)(U) exportfs microcode iTCO_wdt iTCO_vendor_support dcdbas power_meter acpi_ipmi ipmi_si ipmi_msghandler sb_edac edac_core lpc_ich mfd_core shpchp joydev ses enclosure sg ixgbe dca mdio tg3 ext4 jbd2 mbcache sr_mod cdrom usb_storage sd_mod crc_t10dif mpt2sas scsi_transport_sas raid_class ahci qla2xxx scsi_transport_fc scsi_tgt mlx4_ib ib_sa ib_mad ib_core ib_addr ipv6 mlx4_en ptp pps_core mlx4_core megaraid_sas wmi dm_mirror dm_region_hash dm_log dm_mod [last unloaded: speedstep_lib]
Pid: 11034, comm: vxconfigd Tainted: P -------------- 2.6.32-754.6.3.el6.x86_64 #1
Call Trace:
[<ffffffff810805c1>] ? warn_slowpath_common+0x91/0xe0
[<ffffffff8108062a>] ? warn_slowpath_null+0x1a/0x20
[<ffffffff812978af>] ? blk_throtl_drain+0xff/0x180
[<ffffffff81286cd1>] ? __blk_drain_queue+0x91/0x140
[<ffffffff81286fab>] ? blk_cleanup_queue+0xeb/0x1d0
[<ffffffffa09c7c38>] ? vxvm_put_gendisk+0x68/0xf0 [vxio]
[<ffffffffa09c7ce2>] ? volsys_unset_device+0x22/0x40 [vxio]
[<ffffffffa0a0ef82>] ? vol_rmgroup_devices+0x82/0x100 [vxio]
[<ffffffffa0a10eaa>] ? voldg_delete+0x1a/0x130 [vxio]
[<ffffffffa0ba2390>] ? vol_rv_dgdelete_prepare_group+0x30/0x60 [vxio]
[<ffffffffa0a1233b>] ? vol_delete_group+0x23b/0x290 [vxio]
[<ffffffffa0ad3bf3>] ? volconfig_ioctl+0x603/0x6b0 [vxio]
[<ffffffff8124326f>] ? security_capable+0x2f/0x40
[<ffffffff8108e77a>] ? capable+0x2a/0x60
[<ffffffffa0adcdb2>] ? volsioctl_real+0x412/0x550 [vxio]
[<ffffffff81054e64>] ? __do_page_fault+0x1f4/0x500
[<ffffffffa030a13c>] ? vols_ioctl+0x5c/0x80 [vxspec]
[<ffffffffa030a37d>] ? vols_compat_ioctl+0x3d/0x60 [vxspec]
[<ffffffff811fb1d8>] ? compat_sys_ioctl+0xf8/0x520
[<ffffffff81565be7>] ? sysenter_dispatch+0x20/0x4b
---[ end trace d7e320c384a74d7d ]---
Kernel panic - not syncing: Hard LOCKUP
- PowerPath:
kernel: WARNING: at block/blk-throttle.c:1222 blk_throtl_drain+0xff/0x180() (Tainted: P -- ------------ )
kernel: Hardware name: ProLiant DL380p Gen8
kernel: Modules linked in: oracleacfs(P)(U) oracleadvm(P)(U) oracleoks(P)(U) autofs4 nfs lockd fscache auth_rpcgss nfs_acl sunrpc cpufreq_ondemand freq_table pcc_cpufreq bonding vfat fat emcpdm(P)(U) emcpmpx(P)(U) emcpgpx(P)(U) emcp(P)(U) microcode iTCO_wdt iTCO_vendor_support joydev hpilo hpwdt be2net serio_raw lpc_ich mfd_core bnx2x ptp pps_core libcrc32c mdio sg power_meter acpi_ipmi ipmi_si ipmi_msghandler shpchp ext3 jbd mbcache sd_mod hpsa(U) scsi_transport_sas lpfc scsi_transport_fc scsi_tgt crc_t10dif pata_acpi ata_generic ata_piix dm_mirror dm_region_hash dm_log dm_mod [last unloaded: emcpioc]
kernel: Pid: 7622, comm: emcpadm Tainted: P -- ------------ 2.6.32-754.6.3.el6.x86_64 #1
kernel: Call Trace:
kernel: [<ffffffff810805c1>] ? warn_slowpath_common+0x91/0xe0
kernel: [<ffffffff8108062a>] ? warn_slowpath_null+0x1a/0x20
kernel: [<ffffffff812978af>] ? blk_throtl_drain+0xff/0x180
kernel: [<ffffffff81286cd1>] ? __blk_drain_queue+0x91/0x140
kernel: [<ffffffff81286fab>] ? blk_cleanup_queue+0xeb/0x1d0
kernel: [<ffffffffa0381494>] ? emcpCleanupPDev+0x84/0x120 [emcp]
kernel: [<ffffffffa038bfc7>] ? EmsLnxUnmap+0x127/0x1a0 [emcp]
kernel: [<ffffffffa0390cbf>] ? EmcpUnmapTopGuts+0x5f/0x100 [emcp]
kernel: [<ffffffffa0385a5b>] ? EmsSuspendIoAtTop+0xfb/0x150 [emcp]
kernel: [<ffffffffa0390ea9>] ? EmcpUnmapGuts+0x59/0xb0 [emcp]
kernel: [<ffffffffa03911d7>] ? EmcpUnmapDeviceId+0x67/0xc0 [emcp]
kernel: [<ffffffffa039126a>] ? EmcpIocmdUnmapDeviceId+0x3a/0x50 [emcp]
kernel: [<ffffffffa039419b>] ? PowerIocmdHandler+0x1bb/0x260 [emcp]
kernel: [<ffffffffa03943ae>] ? EmcpIocmd+0x16e/0x1e0 [emcp]
kernel: [<ffffffffa037cf8a>] ? PowerPlatformMalloc+0x6a/0x150 [emcp]
kernel: [<ffffffffa03945c4>] ? emcp_iocmd+0x1a4/0x200 [emcp]
kernel: [<ffffffffa0394946>] ? emcp_pseudo_ctl_ioctl+0x326/0x490 [emcp]
kernel: [<ffffffff811b71c2>] ? vfs_ioctl+0x92/0xc0
kernel: [<ffffffff811b7314>] ? do_vfs_ioctl+0x84/0x590
kernel: [<ffffffff811b78a1>] ? sys_ioctl+0x81/0xa0
kernel: [<ffffffff81564272>] ? system_call_after_swapgs+0xa2/0x152
kernel: [<ffffffff8156427e>] ? system_call_after_swapgs+0xae/0x152
kernel: [<ffffffff81564272>] ? system_call_after_swapgs+0xa2/0x152
kernel: [<ffffffff81564357>] ? system_call_fastpath+0x35/0x3a
kernel: [<ffffffff8156427e>] ? system_call_after_swapgs+0xae/0x152
kernel: ---[ end trace 7b23eba818a6efe0 ]---
Kernel panic - not syncing: Hard LOCKUP
Environment
- Red Hat Enterprise Linux 6.10
- kernel-2.6.32-754.6.3.el6
- Either:
- Veritas VxDMP
- EMC PowerPath
- Veeam Agent for Linux
Subscriber exclusive content
A Red Hat subscription provides unlimited access to our knowledgebase, tools, and much more.