I/O to multipath map hangs even after all paths are failed and configured to not queue in RHEL 6

Issue

• Multipath map is configured with no_path_retry fail, but I/O hangs even after all paths have failed
• Cluster node was not evicted by qdiskd after all paths failed, even though the map is configured to not queue. qdiskd was reported as blocked by the kernel:

Jul 9 16:56:42,node1,[daemon.notice],multipathd:, 36006016061602d008296e08eb281e211: remaining active paths: 0
[...]
Jul 9 17:00:12,node1,[kern.err],kernel:,INFO: task qdiskd:24750 blocked for more than 120 seconds.
Jul 9 17:00:12,node1,[kern.err],kernel:,"echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
Jul 9 17:00:12,node1,[kern.info],kernel:,qdiskd D 0000000000000004 0 24750 1 0x00000000
Jul 9 17:00:12,node1,[kern.warning],kernel:, ffff8804706dfb18 0000000000000082 0000000000000000 ffffffffa00041fc
Jul 9 17:00:12,node1,[kern.warning],kernel:, ffff8804706dfae8 00000000e4bd82fc ffff8804706dfb08 ffff880875e5ad80
Jul 9 17:00:12,node1,[kern.warning],kernel:, ffff8804726ad0f8 ffff8804706dffd8 000000000000f4e8 ffff8804726ad0f8
Jul 9 17:00:12,node1,[kern.warning],kernel:,Call Trace:
Jul 9 17:00:12,node1,[kern.warning],kernel:, [] ? dm_table_unplug_all+0x5c/0x100 [dm_mod]
Jul 9 17:00:12,node1,[kern.warning],kernel:, [] io_schedule+0x73/0xc0
Jul 9 17:00:12,node1,[kern.warning],kernel:, [] __blockdev_direct_IO_newtrunc+0x6fe/0xb90
Jul 9 17:00:12,node1,[kern.warning],kernel:, [] __blockdev_direct_IO+0x5e/0xd0
Jul 9 17:00:12,node1,[kern.warning],kernel:, [] ? blkdev_get_blocks+0x0/0xc0
Jul 9 17:00:12,node1,[kern.warning],kernel:, [] blkdev_direct_IO+0x57/0x60
Jul 9 17:00:12,node1,[kern.warning],kernel:, [] ? blkdev_get_blocks+0x0/0xc0
Jul 9 17:00:12,node1,[kern.warning],kernel:, [] generic_file_aio_read+0x6bb/0x700
Jul 9 17:00:12,node1,[kern.warning],kernel:, [] ? futex_requeue+0x310/0x890
Jul 9 17:00:12,node1,[kern.warning],kernel:, [] do_sync_read+0xfa/0x140
Jul 9 17:00:12,node1,[kern.warning],kernel:, [] ? autoremove_wake_function+0x0/0x40
Jul 9 17:00:12,node1,[kern.warning],kernel:, [] ? unmap_region+0x110/0x130
Jul 9 17:00:12,node1,[kern.warning],kernel:, [] ? security_file_permission+0x16/0x20
Jul 9 17:00:12,node1,[kern.warning],kernel:, [] vfs_read+0xb5/0x1a0
Jul 9 17:00:12,node1,[kern.warning],kernel:, [] sys_read+0x51/0x90
Jul 9 17:00:12,node1,[kern.warning],kernel:, [] system_call_fastpath+0x16/0x1b