RHEL7: One program writing to NFS hangs under nfs_wait_bit_uninterruptible while the rest of NFS remains responsive.

Issue

• Defunct process is stuck in in nfs_wait_bit_interruptible
• Process writing to an NFS file becomes stuck while other processes access NFS without issue
• RHEL 7.2 system is getting hung while taking backup on NFS

Apr 25 22:44:30 localhost kernel: INFO: task kernel:22442 blocked for more than 120 seconds.
Apr 25 22:44:30 localhost kernel: "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
Apr 25 22:44:30 localhost kernel: kernel          D ffff8823c49d7b30     0 22442   5299 0x00000080
Apr 25 22:44:30 localhost kernel: ffff8823c49d79d0 0000000000000086 ffff880a04789700 ffff8823c49d7fd8
Apr 25 22:44:30 localhost kernel: ffff8823c49d7fd8 ffff8823c49d7fd8 ffff880a04789700 ffff88283fc14780
Apr 25 22:44:30 localhost kernel: 0000000000000000 7fffffffffffffff ffffffffa0598bc0 ffff8823c49d7b30
Apr 25 22:44:30 localhost kernel: Call Trace:
Apr 25 22:44:30 localhost kernel: [<ffffffffa0598bc0>] ? nfs_pgio_header_free+0x20/0x20 [nfs]
Apr 25 22:44:30 localhost kernel: [<ffffffff8163a879>] schedule+0x29/0x70
Apr 25 22:44:30 localhost kernel: [<ffffffff81638569>] schedule_timeout+0x209/0x2d0
Apr 25 22:44:30 localhost kernel: [<ffffffff8101cd15>] ? native_sched_clock+0x35/0x80
Apr 25 22:44:30 localhost kernel: [<ffffffff8101c829>] ? read_tsc+0x9/0x10
Apr 25 22:44:30 localhost kernel: [<ffffffffa0598bc0>] ? nfs_pgio_header_free+0x20/0x20 [nfs]
Apr 25 22:44:30 localhost kernel: [<ffffffff81639eae>] io_schedule_timeout+0xae/0x130
Apr 25 22:44:30 localhost kernel: [<ffffffff81639f48>] io_schedule+0x18/0x20
Apr 25 22:44:30 localhost kernel: [<ffffffffa0598bce>] nfs_wait_bit_uninterruptible+0xe/0x20 [nfs]
Apr 25 22:44:30 localhost kernel: [<ffffffff816386f0>] __wait_on_bit+0x60/0x90
Apr 25 22:44:30 localhost kernel: [<ffffffffa0598bc0>] ? nfs_pgio_header_free+0x20/0x20 [nfs]
Apr 25 22:44:30 localhost kernel: [<ffffffff816387a7>] out_of_line_wait_on_bit+0x87/0xb0
Apr 25 22:44:30 localhost kernel: [<ffffffff810a6b60>] ? wake_atomic_t_function+0x40/0x40
Apr 25 22:44:30 localhost kernel: [<ffffffffa059a593>] nfs_wait_on_request+0x33/0x40 [nfs]
Apr 25 22:44:30 localhost kernel: [<ffffffffa059f161>] nfs_updatepage+0x121/0x8a0 [nfs]
Apr 25 22:44:30 localhost kernel: [<ffffffffa058efa1>] nfs_write_end+0x121/0x350 [nfs]
Apr 25 22:44:30 localhost kernel: [<ffffffff81169d74>] generic_file_buffered_write+0x184/0x290
Apr 25 22:44:30 localhost kernel: [<ffffffff8116b235>] __generic_file_aio_write+0x1d5/0x3e0
Apr 25 22:44:30 localhost kernel: [<ffffffff8116b49d>] generic_file_aio_write+0x5d/0xc0
Apr 25 22:44:30 localhost kernel: [<ffffffffa058e03b>] nfs_file_write+0xbb/0x1d0 [nfs]
Apr 25 22:44:30 localhost kernel: [<ffffffff811ddcbd>] do_sync_write+0x8d/0xd0
Apr 25 22:44:30 localhost kernel: [<ffffffff811de4dd>] vfs_write+0xbd/0x1e0
Apr 25 22:44:30 localhost kernel: [<ffffffff811def7f>] SyS_write+0x7f/0xe0
Apr 25 22:44:30 localhost kernel: [<ffffffff816458c9>] system_call_fastpath+0x16/0x1