'cp' over an NFS share hangs and triggers hung_task_timeout on Red Hat Enterprise Linux 5.8

Issue

• Periodically, cp becomes blocked and triggers hung_task_timeout.
• The backtrace seen in messages file looks like this

INFO: task cp:27324 blocked for more than 120 seconds.
"echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
cp            D ffffffff80157271     0 27324  20865                     (NOTLB)
 ffff811457239da8 0000000000000082 0000000000000001 ffffffff8033f0e8
 0000000000000000 0000000000000007 ffff81154e4f2860 ffff81407f8bf080
 00005dd4b605f5d5 000000000000049f ffff81154e4f2a48 000000491dc5baf8
Call Trace:
 [<ffffffff8006ece7>] do_gettimeofday+0x40/0x90
 [<ffffffff88d5f4c9>] :nfs:nfs_wait_bit_uninterruptible+0x0/0xd
 [<ffffffff800637de>] io_schedule+0x3f/0x67
 [<ffffffff88d5f4d2>] :nfs:nfs_wait_bit_uninterruptible+0x9/0xd
 [<ffffffff80063a0a>] __wait_on_bit+0x40/0x6e
 [<ffffffff88d5f4c9>] :nfs:nfs_wait_bit_uninterruptible+0x0/0xd
 [<ffffffff80063aa4>] out_of_line_wait_on_bit+0x6c/0x78
 [<ffffffff800a34f9>] wake_bit_function+0x0/0x23
 [<ffffffff88d64218>] :nfs:nfs_sync_inode_wait+0xeb/0x2df
 [<ffffffff88d59ecd>] :nfs:nfs_do_fsync+0x22/0x42
 [<ffffffff80023d0e>] filp_close+0x36/0x64
 [<ffffffff8001e13b>] sys_close+0x88/0xbd