dmesg reports nfs_wait_bit_uninterruptible (CPU Soft Locks)

Issue

• A new RHEL 5.4 installation seeing intermittent conectivity issues.

• syslog and dmesg report the following messages:

    INFO: task jboss_get_stage:22410 blocked for more than 120 seconds.
    "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
    jboss_get_sta D ffff810001015120     0 22410  22409                     (NOTLB)
     ffff810021bb9d48 0000000000000082 ffff810129948ce0 ffff810125ac1240
     ffff81001730a040 0000000000000007 ffff8100528f3040 ffff81010439d080
     000016ead9a85d5d 00000000000006ba ffff8100528f3228 000000022f2e1eb8
    Call Trace:
     [<ffffffff8006e1db>] do_gettimeofday+0x40/0x90
     [<ffffffff8855a6e5>] :nfs:nfs_wait_bit_uninterruptible+0x0/0xd
     [<ffffffff800637ea>] io_schedule+0x3f/0x67
     [<ffffffff8855a6ee>] :nfs:nfs_wait_bit_uninterruptible+0x9/0xd
     [<ffffffff80063a16>] __wait_on_bit+0x40/0x6e
     [<ffffffff8855a6e5>] :nfs:nfs_wait_bit_uninterruptible+0x0/0xd
     [<ffffffff80063ab0>] out_of_line_wait_on_bit+0x6c/0x78
     [<ffffffff800a0a06>] wake_bit_function+0x0/0x23
     [<ffffffff8855e106>] :nfs:nfs_wait_on_requests_locked+0x70/0xca
     [<ffffffff8855f146>] :nfs:nfs_sync_inode_wait+0x60/0x1db
     [<ffffffff88555167>] :nfs:nfs_do_fsync+0x1f/0x3f
     [<ffffffff88555a06>] :nfs:nfs_file_flush+0x84/0xac
     [<ffffffff80023abe>] filp_close+0x36/0x64
     [<ffffffff8001dfac>] sys_close+0x88/0xbd
     [<ffffffff8005d28d>] tracesys+0xd5/0xe0

• This was reproducible by copying a 500M file to an NFS share. The copy hangs, producing these errors.