kswapd blocked trying to free dirty NFS pages on RHEL 5

Solution Verified - Updated -

Issue

  • RHEL 5 NFS client freezes under heavy NFS load
  • NFS Client running on RHEL 5 hung temporarily for around 5-7 minutes while performing a Database backup to an NFS mount.
  • Slow performance when copying a file between two mount points on a Netapp Filer
  • Lock-up writing to nfs share
  • System was rebooted. Message "task kswapd0:1503 blocked for more than 120 seconds" seen in messages file.
INFO: task kswapd0:670 blocked for more than 120 seconds.
"echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
kswapd0       D ffff810001004420     0   670    236           671   669 (L-TLB)
 ffff8103475399e0 0000000000000046 ffff81010b8572c0 ffff8102c452d000
 000000000000000c 000000000000000a ffff8106474a6100 ffffffff80310b60
 00001dc1f306c3a7 0000000000007f6c ffff8106474a62e8 00000000800de2c3
Call Trace:
 [<ffffffff8006ec4e>] do_gettimeofday+0x40/0x90
 [<ffffffff89337941>] :nfs:nfs_wait_bit_uninterruptible+0x0/0xd
 [<ffffffff800637ca>] io_schedule+0x3f/0x67
 [<ffffffff8933794a>] :nfs:nfs_wait_bit_uninterruptible+0x9/0xd
 [<ffffffff800639f6>] __wait_on_bit+0x40/0x6e
 [<ffffffff89337941>] :nfs:nfs_wait_bit_uninterruptible+0x0/0xd
 [<ffffffff80063a90>] out_of_line_wait_on_bit+0x6c/0x78
 [<ffffffff800a28e2>] wake_bit_function+0x0/0x23
 [<ffffffff8933b30a>] :nfs:nfs_wait_on_requests_locked+0x70/0xca
 [<ffffffff8933c340>] :nfs:nfs_sync_inode_wait+0x60/0x1db
 [<ffffffff893322c8>] :nfs:nfs_release_page+0x2c/0x4d
 [<ffffffff800cd32c>] shrink_inactive_list+0x511/0x8d8
 [<ffffffff800cc81f>] isolate_lru_pages+0x98/0xbf
 [<ffffffff80048014>] __pagevec_release+0x19/0x22
 [<ffffffff800cccfa>] shrink_active_list+0x4b4/0x4c4
 [<ffffffff800131c7>] shrink_zone+0x127/0x18d
 [<ffffffff80057c0a>] kswapd+0x33d/0x495
 [<ffffffff800a28b4>] autoremove_wake_function+0x0/0x2e
 [<ffffffff800578cd>] kswapd+0x0/0x495
 [<ffffffff800a269c>] keventd_create_kthread+0x0/0xc4
 [<ffffffff80032996>] kthread+0xfe/0x132
 [<ffffffff8005dfb1>] child_rip+0xa/0x11
 [<ffffffff800a269c>] keventd_create_kthread+0x0/0xc4
 [<ffffffff80032898>] kthread+0x0/0x132
 [<ffffffff8005dfa7>] child_rip+0x0/0x11

Environment

  • Red Hat Enterprise Linux 5.6 (NFS client)

Subscriber exclusive content

A Red Hat subscription provides unlimited access to our knowledgebase, tools, and much more.

Current Customers and Partners

Log in for full access

Log In

New to Red Hat?

Learn more about Red Hat subscriptions

Using a Red Hat product through a public cloud?

How to access this content