The watchdog timer and vxfs causes INFO: task khugepaged:pppp blocked for more than xxx seconds type messages to be logged and hung IO in RHEL 6.

Solution Unverified - Updated -

Issue

  • We log messages of the below type stating that a task/s have been hung for more that xxx seconds. The default time is 120 seconds.
INFO: task khugepaged:421 blocked for more than 120 seconds. 
"echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. 
khugepaged    D 0000000000000020     0   421      2 0x00000000 
 ffff8820100e3c90 0000000000000046 0000000000000000 ffff88000002dd40 
 ffff8820100e1540 00000000004352da ffff8820100e3d40 ffffffff8112bae3 
 ffff8820100e1af8 ffff8820100e3fd8 000000000000fb88 ffff8820100e1af8 
Call Trace: 
 [<ffffffff8112bae3>] ? __alloc_pages_nodemask+0x113/0x8d0 
 [<ffffffff8150fc45>] rwsem_down_failed_common+0x95/0x1d0 
 [<ffffffff8150fda3>] rwsem_down_write_failed+0x23/0x30 
 [<ffffffff812833e3>] call_rwsem_down_write_failed+0x13/0x20 
 [<ffffffff8150f2a2>] ? down_write+0x32/0x40 
 [<ffffffff81179eb6>] khugepaged+0x7f6/0x1310 
 [<ffffffff81096c80>] ? autoremove_wake_function+0x0/0x40 
 [<ffffffff811796c0>] ? khugepaged+0x0/0x1310 
 [<ffffffff81096916>] kthread+0x96/0xa0 
 [<ffffffff8100c0ca>] child_rip+0xa/0x20 
 [<ffffffff81096880>] ? kthread+0x0/0xa0 
 [<ffffffff8100c0c0>] ? child_rip+0x0/0x20 
<snip>
INFO: task vx_naio_worker:41196 blocked for more than 120 seconds. 
"echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. 
vx_naio_worke D 0000000000000000     0 41196      2 0x00000080 
 ffff8809e9ee3c40 0000000000000046 ffff880099b56798 ffff880099b56768 
 ffff880123a95578 000000000022feba ffff880123a95540 0000000000000000 
 ffff88080815f098 ffff8809e9ee3fd8 000000000000fb88 ffff88080815f098 
Call Trace: 
 [<ffffffffa05610ce>] vx_svar_sleep_unlock+0x6e/0xd0 [vxfs] 
 [<ffffffffa0560e60>] ? vx_wq_wakeup_function+0x0/0x20 [vxfs] 
 [<ffffffff8106317e>] ? try_to_wake_up+0x24e/0x3e0 
 [<ffffffffa0537b09>] vx_rwsleep_rec_lock+0xd9/0x110 [vxfs] 
 [<ffffffffa05044b2>] vx_recsmp_lock+0x12/0x20 [vxfs] 
 [<ffffffffa0620f75>] vx_write_common+0x1e5/0x860 [vxfs] 
 [<ffffffff81051419>] ? __wake_up_common+0x59/0x90 
 [<ffffffffa059f110>] vx_vop_write+0x20/0x30 [vxfs] 
 [<ffffffffa057f2e2>] vx_naio_do_work+0x1a2/0x440 [vxfs] 
 [<ffffffffa057f609>] vx_naio_worker+0x89/0x2b0 [vxfs] 
 [<ffffffffa057f580>] ? vx_naio_worker+0x0/0x2b0 [vxfs] 
 [<ffffffffa059388b>] vx_kthread_init+0x7b/0x90 [vxfs] 
 [<ffffffffa057f580>] ? vx_naio_worker+0x0/0x2b0 [vxfs] 
 [<ffffffff8100c0ca>] child_rip+0xa/0x20 
 [<ffffffffa057f580>] ? vx_naio_worker+0x0/0x2b0 [vxfs] 
 [<ffffffffa0593810>] ? vx_kthread_init+0x0/0x90 [vxfs] 
 [<ffffffff8100c0c0>] ? child_rip+0x0/0x20 

Environment

  • Red Hat Enterprise Linux 6

Subscriber exclusive content

A Red Hat subscription provides unlimited access to our knowledgebase, tools, and much more.

Current Customers and Partners

Log in for full access

Log In

New to Red Hat?

Learn more about Red Hat subscriptions

Using a Red Hat product through a public cloud?

How to access this content