INFO: task vmtoolsd:XXXX blocked for more than 120 seconds.

Solution Verified - Updated -

Issue

  • Server frequently getting hanged with multiple tasks blocked for more than 120 seconds.
INFO: task vmtoolsd:3288 blocked for more than 120 seconds.
INFO: task rs:main Q:Reg:4227 blocked for more than 120 seconds.
INFO: task nscd:4362 blocked for more than 120 seconds.
INFO: task nscd:4364 blocked for more than 120 seconds.
INFO: task nonstop_linux_i:4489 blocked for more than 120 seconds.
INFO: task java:4607 blocked for more than 120 seconds.
INFO: task discagnt:4957 blocked for more than 120 seconds.
INFO: task klzagent:5583 blocked for more than 120 seconds.
INFO: task java:28691 blocked for more than 120 seconds.
INFO: task ssh-broker-g3:24970 blocked for more than 120 seconds.
  • On Red Hat Enterprise Linux 6, vmtoolsd task blocked with following call traces.
INFO: task vmtoolsd:1213 blocked for more than 120 seconds.
"echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
vmtoolsd      D 0000000000000000     0  1213      1 0x00000000
 ffff88007c771b28 0000000000000086 ffff88007f823800 ffff88007f8213c0
 ffff88007c771af8 ffffffff811626ab ffff88007c771bd8 00000000f4899f64
 ffff88007b36a5f8 ffff88007c771fd8 000000000000fb88 ffff88007b36a5f8
Call Trace:
 [<ffffffff811626ab>] ? cache_alloc_refill+0x15b/0x240
 [<ffffffff810923be>] ? prepare_to_wait+0x4e/0x80
 [<ffffffffa0098005>] start_this_handle+0x2f5/0x4d0 [jbd2]
 [<ffffffff811626ab>] ? cache_alloc_refill+0x15b/0x240
 [<ffffffff810920d0>] ? autoremove_wake_function+0x0/0x40
 [<ffffffffa00983c5>] jbd2_journal_start+0xb5/0x100 [jbd2]
 [<ffffffffa00e3dc8>] ext4_journal_start_sb+0x58/0x90 [ext4]
 [<ffffffffa00c9fba>] ext4_dirty_inode+0x2a/0x60 [ext4]
 [<ffffffff811a4b1b>] __mark_inode_dirty+0x3b/0x160
 [<ffffffff8119520d>] touch_atime+0x12d/0x170
 [<ffffffff81115dc0>] generic_file_aio_read+0x380/0x700
 [<ffffffff8117afda>] do_sync_read+0xfa/0x140
 [<ffffffff810920d0>] ? autoremove_wake_function+0x0/0x40
 [<ffffffff81145fca>] ? do_mmap_pgoff+0x33a/0x380
 [<ffffffff81213266>] ? security_file_permission+0x16/0x20
 [<ffffffff8117b9e5>] vfs_read+0xb5/0x1a0
 [<ffffffff8117bb21>] sys_read+0x51/0x90
 [<ffffffff8100b0f2>] system_call_fastpath+0x16/0x1b
  • On Red Hat Enterprise Linux 5, vmtoolsd task blocked with following call traces.
INFO: task vmtoolsd:3288 blocked for more than 120 seconds.
"echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
vmtoolsd      D ffff810009017120     0  3288      1          3510  1744 (NOTLB)
 ffff812037e6bb88 0000000000000086 0000000000000000 0000000000000000
 0000000000000096 0000000000000007 ffff81203fab27a0 ffff810170f5b7a0
 00004e8664e41920 000000000010ebf2 ffff81203fab2988 0000000200040000
Call Trace:
 [<ffffffff8012da79>] avc_has_perm+0x46/0x58
 [<ffffffff8803badd>] :jbd2:start_this_handle+0x128/0x3b3
 [<ffffffff800a2964>] autoremove_wake_function+0x0/0x2e
 [<ffffffff8803be09>] :jbd2:jbd2_journal_start+0xa1/0xd8
 [<ffffffff8805e0d8>] :ext4:ext4_dirty_inode+0x1a/0x46
 [<ffffffff80013d67>] __mark_inode_dirty+0x29/0x16e
 [<ffffffff8000c5f5>] do_generic_mapping_read+0x347/0x359
 [<ffffffff8000d279>] file_read_actor+0x0/0x159
 [<ffffffff8000c753>] __generic_file_aio_read+0x14c/0x198
 [<ffffffff80016ed0>] generic_file_aio_read+0x34/0x39
 [<ffffffff8000cfa2>] do_sync_read+0xc7/0x104
 [<ffffffff800a2964>] autoremove_wake_function+0x0/0x2e
 [<ffffffff8000e2a5>] do_mmap_pgoff+0x615/0x780
 [<ffffffff8000b78d>] vfs_read+0xcb/0x171
 [<ffffffff80011d2f>] sys_read+0x45/0x6e
 [<ffffffff8005d116>] system_call+0x7e/0x83

Environment

  • Red Hat Enterprise Linux 6
  • Red Hat Enterprise Linux 5
  • VMware Tools ( vmtoolsd )

Subscriber exclusive content

A Red Hat subscription provides unlimited access to our knowledgebase, tools, and much more.

Current Customers and Partners

Log in for full access

Log In

New to Red Hat?

Learn more about Red Hat subscriptions

Using a Red Hat product through a public cloud?

How to access this content