INFO: task vmtoolsd:XXXX blocked for more than 120 seconds.
Issue
- Server frequently getting hanged with multiple tasks blocked for more than 120 seconds.
INFO: task vmtoolsd:3288 blocked for more than 120 seconds.
INFO: task rs:main Q:Reg:4227 blocked for more than 120 seconds.
INFO: task nscd:4362 blocked for more than 120 seconds.
INFO: task nscd:4364 blocked for more than 120 seconds.
INFO: task nonstop_linux_i:4489 blocked for more than 120 seconds.
INFO: task java:4607 blocked for more than 120 seconds.
INFO: task discagnt:4957 blocked for more than 120 seconds.
INFO: task klzagent:5583 blocked for more than 120 seconds.
INFO: task java:28691 blocked for more than 120 seconds.
INFO: task ssh-broker-g3:24970 blocked for more than 120 seconds.
- On Red Hat Enterprise Linux 6, vmtoolsd task blocked with following call traces.
INFO: task vmtoolsd:1213 blocked for more than 120 seconds.
"echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
vmtoolsd D 0000000000000000 0 1213 1 0x00000000
ffff88007c771b28 0000000000000086 ffff88007f823800 ffff88007f8213c0
ffff88007c771af8 ffffffff811626ab ffff88007c771bd8 00000000f4899f64
ffff88007b36a5f8 ffff88007c771fd8 000000000000fb88 ffff88007b36a5f8
Call Trace:
[<ffffffff811626ab>] ? cache_alloc_refill+0x15b/0x240
[<ffffffff810923be>] ? prepare_to_wait+0x4e/0x80
[<ffffffffa0098005>] start_this_handle+0x2f5/0x4d0 [jbd2]
[<ffffffff811626ab>] ? cache_alloc_refill+0x15b/0x240
[<ffffffff810920d0>] ? autoremove_wake_function+0x0/0x40
[<ffffffffa00983c5>] jbd2_journal_start+0xb5/0x100 [jbd2]
[<ffffffffa00e3dc8>] ext4_journal_start_sb+0x58/0x90 [ext4]
[<ffffffffa00c9fba>] ext4_dirty_inode+0x2a/0x60 [ext4]
[<ffffffff811a4b1b>] __mark_inode_dirty+0x3b/0x160
[<ffffffff8119520d>] touch_atime+0x12d/0x170
[<ffffffff81115dc0>] generic_file_aio_read+0x380/0x700
[<ffffffff8117afda>] do_sync_read+0xfa/0x140
[<ffffffff810920d0>] ? autoremove_wake_function+0x0/0x40
[<ffffffff81145fca>] ? do_mmap_pgoff+0x33a/0x380
[<ffffffff81213266>] ? security_file_permission+0x16/0x20
[<ffffffff8117b9e5>] vfs_read+0xb5/0x1a0
[<ffffffff8117bb21>] sys_read+0x51/0x90
[<ffffffff8100b0f2>] system_call_fastpath+0x16/0x1b
- On Red Hat Enterprise Linux 5, vmtoolsd task blocked with following call traces.
INFO: task vmtoolsd:3288 blocked for more than 120 seconds.
"echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
vmtoolsd D ffff810009017120 0 3288 1 3510 1744 (NOTLB)
ffff812037e6bb88 0000000000000086 0000000000000000 0000000000000000
0000000000000096 0000000000000007 ffff81203fab27a0 ffff810170f5b7a0
00004e8664e41920 000000000010ebf2 ffff81203fab2988 0000000200040000
Call Trace:
[<ffffffff8012da79>] avc_has_perm+0x46/0x58
[<ffffffff8803badd>] :jbd2:start_this_handle+0x128/0x3b3
[<ffffffff800a2964>] autoremove_wake_function+0x0/0x2e
[<ffffffff8803be09>] :jbd2:jbd2_journal_start+0xa1/0xd8
[<ffffffff8805e0d8>] :ext4:ext4_dirty_inode+0x1a/0x46
[<ffffffff80013d67>] __mark_inode_dirty+0x29/0x16e
[<ffffffff8000c5f5>] do_generic_mapping_read+0x347/0x359
[<ffffffff8000d279>] file_read_actor+0x0/0x159
[<ffffffff8000c753>] __generic_file_aio_read+0x14c/0x198
[<ffffffff80016ed0>] generic_file_aio_read+0x34/0x39
[<ffffffff8000cfa2>] do_sync_read+0xc7/0x104
[<ffffffff800a2964>] autoremove_wake_function+0x0/0x2e
[<ffffffff8000e2a5>] do_mmap_pgoff+0x615/0x780
[<ffffffff8000b78d>] vfs_read+0xcb/0x171
[<ffffffff80011d2f>] sys_read+0x45/0x6e
[<ffffffff8005d116>] system_call+0x7e/0x83
Environment
- Red Hat Enterprise Linux 6
- Red Hat Enterprise Linux 5
- VMware Tools ( vmtoolsd )
Subscriber exclusive content
A Red Hat subscription provides unlimited access to our knowledgebase, tools, and much more.