hung tasks due to real-time processes starving CPUs

Solution Unverified - Updated -

Issue

  • Processes hanging/deadlocking on host
  • System reported hung tasks:
INFO: task jbd2/sda2-8:897 blocked for more than 120 seconds.
"echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
jbd2/sda2-8   D 0000000000000000     0   897      2 0x00000000
 ffff881066385c20 0000000000000046 0000000000000000 ffff881066385be4
 0000000100000000 ffff88107fc28400 ffff880028256700 0000000000000200
 ffff881066f85ab8 ffff881066385fd8 000000000000fb88 ffff881066f85ab8
Call Trace:
 [<ffffffff811b63f0>] ? sync_buffer+0x0/0x50
 [<ffffffff811b63f0>] ? sync_buffer+0x0/0x50
 [<ffffffff8150f273>] io_schedule+0x73/0xc0
 [<ffffffff811b6430>] sync_buffer+0x40/0x50
 [<ffffffff8150fc2f>] __wait_on_bit+0x5f/0x90
 [<ffffffff811b63f0>] ? sync_buffer+0x0/0x50
 [<ffffffff8150fcd8>] out_of_line_wait_on_bit+0x78/0x90
 [<ffffffff81096dd0>] ? wake_bit_function+0x0/0x50
 [<ffffffff811b63e6>] __wait_on_buffer+0x26/0x30
 [<ffffffffa0074116>] jbd2_journal_commit_transaction+0xaa6/0x1500 [jbd2]
 [<ffffffff810096f0>] ? __switch_to+0xd0/0x320
 [<ffffffff81096d90>] ? autoremove_wake_function+0x0/0x40
 [<ffffffffa007a148>] kjournald2+0xb8/0x220 [jbd2]
 [<ffffffff81096d90>] ? autoremove_wake_function+0x0/0x40
 [<ffffffffa007a090>] ? kjournald2+0x0/0x220 [jbd2]
 [<ffffffff81096a26>] kthread+0x96/0xa0
 [<ffffffff8100c0ca>] child_rip+0xa/0x20
 [<ffffffff81096990>] ? kthread+0x0/0xa0
 [<ffffffff8100c0c0>] ? child_rip+0x0/0x20

Environment

  • Red Hat Enterprise Linux 6
  • CFQ scheduler
  • RT scheduler class

Subscriber exclusive content

A Red Hat subscription provides unlimited access to our knowledgebase of over 48,000 articles and solutions.

Current Customers and Partners

Log in for full access

Log In