The kernel crashed upon blocked task detection where many tasks stuck in I/O wait on a virtio_blk device.

Solution Unverified - Updated -

Issue

  • The kernel crashed upon blocked task detection where many tasks stuck in I/O wait on a virtio_blk device.
     RELEASE: 2.6.32-754.35.1.el6.x86_64
       PANIC: "Kernel panic - not syncing: hung_task: blocked tasks"

INFO: task jbd2/dm-1-8:494 blocked for more than 120 seconds.
      Not tainted 2.6.32-754.35.1.el6.x86_64 #1
"echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
jbd2/dm-1-8   D 0000000000000006     0   494      2 0x00000000
 ffff88082c817c20 0000000000000046 ffff88082c817ba0 ffffffff81049268
 ffff8808ffffffff 003560365eb081d1 ffff88082cf08e40 ffff880833c6bde0
 000000006309857e ffffffff9fba877a ffff88082dcd3068 ffff88082c817fd8
Call Trace:
 [<ffffffff81049268>] ? pvclock_clocksource_read+0x58/0xd0
 [<ffffffff810b8394>] ? ktime_get_ts+0xc4/0x100
 [<ffffffff811da9f0>] ? sync_buffer+0x0/0x50
 [<ffffffff811da9f0>] ? sync_buffer+0x0/0x50
 [<ffffffff8155b313>] io_schedule+0x73/0xc0
 [<ffffffff811daa33>] sync_buffer+0x43/0x50
 [<ffffffff8155be01>] __wait_on_bit+0x61/0x90
 [<ffffffff811da9f0>] ? sync_buffer+0x0/0x50
 [<ffffffff8155bea8>] out_of_line_wait_on_bit+0x78/0x90
 [<ffffffff810ac280>] ? wake_bit_function+0x0/0x50
 [<ffffffff811da9e6>] __wait_on_buffer+0x26/0x30
 [<ffffffffa0075176>] jbd2_journal_commit_transaction+0xaa6/0x1520 [jbd2]
 [<ffffffff810951eb>] ? try_to_del_timer_sync+0x7b/0xe0
 [<ffffffffa007aab8>] kjournald2+0xb8/0x220 [jbd2]
 [<ffffffff810ac200>] ? autoremove_wake_function+0x0/0x40
 [<ffffffffa007aa00>] ? kjournald2+0x0/0x220 [jbd2]
 [<ffffffff810abd60>] kthread+0xa0/0xc0
 [<ffffffff81567820>] child_rip+0x20/0x30
 [<ffffffff810abcc0>] ? kthread+0x0/0xc0
 [<ffffffff81567800>] ? child_rip+0x0/0x30

Environment

  • rhel6.10.z (kernel-2.6.32-754.35.1.el6)
  • Hypervisor: Ubuntu-1.8.2-1ubuntu1~cloud0

Subscriber exclusive content

A Red Hat subscription provides unlimited access to our knowledgebase, tools, and much more.

Current Customers and Partners

Log in for full access

Log In