The kernel crashed upon blocked task detection where many tasks stuck in I/O wait on a virtio_blk device.

Solution Unverified - Updated -

Issue

  • The kernel crashed upon blocked task detection where many tasks stuck in I/O wait on a virtio_blk device.
     RELEASE: 2.6.32-754.35.1.el6.x86_64
       PANIC: "Kernel panic - not syncing: hung_task: blocked tasks"

INFO: task jbd2/dm-1-8:494 blocked for more than 120 seconds.
      Not tainted 2.6.32-754.35.1.el6.x86_64 #1
"echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
jbd2/dm-1-8   D 0000000000000006     0   494      2 0x00000000
 ffff88082c817c20 0000000000000046 ffff88082c817ba0 ffffffff81049268
 ffff8808ffffffff 003560365eb081d1 ffff88082cf08e40 ffff880833c6bde0
 000000006309857e ffffffff9fba877a ffff88082dcd3068 ffff88082c817fd8
Call Trace:
 [<ffffffff81049268>] ? pvclock_clocksource_read+0x58/0xd0
 [<ffffffff810b8394>] ? ktime_get_ts+0xc4/0x100
 [<ffffffff811da9f0>] ? sync_buffer+0x0/0x50
 [<ffffffff811da9f0>] ? sync_buffer+0x0/0x50
 [<ffffffff8155b313>] io_schedule+0x73/0xc0
 [<ffffffff811daa33>] sync_buffer+0x43/0x50
 [<ffffffff8155be01>] __wait_on_bit+0x61/0x90
 [<ffffffff811da9f0>] ? sync_buffer+0x0/0x50
 [<ffffffff8155bea8>] out_of_line_wait_on_bit+0x78/0x90
 [<ffffffff810ac280>] ? wake_bit_function+0x0/0x50
 [<ffffffff811da9e6>] __wait_on_buffer+0x26/0x30
 [<ffffffffa0075176>] jbd2_journal_commit_transaction+0xaa6/0x1520 [jbd2]
 [<ffffffff810951eb>] ? try_to_del_timer_sync+0x7b/0xe0
 [<ffffffffa007aab8>] kjournald2+0xb8/0x220 [jbd2]
 [<ffffffff810ac200>] ? autoremove_wake_function+0x0/0x40
 [<ffffffffa007aa00>] ? kjournald2+0x0/0x220 [jbd2]
 [<ffffffff810abd60>] kthread+0xa0/0xc0
 [<ffffffff81567820>] child_rip+0x20/0x30
 [<ffffffff810abcc0>] ? kthread+0x0/0xc0
 [<ffffffff81567800>] ? child_rip+0x0/0x30

Environment

  • rhel6.10.z (kernel-2.6.32-754.35.1.el6)
  • Hypervisor: Ubuntu-1.8.2-1ubuntu1~cloud0

Subscriber exclusive content

A Red Hat subscription provides unlimited access to our knowledgebase, tools, and much more.

Current Customers and Partners

Log in for full access

Log In

New to Red Hat?

Learn more about Red Hat subscriptions

Using a Red Hat product through a public cloud?

How to access this content