System panicked due to hung tasks while dumping application core
Issue
- One of our production database servers crashed, and in doing so, generated a vmcore dump image.
- Following message found in system log:
mserver5[32148]: segfault at 7f1d8bff5000 ip 00007f1f38e74250 sp 00007f1f1e255650 error 4 in libbat.so.9.0.1[7f1f38e2b000+2f6000]
mserver5[23948]: segfault at 0 ip 00007f452a2627f5 sp 00007f44259eb9f0 error 4 in lib_sql.so[7f452a1d1000+149000]
mserver5[29386]: segfault at 0 ip 0000003066927a96 sp 00007f46f0afec18 error 4
mserver5[29383]: segfault at 0 ip 0000003066928544 sp 00007f46f17bec18 error 4 in libc-2.12.so[3066800000+18a000] in libc-2.12.so[3066800000+18a000]
mserver5[29399]: segfault at 0 ip 0000003066927e44 sp 00007f1590c4cc18 error 4
mserver5[29398]: segfault at 0 ip 0000003066927e44 sp 00007f1590e4dc18 error 4 in libc-2.12.so[3066800000+18a000] in libc-2.12.so[3066800000+18a000]
INFO: task mserver5:29403 blocked for more than 120 seconds.
"echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
mserver5 D 0000000000000011 0 29403 32044 0x00000080
ffff88101eb8dc98 0000000000000082 0000000000000000 ffffffffa00d54cb
ffff88101eb8dc68 ffffffff810aa590 ffff882024dafa00 ffff880be9042cc0
ffff881028247058 ffff88101eb8dfd8 000000000000fb88 ffff881028247058
Call Trace:
[<ffffffffa00d54cb>] ? __ext4_handle_dirty_metadata+0x7b/0x100 [ext4]
[<ffffffff810aa590>] ? exit_robust_list+0x90/0x160
[<ffffffff81072f95>] exit_mm+0x95/0x180
[<ffffffff810733df>] do_exit+0x15f/0x870
[<ffffffff81073b48>] do_group_exit+0x58/0xd0
[<ffffffff81088e36>] get_signal_to_deliver+0x1f6/0x460
[<ffffffff8100a265>] do_signal+0x75/0x800
[<ffffffff810a1a60>] ? ktime_get_ts+0x40/0xe0
[<ffffffff81196f88>] ? poll_select_copy_remaining+0xf8/0x150
[<ffffffff8100aa80>] do_notify_resume+0x90/0xc0
[<ffffffff8100b341>] int_signal+0x12/0x17
Kernel panic - not syncing: hung_task: blocked tasks
Environment
- Redhat Enterprise Linux 5
- Redhat Enterprise Linux 6
- application coredumping
Subscriber exclusive content
A Red Hat subscription provides unlimited access to our knowledgebase, tools, and much more.