System panicked in jbd2_journal_grab_journal_head() due to memory corruption.
Issue
- A couple of servers rebooted unexpectedly; would like root cause analysis
- No changes were made to the servers prior to the reboot
- System panicked with:
general protection fault: 0000 [#1] SMP
last sysfs file: /sys/devices/virtual/net/bond0/bonding/slaves
CPU 17
Modules linked in: iptable_filter ip_tables sunrpc ipmi_devintf bonding 8021q garp stp llc ipv6 power_meter hpwdt hpilo onload(U) sfc_char(U) sfc_resource(U) sfc_affinity(U) sfc_tune(U) sfc(U) i2c_algo_bit i2c_core ptp pps_core mdio sg tg3 microcode serio_raw iTCO_wdt iTCO_vendor_support ioatdma dca shpchp ext4 jbd2 mbcache sd_mod crc_t10dif hpsa dm_mirror dm_region_hash dm_log dm_mod [last unloaded: ip_tables]
Pid: 311, comm: khugepaged Not tainted 2.6.32-358.18.1.el6.x86_64 #1 HP ProLiant DL560 Gen8
RIP: 0010:[<ffffffffa0065f29>] [<ffffffffa0065f29>] jbd2_journal_grab_journal_head+0x9/0x40 [jbd2]
RSP: 0018:ffff881027c6d5f0 EFLAGS: 00010286
RAX: 0000000000000000 RBX: 3836343030303030 RCX: 00c0000000000809
RDX: 4220313030b63634 RSI: ffffea0044edb9d8 RDI: 3836343030303030
RBP: ffff881027c6d5f0 R08: ffffea0044edba00 R09: 00c000000000082c
R10: ffffea006422acd0 R11: 0000000000000000 R12: 0000000000000000
R13: ffff881c5e4542d0 R14: ffffea0044edb9d8 R15: ffff880fd664539c
FS: 0000000000000000(0000) GS:ffff882078820000(0000) knlGS:0000000000000000
CS: 0010 DS: 0018 ES: 0018 CR0: 000000008005003b
CR2: 00007fbd31519010 CR3: 0000000001a85000 CR4: 00000000000407e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
Process khugepaged (pid: 311, threadinfo ffff881027c6c000, task ffff881027c4e080)
Stack:
ffff881027c6d640 ffffffffa005ccf8 ffff881027c6d798 00000000eda61df8
<d> ffff881ceda61de0 ffffea0044edb9d8 ffff880fd6645000 ffffea0044edba00
<d> ffffea0044edb9d8 0000000000000001 ffff881027c6d670 ffffffffa00828b5
Call Trace:
[<ffffffffa005ccf8>] jbd2_journal_try_to_free_buffers+0x48/0x150 [jbd2]
[<ffffffffa00828b5>] ext4_releasepage+0x55/0x90 [ext4]
[<ffffffff81119100>] try_to_release_page+0x30/0x60
[<ffffffff811328e7>] shrink_page_list.clone.3+0x517/0x650
[<ffffffff81133403>] shrink_inactive_list+0x343/0x830
[<ffffffff8112fe41>] ? lru_cache_add_lru+0x21/0x40
[<ffffffff81131e2f>] ? putback_lru_page+0x8f/0x100
[<ffffffff81170ae0>] ? mem_cgroup_get_reclaim_stat_from_page+0x20/0x70
[<ffffffff81133c9e>] shrink_mem_cgroup_zone+0x3ae/0x610
[<ffffffff8117286d>] ? mem_cgroup_iter+0xfd/0x280
[<ffffffff81133f63>] shrink_zone+0x63/0xb0
[<ffffffff811340c5>] do_try_to_free_pages+0x115/0x610
[<ffffffff81134792>] try_to_free_pages+0x92/0x120
[<ffffffff8112bf78>] __alloc_pages_nodemask+0x478/0x8d0
[<ffffffff81160d3a>] alloc_pages_vma+0x9a/0x150
[<ffffffff8117a1f7>] khugepaged+0x767/0x1310
[<ffffffff81096da0>] ? autoremove_wake_function+0x0/0x40
[<ffffffff81179a90>] ? khugepaged+0x0/0x1310
[<ffffffff81096a36>] kthread+0x96/0xa0
[<ffffffff8100c0ca>] child_rip+0xa/0x20
[<ffffffff810969a0>] ? kthread+0x0/0xa0
[<ffffffff8100c0c0>] ? child_rip+0x0/0x20
Code: fd ff ff eb 94 4c 89 e7 e8 25 05 15 e1 0f 1f 44 00 00 eb c6 66 66 66 66 66 2e 0f 1f 84 00 00 00 00 00 55 48 89 e5 0f 1f 44 00 00 <f0> 0f ba 2f 17 19 c0 85 c0 75 1c 48 8b 17 31 c0 f7 c2 00 00 01
RIP [<ffffffffa0065f29>] jbd2_journal_grab_journal_head+0x9/0x40 [jbd2]
RSP <ffff881027c6d5f0>
Environment
- Red Hat Enterprise Linux 6.4
- ext4/jbd2
Subscriber exclusive content
A Red Hat subscription provides unlimited access to our knowledgebase of over 48,000 articles and solutions.
Welcome! Check out the Getting Started with Red Hat page for quick tours and guides for common tasks.
