RHEL6.3: kernel crashes in mpage_put_bnr_to_bhs while running db2 restore on 28GB ext4

Solution Unverified - Updated -

Issue

  • Kernel crash in ext4 code after applying kernel patch to fix EXT4 filesystem corruption after unmounting
  • Kernel crashes while running db2 restore of a 28GB file. The restore gets through around 10GB and then the system crashes.
  • When using a 4 piece 28 GB total backup it fails after 10 GB, but it does not fail as 1 single 28 GB file
  • Machine crash is repeatable every time the db2 restore is attempted.
  • Kernel crashes in mpage_put_bnr_to_bhs
------------[ cut here ]------------
kernel BUG at fs/ext4/inode.c:2185!
invalid opcode: 0000 [#1] SMP 
last sysfs file: /sys/module/ipv6/initstate
CPU 0 
Modules linked in: ext3 jbd bridge stp llc sunrpc ipv6 vsock(U) vmsync(U) vmware_balloon vmblock(U) fuse ppdev parport_pc parport e1000 microco
de vmci(U) i2c_piix4 i2c_core sg shpchp ext4 mbcache jbd2 sd_mod crc_t10dif sr_mod cdrom vmw_pvscsi pata_acpi ata_generic ata_piix dm_multipath
 dm_mirror dm_region_hash dm_log dm_mod [last unloaded: vmware_balloon]

Pid: 5266, comm: db2sysc Not tainted 2.6.32-279.5.2.el6.x86_64 #1 VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform
RIP: 0010:[<ffffffffa00aee43>]  [<ffffffffa00aee43>] mpage_put_bnr_to_bhs+0x223/0x2a0 [ext4]
RSP: 0018:ffff8801f7425948  EFLAGS: 00010212
RAX: 000000000000000e RBX: 0000000000596ef0 RCX: ffff8802363b3c00
RDX: ffff88018da3ecf8 RSI: ffff88023683b200 RDI: 0000000000000000
RBP: ffff8801f7425a18 R08: 0000000000596ef1 R09: ffff88018da3ecf8
R10: 0000000000000006 R11: ffff8801f74259b0 R12: 00000000008f8ef0
R13: 0000000000596f00 R14: ffff8801c4883cc0 R15: 0000000000596eea
FS:  00007fffe1bfe700(0000) GS:ffff880028200000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007ffff4420158 CR3: 000000020e9b9000 CR4: 00000000000006f0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
Process db2sysc (pid: 5266, threadinfo ffff8801f7424000, task ffff8801f7584040)
Stack:
 0000000000596eea ffff8801f7425980 ffff8801c4883de0 0000000000596eff
<d> 000000000000000e 0000000000000000 ffffea0005347088 ffffea00053470c0
<d> ffffea00053470f8 ffffea0005347130 ffffea0005347168 ffffea00053471a0
Call Trace:
 [<ffffffffa00b3dd7>] mpage_da_map_and_submit+0x297/0x450 [ext4]
 [<ffffffffa00b47de>] ext4_da_writepages+0x2ee/0x620 [ext4]
 [<ffffffff81129c61>] do_writepages+0x21/0x40
 [<ffffffff81114b7b>] __filemap_fdatawrite_range+0x5b/0x60
 [<ffffffff81114bda>] filemap_write_and_wait_range+0x5a/0x90
 [<ffffffff811aa1fe>] vfs_fsync_range+0x7e/0xe0
 [<ffffffff811aa2ab>] generic_write_sync+0x4b/0x50
 [<ffffffff811169be>] generic_file_aio_write+0xbe/0xe0
 [<ffffffffa00ab131>] ext4_file_write+0x61/0x1e0 [ext4]
 [<ffffffff8117afea>] do_sync_write+0xfa/0x140
 [<ffffffff810920d0>] ? autoremove_wake_function+0x0/0x40
 [<ffffffff81012bd9>] ? read_tsc+0x9/0x20
 [<ffffffff8109ce29>] ? ktime_get_ts+0xa9/0xe0
 [<ffffffff81213386>] ? security_file_permission+0x16/0x20
 [<ffffffff8117b2e8>] vfs_write+0xb8/0x1a0
 [<ffffffff810d6c62>] ? audit_syscall_entry+0x272/0x2a0
 [<ffffffff8117bdc2>] sys_pwrite64+0x82/0xa0
 [<ffffffff8100b0f2>] system_call_fastpath+0x16/0x1b
Code: b1 10 01 00 00 48 39 72 30 75 72 83 e7 01 74 10 f0 80 62 01 fd 4c 89 62 18 eb ad 0f 1f 44 00 00 f0 80 62 01 bf 4c 3b 62 18 74 9d <0f> 0b eb fe 66 0f 1f 84 00 00 00 00 00 8b 95 50 ff ff ff 85 d2 
RIP  [<ffffffffa00aee43>] mpage_put_bnr_to_bhs+0x223/0x2a0 [ext4]
 RSP <ffff8801f7425948>

Environment

  • Red Hat Enterprise Linux 6.3
    • kernel-2.6.32-279.5.2.el6.x86_64
  • ext4
  • large files (28 GB) being backed up by splitting into 4 smaller files

Subscriber exclusive content

A Red Hat subscription provides unlimited access to our knowledgebase, tools, and much more.

Current Customers and Partners

Log in for full access

Log In

New to Red Hat?

Learn more about Red Hat subscriptions

Using a Red Hat product through a public cloud?

How to access this content