Server crashed with "WARNING: at fs/buffer.c:1161 mark_buffer_dirty+0x82/0xa0() (Tainted: P)"
Issue
During the HA test(Hitachi HDLM), system was hang
There is some log in /var/log/messages
Feb 19 18:50:36 hostname kernel: __ratelimit: 94 callbacks suppressed
Feb 19 18:50:36 hostname kernel: Buffer I/O error on device dm-3, logical block 268435455
Feb 19 18:50:36 hostname kernel: Buffer I/O error on device dm-3, logical block 268435455
Feb 19 18:50:36 hostname kernel: Buffer I/O error on device dm-3, logical block 268435423
Feb 19 22:17:04 hostname kernel: lost page write due to I/O error on dm-3
Feb 19 22:17:04 hostname kernel: EXT4-fs error (device dm-3) in ext4_new_inode: IO failure
Feb 19 22:17:04 hostname kernel: EXT4-fs (dm-3): previous I/O error to superblock detected
Feb 19 22:17:04 hostname kernel: Buffer I/O error on device dm-3, logical block 0
Feb 19 22:17:04 hostname kernel: lost page write due to I/O error on dm-3
Feb 19 22:29:32 hostname kernel: ------------[ cut here ]------------
Feb 19 22:29:32 hostname kernel: WARNING: at fs/buffer.c:1161 mark_buffer_dirty+0x82/0xa0() (Tainted: P --------------- )
Feb 19 22:29:32 hostname kernel: Hardware name: System x3650 M4 -[7915OSF]-
Feb 19 22:29:32 hostname kernel: Modules linked in: fuse snapapi26(P)(U) bonding 8021q garp stp llc ipv6 sddlmfdrv(P)(U) sddlmadrv(P)(U) uinput microcode osst st cdc_ether usbnet mii i2c_i801 i2c_core iTCO_wdt iTCO_vendor_support shpchp ioatdma igb dca sg ext4 mbcache jbd2 sr_mod cdrom sd_mod crc_t10dif usb_storage ahci megaraid_sas lpfc scsi_transport_fc scsi_tgt dm_mirror dm_region_hash dm_log dm_mod [last unloaded: scsi_wait_scan]
Feb 19 22:29:32 hostname kernel: Pid: 7037, comm: jbd2/dm-3-8 Tainted: P --------------- 2.6.32-279.el6.x86_64 #1
Feb 19 22:29:32 hostname kernel: Call Trace:
Feb 19 22:29:32 hostname kernel: [<ffffffff8106b747>] ? warn_slowpath_common+0x87/0xc0
Feb 19 22:29:32 hostname kernel: [<ffffffff8106b79a>] ? warn_slowpath_null+0x1a/0x20
Feb 19 22:29:32 hostname kernel: [<ffffffff811adb62>] ? mark_buffer_dirty+0x82/0xa0
Feb 19 22:29:32 hostname kernel: [<ffffffffa01522d5>] ? __jbd2_journal_temp_unlink_buffer+0xa5/0x140 [jbd2]
Feb 19 22:29:32 hostname kernel: [<ffffffffa0152386>] ? __jbd2_journal_unfile_buffer+0x16/0x30 [jbd2]
Feb 19 22:29:32 hostname kernel: [<ffffffffa01526e8>] ? __jbd2_journal_refile_buffer+0xc8/0xf0 [jbd2]
Feb 19 22:29:32 hostname kernel: [<ffffffffa0155360>] ? jbd2_journal_commit_transaction+0xce0/0x14b0 [jbd2]
Feb 19 22:29:32 hostname kernel: [<ffffffff810096f0>] ? __switch_to+0xd0/0x320
Feb 19 22:29:32 hostname kernel: [<ffffffff8107eabb>] ? try_to_del_timer_sync+0x7b/0xe0
Feb 19 22:29:32 hostname kernel: [<ffffffffa015af78>] ? kjournald2+0xb8/0x220 [jbd2]
Feb 19 22:29:32 hostname kernel: [<ffffffff810920d0>] ? autoremove_wake_function+0x0/0x40
Feb 19 22:29:32 hostname kernel: [<ffffffffa015aec0>] ? kjournald2+0x0/0x220 [jbd2]
Feb 19 22:29:32 hostname kernel: [<ffffffff81091d66>] ? kthread+0x96/0xa0
Feb 19 22:29:32 hostname kernel: [<ffffffff8100c14a>] ? child_rip+0xa/0x20
Feb 19 22:29:32 hostname kernel: [<ffffffff81091cd0>] ? kthread+0x0/0xa0
Feb 19 22:29:32 hostname kernel: [<ffffffff8100c140>] ? child_rip+0x0/0x20
Feb 19 22:29:32 hostname kernel: ---[ end trace 5b4bf20c2d10fe87 ]---
Feb 20 11:36:02 hostname bonobo-activation-server (root-13140): could not associate with desktop session: Failed to connect to socket /tmp/dbus-xrNas7FGRC: Connection refused:
Environment
O/S : Red Hat Enterprise Linux 6.3 (after heter RHEL6.3)
Kernel : 2.6.32-279.el6
Hardware : IBM System x3650 M3
Storage : Hitachi AMS2300
Multipath : Hitachi HDLM ( no multipath provided by RedHat )
Subscriber exclusive content
A Red Hat subscription provides unlimited access to our knowledgebase, tools, and much more.