Server crashed with "WARNING: at fs/buffer.c:1161 mark_buffer_dirty+0x82/0xa0() (Tainted: P)"
Issue
During the HA test(Hitachi HDLM), system was hang
There is some log in /var/log/messages
Feb 19 18:50:36 hostname kernel: __ratelimit: 94 callbacks suppressed
Feb 19 18:50:36 hostname kernel: Buffer I/O error on device dm-3, logical block 268435455
Feb 19 18:50:36 hostname kernel: Buffer I/O error on device dm-3, logical block 268435455
Feb 19 18:50:36 hostname kernel: Buffer I/O error on device dm-3, logical block 268435423
Feb 19 22:17:04 hostname kernel: lost page write due to I/O error on dm-3
Feb 19 22:17:04 hostname kernel: EXT4-fs error (device dm-3) in ext4_new_inode: IO failure
Feb 19 22:17:04 hostname kernel: EXT4-fs (dm-3): previous I/O error to superblock detected
Feb 19 22:17:04 hostname kernel: Buffer I/O error on device dm-3, logical block 0
Feb 19 22:17:04 hostname kernel: lost page write due to I/O error on dm-3
Feb 19 22:29:32 hostname kernel: ------------[ cut here ]------------
Feb 19 22:29:32 hostname kernel: WARNING: at fs/buffer.c:1161 mark_buffer_dirty+0x82/0xa0() (Tainted: P --------------- )
Feb 19 22:29:32 hostname kernel: Hardware name: System x3650 M4 -[7915OSF]-
Feb 19 22:29:32 hostname kernel: Modules linked in: fuse snapapi26(P)(U) bonding 8021q garp stp llc ipv6 sddlmfdrv(P)(U) sddlmadrv(P)(U) uinput microcode osst st cdc_ether usbnet mii i2c_i801 i2c_core iTCO_wdt iTCO_vendor_support shpchp ioatdma igb dca sg ext4 mbcache jbd2 sr_mod cdrom sd_mod crc_t10dif usb_storage ahci megaraid_sas lpfc scsi_transport_fc scsi_tgt dm_mirror dm_region_hash dm_log dm_mod [last unloaded: scsi_wait_scan]
Feb 19 22:29:32 hostname kernel: Pid: 7037, comm: jbd2/dm-3-8 Tainted: P --------------- 2.6.32-279.el6.x86_64 #1
Feb 19 22:29:32 hostname kernel: Call Trace:
Feb 19 22:29:32 hostname kernel: [<ffffffff8106b747>] ? warn_slowpath_common+0x87/0xc0
Feb 19 22:29:32 hostname kernel: [<ffffffff8106b79a>] ? warn_slowpath_null+0x1a/0x20
Feb 19 22:29:32 hostname kernel: [<ffffffff811adb62>] ? mark_buffer_dirty+0x82/0xa0
Feb 19 22:29:32 hostname kernel: [<ffffffffa01522d5>] ? __jbd2_journal_temp_unlink_buffer+0xa5/0x140 [jbd2]
Feb 19 22:29:32 hostname kernel: [<ffffffffa0152386>] ? __jbd2_journal_unfile_buffer+0x16/0x30 [jbd2]
Feb 19 22:29:32 hostname kernel: [<ffffffffa01526e8>] ? __jbd2_journal_refile_buffer+0xc8/0xf0 [jbd2]
Feb 19 22:29:32 hostname kernel: [<ffffffffa0155360>] ? jbd2_journal_commit_transaction+0xce0/0x14b0 [jbd2]
Feb 19 22:29:32 hostname kernel: [<ffffffff810096f0>] ? __switch_to+0xd0/0x320
Feb 19 22:29:32 hostname kernel: [<ffffffff8107eabb>] ? try_to_del_timer_sync+0x7b/0xe0
Feb 19 22:29:32 hostname kernel: [<ffffffffa015af78>] ? kjournald2+0xb8/0x220 [jbd2]
Feb 19 22:29:32 hostname kernel: [<ffffffff810920d0>] ? autoremove_wake_function+0x0/0x40
Feb 19 22:29:32 hostname kernel: [<ffffffffa015aec0>] ? kjournald2+0x0/0x220 [jbd2]
Feb 19 22:29:32 hostname kernel: [<ffffffff81091d66>] ? kthread+0x96/0xa0
Feb 19 22:29:32 hostname kernel: [<ffffffff8100c14a>] ? child_rip+0xa/0x20
Feb 19 22:29:32 hostname kernel: [<ffffffff81091cd0>] ? kthread+0x0/0xa0
Feb 19 22:29:32 hostname kernel: [<ffffffff8100c140>] ? child_rip+0x0/0x20
Feb 19 22:29:32 hostname kernel: ---[ end trace 5b4bf20c2d10fe87 ]---
Feb 20 11:36:02 hostname bonobo-activation-server (root-13140): could not associate with desktop session: Failed to connect to socket /tmp/dbus-xrNas7FGRC: Connection refused:
Environment
O/S : Red Hat Enterprise Linux 6.3 (after heter RHEL6.3)
Kernel : 2.6.32-279.el6
Hardware : IBM System x3650 M3
Storage : Hitachi AMS2300
Multipath : Hitachi HDLM ( no multipath provided by RedHat )
Subscriber exclusive content
A Red Hat subscription provides unlimited access to our knowledgebase of over 48,000 articles and solutions.
Welcome! Check out the Getting Started with Red Hat page for quick tours and guides for common tasks.
