System became unresponsive after "WARNING: at lib/list_debug.c:26 __list_add+0x6d/0xa0()".

Issue

• System became unresponsive after one of the following trace:

WARNING: at lib/list_debug.c:26 __list_add+0x6d/0xa0() (Tainted: P           ---------------   )
Hardware name: PowerEdge R810
list_add corruption. next->prev should be prev (ffffffff81ad26e0), but was ffff8806a5c10998. (next=ffff880294437d98).
Modules linked in: mpt2sas scsi_transport_sas raid_class mptctl mptbase ipmi_devintf dell_rbu cvfs(P)(U) autofs4 fcoe libfcoe libfc sunrpc bonding 8021q garp stp llc ipv6 xfs exportfs emcpvlumd(P)(U) emcpxcrypt(P)(U) emcpdm(P)(U) emcpgpx(P)(U) emcpmpx(P)(U) emcp(P)(U) dm_mirror dm_region_hash dm_log power_meter dcdbas serio_raw iTCO_wdt iTCO_vendor_support i7core_edac edac_core ses enclosure sg bnx2 ext4 mbcache jbd2 sr_mod cdrom usb_storage sd_mod crc_t10dif pata_acpi ata_generic ata_piix qla2xxx scsi_transport_fc scsi_tgt megaraid_sas dm_mod [last unloaded: microcode]
Pid: 22092, comm: record Tainted: P           ---------------    2.6.32-358.el6.x86_64 #1
Call Trace:
[<ffffffff8106e2e7>] ? warn_slowpath_common+0x87/0xc0
[<ffffffff8106e3d6>] ? warn_slowpath_fmt+0x46/0x50
[<ffffffff81288e7d>] ? __list_add+0x6d/0xa0
[<ffffffff8119ddb4>] ? iget_locked+0x114/0x170
[<ffffffff811e9936>] ? proc_get_inode+0x16/0x130
[<ffffffff811f0421>] ? proc_lookup_de+0x91/0x110
[<ffffffff811f04bb>] ? proc_lookup+0x1b/0x20
[<ffffffff811e9bf7>] ? proc_root_lookup+0x27/0x50
[<ffffffff81190405>] ? do_lookup+0x1a5/0x230
[<ffffffff81190bc4>] ? __link_path_walk+0x734/0x1030
[<ffffffff81096c80>] ? autoremove_wake_function+0x0/0x40
[<ffffffff8119174a>] ? path_walk+0x6a/0xe0
[<ffffffff8119191b>] ? do_path_lookup+0x5b/0xa0
[<ffffffff81182460>] ? get_empty_filp+0xa0/0x180
[<ffffffff8119285b>] ? do_filp_open+0xfb/0xdd0
[<ffffffff814a83cc>] ? udp_ioctl+0x4c/0x90
[<ffffffff8119f562>] ? alloc_fd+0x92/0x160
[<ffffffff8117de79>] ? do_sys_open+0x69/0x140
[<ffffffff8117df90>] ? sys_open+0x20/0x30
[<ffffffff8100b072>] ? system_call_fastpath+0x16/0x1b

• More call traces from messages file :

 rpc.mountd[38141]: Version 1.2.3 starting
 kernel: ------------[ cut here ]------------
 kernel: WARNING: at lib/list_debug.c:26 __list_add+0x6d/0xa0() (Not tainted)
 kernel: Hardware name: ProLiant BL460c Gen8
 kernel: list_add corruption. next->prev should be prev (ffffffff81ad26e0), but was ffff88101b376bc8. (next=ffff881e3b54d198).
 kernel: Modules linked in: nfsd lockd nfs_acl auth_rpcgss nls_utf8 ipt_REJECT xt_multiport xt_state xt_limit iptable_nat nf_nat nf_conntrack_ipv4 nf_conntrack nf_defrag_ipv4 iptable_filter ip_tables bridge autofs4 sunrpc bonding ipv6 dm_round_robin scsi_dh_emc dm_multipath xfs exportfs 8021q garp stp llc hpilo hpwdt microcode power_meter sg bnx2x libcrc32c mdio serio_raw iTCO_wdt iTCO_vendor_support ioatdma dca shpchp ext4 mbcache jbd2 sd_mod crc_t10dif qla2xxx scsi_transport_fc scsi_tgt hpsa video output dm_mirror dm_region_hash dm_log dm_mod [last unloaded: scsi_wait_scan]
 kernel: Pid: 41147, comm: dnssec-keygen Not tainted 2.6.32-358.6.2.el6.x86_64 #1
 kernel: Call Trace:
 kernel: [<ffffffff8106e2e7>] ? warn_slowpath_common+0x87/0xc0
 kernel: [<ffffffff8106e3d6>] ? warn_slowpath_fmt+0x46/0x50
 kernel: [<ffffffffa02cadb5>] ? xfs_buf_rele+0x55/0x100 [xfs]
 kernel: [<ffffffff8128900d>] ? __list_add+0x6d/0xa0
 kernel: [<ffffffff8119def1>] ? inode_add_to_lists+0x61/0xc0
 kernel: [<ffffffffa02d1eb0>] ? xfs_setup_inode+0x40/0x260 [xfs]
 kernel: [<ffffffffa02a713c>] ? xfs_iget+0x49c/0x6e0 [xfs]
 kernel: [<ffffffffa02a9bc4>] ? xfs_ialloc+0x94/0x6e0 [xfs]
 kernel: [<ffffffffa02c24c4>] ? xfs_dir_ialloc+0x74/0x2b0 [xfs]
 kernel: [<ffffffffa02c45a0>] ? xfs_create+0x440/0x640 [xfs]
 kernel: [<ffffffffa02d174d>] ? xfs_vn_mknod+0xad/0x1c0 [xfs]
 kernel: [<ffffffffa02d1890>] ? xfs_vn_create+0x10/0x20 [xfs]
 kernel: [<ffffffff8118f8a4>] ? vfs_create+0xb4/0xe0
 kernel: [<ffffffff811933fb>] ? do_filp_open+0xbbb/0xdd0
 kernel: [<ffffffff8104757c>] ? __do_page_fault+0x1ec/0x480
 kernel: [<ffffffff8119f5e2>] ? alloc_fd+0x92/0x160
 kernel: [<ffffffff8117df59>] ? do_sys_open+0x69/0x140
 kernel: [<ffffffff8117e070>] ? sys_open+0x20/0x30
 kernel: [<ffffffff8100b072>] ? system_call_fastpath+0x16/0x1b
 kernel: ---[ end trace 88bcecf7ac9c3703 ]---

 kernel: ------------[ cut here ]------------
 kernel: WARNING: at lib/list_debug.c:26 __list_add+0x6d/0xa0() (Tainted: G        W  ---------------   )
 kernel: Hardware name: ProLiant BL460c Gen8
 kernel: list_add corruption. next->prev should be prev (ffff88101b376bc8), but was ffff881e3b54d598. (next=ffff881e3b54d198).
 kernel: Modules linked in: nfsd lockd nfs_acl auth_rpcgss nls_utf8 ipt_REJECT xt_multiport xt_state xt_limit iptable_nat nf_nat nf_conntrack_ipv4 nf_conntrack nf_defrag_ipv4 iptable_filter ip_tables bridge autofs4 sunrpc bonding ipv6 dm_round_robin scsi_dh_emc dm_multipath xfs exportfs 8021q garp stp llc hpilo hpwdt microcode power_meter sg bnx2x libcrc32c mdio serio_raw iTCO_wdt iTCO_vendor_support ioatdma dca shpchp ext4 mbcache jbd2 sd_mod crc_t10dif qla2xxx scsi_transport_fc scsi_tgt hpsa video output dm_mirror dm_region_hash dm_log dm_mod [last unloaded: scsi_wait_scan]
 kernel: Pid: 41147, comm: dnssec-keygen Tainted: G        W  ---------------    2.6.32-358.6.2.el6.x86_64 #1
 kernel: Call Trace:
 kernel: [<ffffffff8106e2e7>] ? warn_slowpath_common+0x87/0xc0
 kernel: [<ffffffff8106e3d6>] ? warn_slowpath_fmt+0x46/0x50
 kernel: [<ffffffff8128900d>] ? __list_add+0x6d/0xa0
 kernel: [<ffffffff811abbef>] ? list_move+0x1f/0x30
 kernel: [<ffffffff811ac2bf>] ? __mark_inode_dirty+0x13f/0x160
 kernel: [<ffffffff8119c3d2>] ? file_update_time+0xf2/0x170
 kernel: [<ffffffffa02ce03b>] ? xfs_file_aio_write_checks+0xdb/0xf0 [xfs]
 kernel: [<ffffffffa02ce349>] ? xfs_file_buffered_aio_write+0x79/0x1b0 [xfs]
 kernel: [<ffffffff81182fd0>] ? __sb_start_write+0x80/0x120
 kernel: [<ffffffffa02ce57f>] ? xfs_file_aio_write+0xff/0x190 [xfs]
 kernel: [<ffffffff81180d7a>] ? do_sync_write+0xfa/0x140
 kernel: [<ffffffff81096ca0>] ? autoremove_wake_function+0x0/0x40
 kernel: [<ffffffff8114a88a>] ? do_mmap_pgoff+0x33a/0x380
 kernel: [<ffffffff8121bc36>] ? security_file_permission+0x16/0x20
 kernel: [<ffffffff81181078>] ? vfs_write+0xb8/0x1a0
 kernel: [<ffffffff81181971>] ? sys_write+0x51/0x90
 kernel: [<ffffffff8100b072>] ? system_call_fastpath+0x16/0x1b
 kernel: ---[ end trace 88bcecf7ac9c3704 ]---

 kernel: ------------[ cut here ]------------
 kernel: WARNING: at lib/list_debug.c:26 __list_add+0x6d/0xa0() (Tainted: G        W  ---------------   )
 kernel: Hardware name: ProLiant BL460c Gen8
 kernel: list_add corruption. next->prev should be prev (ffffffff81ad26e0), but was ffff88101b376bc8. (next=ffff881e3b54d598).
 kernel: Modules linked in: nfsd lockd nfs_acl auth_rpcgss nls_utf8 ipt_REJECT xt_multiport xt_state xt_limit iptable_nat nf_nat nf_conntrack_ipv4 nf_conntrack nf_defrag_ipv4 iptable_filter ip_tables bridge autofs4 sunrpc bonding ipv6 dm_round_robin scsi_dh_emc dm_multipath xfs exportfs 8021q garp stp llc hpilo hpwdt microcode power_meter sg bnx2x libcrc32c mdio serio_raw iTCO_wdt iTCO_vendor_support ioatdma dca shpchp ext4 mbcache jbd2 sd_mod crc_t10dif qla2xxx scsi_transport_fc scsi_tgt hpsa video output dm_mirror dm_region_hash dm_log dm_mod [last unloaded: scsi_wait_scan]
 kernel: Pid: 40736, comm: dnsServerConfig Tainted: G        W  ---------------    2.6.32-358.6.2.el6.x86_64 #1
 kernel: Call Trace:
 kernel: [<ffffffff8106e2e7>] ? warn_slowpath_common+0x87/0xc0
 kernel: [<ffffffff8106e3d6>] ? warn_slowpath_fmt+0x46/0x50
 kernel: [<ffffffff8128900d>] ? __list_add+0x6d/0xa0
 kernel: [<ffffffff8119dfa9>] ? new_inode+0x59/0xb0
 kernel: [<ffffffff8118b165>] ? create_write_pipe+0x35/0x1f0
 kernel: [<ffffffff8118b377>] ? do_pipe_flags+0x57/0x140
 kernel: [<ffffffff8118b484>] ? sys_pipe2+0x24/0x80
 kernel: [<ffffffff8118b4f0>] ? sys_pipe+0x10/0x20
 kernel: [<ffffffff8100b072>] ? system_call_fastpath+0x16/0x1b
 kernel: ---[ end trace 88bcecf7ac9c3705 ]---

 kernel: ------------[ cut here ]------------
 kernel: WARNING: at lib/list_debug.c:26 __list_add+0x6d/0xa0() (Tainted: G        W  ---------------   )
 kernel: Hardware name: ProLiant BL460c Gen8
 kernel: list_add corruption. next->prev should be prev (ffff88101b376bc8), but was ffffffff81ad26e0. (next=ffff881e3b54d598).
 kernel: Modules linked in: nfsd lockd nfs_acl auth_rpcgss nls_utf8 ipt_REJECT xt_multiport xt_state xt_limit iptable_nat nf_nat nf_conntrack_ipv4 nf_conntrack nf_defrag_ipv4 iptable_filter ip_tables bridge autofs4 sunrpc bonding ipv6 dm_round_robin scsi_dh_emc dm_multipath xfs exportfs 8021q garp stp llc hpilo hpwdt microcode power_meter sg bnx2x libcrc32c mdio serio_raw iTCO_wdt iTCO_vendor_support ioatdma dca shpchp ext4 mbcache jbd2 sd_mod crc_t10dif qla2xxx scsi_transport_fc scsi_tgt hpsa video output dm_mirror dm_region_hash dm_log dm_mod [last unloaded: scsi_wait_scan]
 kernel: Pid: 41175, comm: echo Tainted: G        W  ---------------    2.6.32-358.6.2.el6.x86_64 #1
 kernel: Call Trace:
 kernel: [<ffffffff8106e2e7>] ? warn_slowpath_common+0x87/0xc0
 kernel: [<ffffffff8106e3d6>] ? warn_slowpath_fmt+0x46/0x50
 kernel: [<ffffffff8128900d>] ? __list_add+0x6d/0xa0
 kernel: [<ffffffff811abbef>] ? list_move+0x1f/0x30
 kernel: [<ffffffff811ac2bf>] ? __mark_inode_dirty+0x13f/0x160
 kernel: [<ffffffff8119c3d2>] ? file_update_time+0xf2/0x170
 kernel: [<ffffffffa02ce03b>] ? xfs_file_aio_write_checks+0xdb/0xf0 [xfs]
 kernel: [<ffffffffa02ce349>] ? xfs_file_buffered_aio_write+0x79/0x1b0 [xfs]
 kernel: [<ffffffff81182fd0>] ? __sb_start_write+0x80/0x120
 kernel: [<ffffffffa02ce57f>] ? xfs_file_aio_write+0xff/0x190 [xfs]
 kernel: [<ffffffff81180d7a>] ? do_sync_write+0xfa/0x140
 kernel: [<ffffffff81096ca0>] ? autoremove_wake_function+0x0/0x40
 kernel: [<ffffffff8114a88a>] ? do_mmap_pgoff+0x33a/0x380
 kernel: [<ffffffff8121bc36>] ? security_file_permission+0x16/0x20
 kernel: [<ffffffff81181078>] ? vfs_write+0xb8/0x1a0
 kernel: [<ffffffff81181971>] ? sys_write+0x51/0x90
 kernel: [<ffffffff8100b072>] ? system_call_fastpath+0x16/0x1b
 kernel: ---[ end trace 88bcecf7ac9c3706 ]---

 kernel: ------------[ cut here ]------------
 kernel: WARNING: at lib/list_debug.c:26 __list_add+0x6d/0xa0() (Tainted: G        W  ---------------   )
 kernel: Hardware name: ProLiant BL460c Gen8
 kernel: list_add corruption. next->prev should be prev (ffffffff81ad26e0), but was ffff880cbf600998. (next=ffff881e3b54d598).
 kernel: Modules linked in: nfsd lockd nfs_acl auth_rpcgss nls_utf8 ipt_REJECT xt_multiport xt_state xt_limit iptable_nat nf_nat nf_conntrack_ipv4 nf_conntrack nf_defrag_ipv4 iptable_filter ip_tables bridge autofs4 sunrpc bonding ipv6 dm_round_robin scsi_dh_emc dm_multipath xfs exportfs 8021q garp stp llc hpilo hpwdt microcode power_meter sg bnx2x libcrc32c mdio serio_raw iTCO_wdt iTCO_vendor_support ioatdma dca shpchp ext4 mbcache jbd2 sd_mod crc_t10dif qla2xxx scsi_transport_fc scsi_tgt hpsa video output dm_mirror dm_region_hash dm_log dm_mod [last unloaded: scsi_wait_scan]
 kernel: Pid: 41182, comm: mv Tainted: G        W  ---------------    2.6.32-358.6.2.el6.x86_64 #1
 kernel: Call Trace:
 kernel: [<ffffffff8106e2e7>] ? warn_slowpath_common+0x87/0xc0
 kernel: [<ffffffff8106e3d6>] ? warn_slowpath_fmt+0x46/0x50
 kernel: [<ffffffff8128900d>] ? __list_add+0x6d/0xa0
 kernel: [<ffffffff8119de34>] ? iget_locked+0x114/0x170
 kernel: [<ffffffff811e99b6>] ? proc_get_inode+0x16/0x130
 kernel: [<ffffffff811f04a1>] ? proc_lookup_de+0x91/0x110
 kernel: [<ffffffff811f053b>] ? proc_lookup+0x1b/0x20
 kernel: [<ffffffff811e9c77>] ? proc_root_lookup+0x27/0x50
 kernel: [<ffffffff811904e5>] ? do_lookup+0x1a5/0x230
 kernel: [<ffffffff81190ca4>] ? __link_path_walk+0x734/0x1030
 kernel: [<ffffffff81143767>] ? handle_pte_fault+0xf7/0xb50
 kernel: [<ffffffff8119182a>] ? path_walk+0x6a/0xe0
 kernel: [<ffffffff811919fb>] ? do_path_lookup+0x5b/0xa0
 kernel: [<ffffffff81182540>] ? get_empty_filp+0xa0/0x180
 kernel: [<ffffffff8119293b>] ? do_filp_open+0xfb/0xdd0
 kernel: [<ffffffff8104757c>] ? __do_page_fault+0x1ec/0x480
 kernel: [<ffffffff8119f5e2>] ? alloc_fd+0x92/0x160
 kernel: [<ffffffff8117df59>] ? do_sys_open+0x69/0x140
 kernel: [<ffffffff8117e070>] ? sys_open+0x20/0x30
 kernel: [<ffffffff8100b072>] ? system_call_fastpath+0x16/0x1b
 kernel: ---[ end trace 88bcecf7ac9c3707 ]---