System crash with "kernel BUG at drivers/md/raid5.c:(510|527)!"

Solution Unverified - Updated -

Issue

  • System crash with "kernel BUG at drivers/md/raid5.c:510!"
[65940731.114033] kernel BUG at drivers/md/raid5.c:510!
[65940731.114790] invalid opcode: 0000 [#1] SMP 
[65940731.115518] Modules linked in: sctp_diag sctp dccp_diag dccp tcp_diag udp_diag inet_diag unix_diag af_packet_diag netlink_diag dm_mod raid456 async_raid6_recov async_memcpy async_pq rpcsec_gss_krb5 nfsv4 dns_resolver nfs fscache raid6_pq libcrc32c async_xor xor async_tx skx_edac edac_core intel_powerclamp coretemp intel_rapl iosf_mbi kvm_intel kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd ses enclosure pcspkr sg hpilo hpwdt ipmi_si ipmi_devintf ipmi_msghandler mei_me shpchp mei lpc_ich wmi acpi_power_meter nfsd auth_rpcgss nfs_acl lockd grace sunrpc binfmt_misc ip_tables ext4 mbcache jbd2 sd_mod crc_t10dif crct10dif_generic uas usb_storage mgag200 i2c_algo_bit drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm crct10dif_pclmul
[65940731.120018]  crct10dif_common drm crc32c_intel serio_raw smartpqi nvme scsi_transport_sas tg3 nvme_core i2c_core ptp pps_core
[65940731.121508] CPU: 5 PID: 6895 Comm: dataserver Not tainted 3.10.0-693.17.1.el7.x86_64 #1
[65940731.122251] Hardware name: HPE ProLiant DL380 Gen10/ProLiant DL380 Gen10, BIOS U30 11/14/2017
[65940731.122996] task: ffff8809f5ae2f70 ti: ffff883442618000 task.ti: ffff883442618000
[65940731.123735] RIP: 0010:[<ffffffffc044d4a0>]  [<ffffffffc044d4a0>] raid5_get_active_stripe+0x6c0/0x710 [raid456]
[65940731.124482] RSP: 0018:ffff88344261b7a8  EFLAGS: 00010082
[65940731.125214] RAX: 0000000000000000 RBX: ffff882f7f6cd814 RCX: dead000000000200
[65940731.125946] RDX: 0000000000000000 RSI: 0000000000000003 RDI: ffff8801ce53af40
[65940731.126669] RBP: ffff88344261b850 R08: ffff8801ce53af50 R09: 0000000000000400
[65940731.127386] R10: 0000000000000001 R11: 0000000000000000 R12: ffff882f7f6cd800
[65940731.128093] R13: ffff882f7f6cd800 R14: ffff8801ce53af40 R15: 000000000304b998
[65940731.128796] FS:  00007ffff0611700(0000) GS:ffff882f7ff40000(0000) knlGS:0000000000000000
[65940731.129497] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[65940731.130190] CR2: 00007fffc4004780 CR3: 0000005f7d7f4000 CR4: 00000000003607e0
[65940731.130882] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[65940731.131563] DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
[65940731.132231] Call Trace:
[65940731.132889]  [<ffffffffc04490c3>] ? add_stripe_bio+0x1f3/0x770 [raid456]
[65940731.133549]  [<ffffffff812f92d3>] ? blk_check_plugged+0x93/0xb0
[65940731.134201]  [<ffffffffc044d6b5>] raid5_make_request+0x1c5/0xce0 [raid456]
[65940731.134847]  [<ffffffff810b3690>] ? wake_up_atomic_t+0x30/0x30
[65940731.135484]  [<ffffffff81187335>] ? mempool_alloc_slab+0x15/0x20
[65940731.136109]  [<ffffffff8150c644>] md_make_request+0x104/0x290
[65940731.136726]  [<ffffffff812fb015>] generic_make_request+0x105/0x310
[65940731.137336]  [<ffffffff8150c053>] ? md_mergeable_bvec+0x43/0x50
[65940731.137936]  [<ffffffff8123cd18>] ? __bio_add_page+0x208/0x2b0
[65940731.138523]  [<ffffffff812fb290>] submit_bio+0x70/0x150
[65940731.139103]  [<ffffffff816a2941>] ? put_compound_page+0x94/0x174
[65940731.139675]  [<ffffffff812446a8>] do_blockdev_direct_IO+0x1e68/0x2050
[65940731.140240]  [<ffffffff8123ede0>] ? I_BDEV+0x10/0x10
[65940731.140792]  [<ffffffff812448e5>] __blockdev_direct_IO+0x55/0x60
[65940731.141339]  [<ffffffff8123ede0>] ? I_BDEV+0x10/0x10
[65940731.141883]  [<ffffffff8123f6d0>] blkdev_direct_IO+0x60/0xa0
[65940731.142396]  [<ffffffff8123ede0>] ? I_BDEV+0x10/0x10
[65940731.142896]  [<ffffffff8118652d>] generic_file_direct_write+0xcd/0x190
[65940731.143392]  [<ffffffff81186827>] __generic_file_aio_write+0x237/0x400
[65940731.143880]  [<ffffffff812400c6>] blkdev_aio_write+0x56/0xb0
[65940731.144357]  [<ffffffff81240070>] ? bd_acquire+0xe0/0xe0
[65940731.144823]  [<ffffffff81256268>] do_io_submit+0x3b8/0x870
[65940731.145287]  [<ffffffff81256730>] SyS_io_submit+0x10/0x20
[65940731.145751]  [<ffffffff816b89fd>] system_call_fastpath+0x16/0x1b
  • System crash with "kernel BUG at drivers/md/raid5.c:527!"
[1990403.760007] kernel BUG at drivers/md/raid5.c:527!
[1990403.761237] invalid opcode: 0000 [#1] SMP 
[1990403.762407] Modules linked in: tcp_diag inet_diag nf_log_ipv6 nf_log_ipv4 nf_log_common xt_LOG xt_limit xt_pkttype ip6t_REJECT ipt_REJECT nf_reject_ipv6 nf_reject_ipv4 nf_conntrack_ipv6 nf_conntrack_ipv4 nf_defrag_ipv6 nf_defrag_ipv4 xt_comment xt_multiport xt_conntrack nf_conntrack ip6table_filter ip6_tables iptable_filter xfs raid0 raid456 async_raid6_recov async_memcpy async_pq raid6_pq intel_powerclamp coretemp kvm_intel libcrc32c async_xor xor async_tx kvm irqbypass ses enclosure crc32_pclmul ghash_clmulni_intel aesni_intel sg iTCO_wdt iTCO_vendor_support ipmi_devintf lpc_ich sb_edac lrw gf128mul edac_core glue_helper ablk_helper shpchp cryptd mei_me mei pcspkr i2c_i801 ipmi_si ipmi_msghandler wmi openafs(POE) binfmt_misc ip_tables ext4 mbcache jbd2 raid1 sd_mod crc_t10dif crct10dif_generic
[1990403.770154]  ast drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm isci mpt2sas crct10dif_pclmul crct10dif_common crc32c_intel ixgbe libsas raid_class drm igb ahci scsi_transport_sas libahci mdio ptp libata pps_core i2c_algo_bit dca i2c_core fjes dm_mirror dm_region_hash dm_log dm_mod
[1990403.774043] CPU: 5 PID: 23964 Comm: kworker/u65:0 Tainted: P           OE  ------------   3.10.0-514.26.2.el7.x86_64 #1
[1990403.775320] Hardware name: Quanta S810-X52L/S2L, BIOS S2L_4A12 06/24/2014
[1990403.776598] Workqueue: writeback bdi_writeback_workfn (flush-9:117)
[1990403.777853] task: ffff880004010fb0 ti: ffff880075f08000 task.ti: ffff880075f08000
[1990403.779083] RIP: 0010:[<ffffffffa067dd31>]  [<ffffffffa067dd31>] raid5_get_active_stripe+0x711/0x750 [raid456]
[1990403.780342] RSP: 0018:ffff880075f0b6b0  EFLAGS: 00010082
[1990403.781581] RAX: 0000000000000000 RBX: ffff880174653c10 RCX: dead000000000200
[1990403.782803] RDX: 0000000000000000 RSI: 0000000000000002 RDI: ffff88004f6c9cb0
[1990403.784003] RBP: ffff880075f0b758 R08: ffff88004f6c9cc0 R09: 0000000000000040
[1990403.785185] R10: 0000000000000001 R11: 0000000000000000 R12: ffff880174653c00
[1990403.786355] R13: ffff88004f6c9cb0 R14: 0000000000000000 R15: 0000000000000000
[1990403.787526] FS:  0000000000000000(0000) GS:ffff88085fb40000(0000) knlGS:0000000000000000
[1990403.788681] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[1990403.789813] CR2: 00007fac5cd49000 CR3: 00000000019be000 CR4: 00000000001407e0
[1990403.790940] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[1990403.792056] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[1990403.793124] Stack:
[1990403.794185]  ffff88087ffda000 ffff880075f0b6d0 ffffffff81333a8d 0000000022ab9d40
[1990403.795254]  ffff880174653c08 00000001a7572510 ffff880174653de8 ffff880000000002
[1990403.796309]  ffff880174653c00 0000000000000000 ffff88005a731cb0 ffff880075f0b758
[1990403.797371] Call Trace:
[1990403.798396]  [<ffffffff81333a8d>] ? list_del+0xd/0x30
[1990403.799392]  [<ffffffffa067df15>] raid5_make_request+0x1a5/0xd60 [raid456]
[1990403.800389]  [<ffffffff810b1b20>] ? wake_up_atomic_t+0x30/0x30
[1990403.801342]  [<ffffffff814f86dc>] md_make_request+0xdc/0x230
[1990403.802272]  [<ffffffff81183d05>] ? mempool_alloc_slab+0x15/0x20
[1990403.803185]  [<ffffffff812eee69>] generic_make_request+0x109/0x1e0
[1990403.804106]  [<ffffffff812eefb1>] submit_bio+0x71/0x150
[1990403.805053]  [<ffffffffa0751a53>] xfs_submit_ioend_bio.isra.12+0x33/0x40 [xfs]
[1990403.806033]  [<ffffffffa0751b4e>] xfs_submit_ioend+0xee/0x120 [xfs]
[1990403.807000]  [<ffffffffa07524c5>] xfs_do_writepage+0x4e5/0x550 [xfs]
[1990403.807950]  [<ffffffff8118c5d1>] write_cache_pages+0x251/0x4d0
[1990403.808918]  [<ffffffffa0751fe0>] ? xfs_aops_discard_page+0x150/0x150 [xfs]
[1990403.809867]  [<ffffffffa0751df5>] xfs_vm_writepages+0xc5/0xe0 [xfs]
[1990403.810792]  [<ffffffff8118d94e>] do_writepages+0x1e/0x40
[1990403.811740]  [<ffffffff81229010>] __writeback_single_inode+0x40/0x210
[1990403.812659]  [<ffffffff81229cfe>] writeback_sb_inodes+0x25e/0x420
[1990403.813578]  [<ffffffff81229f5f>] __writeback_inodes_wb+0x9f/0xd0
[1990403.814498]  [<ffffffff8122a7a3>] wb_writeback+0x263/0x2f0
[1990403.815395]  [<ffffffff8121905c>] ? get_nr_inodes+0x4c/0x70
[1990403.816265]  [<ffffffff8122c79b>] bdi_writeback_workfn+0x2cb/0x460
[1990403.817115]  [<ffffffff810a845b>] process_one_work+0x17b/0x470
[1990403.817951]  [<ffffffff810a9296>] worker_thread+0x126/0x410
[1990403.818765]  [<ffffffff810a9170>] ? rescuer_thread+0x460/0x460
[1990403.819557]  [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[1990403.820313]  [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[1990403.821054]  [<ffffffff81697758>] ret_from_fork+0x58/0x90
[1990403.821771]  [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[1990403.822469] Code: c4 04 75 a6 0f 0b e8 3f 7d a0 e0 0f 0b 48 8b 55 80 48 c7 c6 c8 82 68 a0 48 c7 c7 90 a0 68 a0 31 c0 e8 54 ed cb e0 e9 45 fc ff ff <0f> 0b 0f 0b 48 c7 c6 31 75 68 a0 48 c7 c7 68 a0 68 a0 31 c0 e8 
[1990403.823948] RIP  [<ffffffffa067dd31>] raid5_get_active_stripe+0x711/0x750 [raid456]
[1990403.824671]  RSP <ffff880075f0b6b0>

Environment

  • Red Hat Enterprise Linux 7

Subscriber exclusive content

A Red Hat subscription provides unlimited access to our knowledgebase, tools, and much more.

Current Customers and Partners

Log in for full access

Log In

New to Red Hat?

Learn more about Red Hat subscriptions

Using a Red Hat product through a public cloud?

How to access this content