RAID controller is hanging and Linux RAID 5 is configured over the JBOD drives attached to the controller.

Solution Verified - Updated -

Issue

  • RAID controller is hanging and Linux RAID 5 is configured over the JBOD drives attached to the controller.
[server1]
[ 7376.965489] perf: interrupt took too long (4975 > 4971), lowering kernel.perf_event_max_sample_rate to 40000
[14004.483654] perf: interrupt took too long (6219 > 6218), lowering kernel.perf_event_max_sample_rate to 32000
[22956.522915] sd 0:0:30:0: Power-on or device reset occurred
[22957.018691] sd 0:0:30:0: Power-on or device reset occurred
[28081.820312] INFO: task md2_raid5:1424 blocked for more than 120 seconds.
[28081.820342] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[28081.820365] md2_raid5       D ffff8f2fbcf9c1c0     0  1424      2 0x00000000
[28081.820369] Call Trace:
[28081.820376]  [<ffffffffa9785da9>] schedule+0x29/0x70
[28081.820380]  [<ffffffffa95aa98d>] md_super_wait+0x6d/0xd0
[28081.820385]  [<ffffffffa90c7780>] ? wake_up_atomic_t+0x30/0x30
[28081.820387]  [<ffffffffa95b11d1>] bitmap_wait_writes+0xc1/0xd0
[28081.820389]  [<ffffffffa95b28ec>] bitmap_unplug+0xec/0x120
[28081.820406]  [<ffffffffc0a7f640>] raid5d+0x150/0x670 [raid456]
[28081.820408]  [<ffffffffa95a33fd>] md_thread+0x16d/0x1e0
[28081.820410]  [<ffffffffa90c7780>] ? wake_up_atomic_t+0x30/0x30
[28081.820412]  [<ffffffffa95a3290>] ? find_pers+0x80/0x80
[28081.820414]  [<ffffffffa90c6691>] kthread+0xd1/0xe0
[28081.820416]  [<ffffffffa90c65c0>] ? insert_kthread_work+0x40/0x40
[28081.820421]  [<ffffffffa9792d37>] ret_from_fork_nospec_begin+0x21/0x21
[28081.820423]  [<ffffffffa90c65c0>] ? insert_kthread_work+0x40/0x40
[28081.820550] INFO: task splunkd:14132 blocked for more than 120 seconds.
[28081.820571] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this messa

[server2]
Aug 29 22:35:14 hostname kernel: INFO: task md2_raid5:1424 blocked for more than 120 seconds.
Aug 29 22:35:14 hostname kernel: "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
Aug 29 22:35:14 hostname kernel: md2_raid5       D ffff9703f7c220e0     0  1424      2 0x00000000
Aug 29 22:35:14 hostname kernel: Call Trace:
Aug 29 22:35:14 hostname kernel: [<ffffffff87385da9>] schedule+0x29/0x70
Aug 29 22:35:14 hostname kernel: [<ffffffff871aa94d>] md_super_wait+0x6d/0xd0
Aug 29 22:35:14 hostname kernel: [<ffffffff86cc7780>] ? wake_up_atomic_t+0x30/0x30
Aug 29 22:35:14 hostname kernel: [<ffffffff871b1191>] bitmap_wait_writes+0xc1/0xd0

Environment

  • Red Hat Enterprise Linux 7

Subscriber exclusive content

A Red Hat subscription provides unlimited access to our knowledgebase, tools, and much more.

Current Customers and Partners

Log in for full access

Log In

New to Red Hat?

Learn more about Red Hat subscriptions

Using a Red Hat product through a public cloud?

How to access this content