Server crashed after RAID 1 Array removal

Solution Unverified - Updated -

Issue

  • A RAID disk was removed from system using following command:

    $ echo 1 > /sys/block/sdb/device/delete
    

    then RAID logical drive was deleted using ssacli:

    $ ssacli ctrl slot=0 ld 2 show
    $ ssacli ctrl slot=0 ld 2 delete
    

    as soon as the above last command was executed, system crashed with following panic messages:

    smartpqi 0000:5c:00.0: removed 0:0:1:0 31s0aed0ecawedwx Direct-Access     ATA      INTEL SSDSC2KG03 AIO+ qd=32    
    smartpqi 0000:5c:00.0: added 0:1:0:2 4000000200000000 Direct-Access     HPE      LOGICAL VOLUME   SSDSmartPathCap+ En+ RAID-1(1+0) 
    scsi 0:1:0:2: Direct-Access     HPE      LOGICAL VOLUME   1.98 PQ: 0 ANSI: 5
    sd 0:1:0:2: Attached scsi generic sg5 type 0
    sd 0:1:0:2: [sdd] 7501410992 512-byte logical blocks: (3.84 TB/3.49 TiB)
    sd 0:1:0:2: [sdd] 4096-byte physical blocks
    sd 0:1:0:2: [sdd] Write Protect is off
    sd 0:1:0:2: [sdd] Mode Sense: 73 00 00 08
    sd 0:1:0:2: [sdd] Write cache: disabled, read cache: enabled, doesn't support DPO or FUA
    sd 0:1:0:2: [sdd] Attached SCSI disk
     sdb: sdb1
     sdb: sdb1
    smartpqi 0000:5c:00.0: removed 0:1:0:1 4000000100000000 Direct-Access     HPE      LOGICAL VOLUME   SSDSmartPathCap- En- RAID-1(1+0) 
    BUG: unable to handle kernel NULL pointer dereference at 0000000000000060
    IP: [<ffffffffbb77d325>] mutex_lock+0x15/0x2f
    PGD 80000017db959067 PUD 17cb9cb067 PMD 0 
    Oops: 0002 [#1] SMP 
    [...]
    CPU: 0 PID: 28116 Comm: ssacli Kdump: loaded Tainted: G           OE  ------------ T 3.10.0-1062.1.2.el7.x86_64 #1
    Hardware name: HPE ProLiant DL360 Gen10/ProLiant DL360 Gen10, BIOS U32 04/18/2019
    task: ffff9b251916a0e0 ti: ffff9af5cf6a0000 task.ti: ffff9af5cf6a0000
    RIP: 0010:[<ffffffffbb77d325>]  [<ffffffffbb77d325>] mutex_lock+0x15/0x2f
    RSP: 0018:ffff9af5cf6a3b88  EFLAGS: 00010246
    RAX: 0000000000000000 RBX: 0000000000000060 RCX: ffff9af5cf6a3fd8
    RDX: 0000000000000000 RSI: ffff9b25184b4d00 RDI: 0000000000000060
    RBP: ffff9af5cf6a3b90 R08: 000000000000000a R09: 0000000000000000
    R10: 00000000000008c7 R11: ffff9af5cf6a36c6 R12: ffff9b0d1b064800
    R13: ffff9b25184b4de8 R14: ffff9b0d1dfe0000 R15: ffff9b0d1dfe0008
    FS:  00007f6062381700(0000) GS:ffff9b0d1fe00000(0000) knlGS:0000000000000000
    CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
    CR2: 0000000000000060 CR3: 00000017bee62000 CR4: 00000000007607f0
    DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
    DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
    PKRU: 55555554
    Call Trace:
     [<ffffffffbb4ebc1e>] scsi_remove_device+0x1e/0x40
     [<ffffffffc02a3b2c>] pqi_update_device_list+0x58c/0x940 [smartpqi]
     [<ffffffffc02a6984>] pqi_update_scsi_devices+0x4e4/0x1020 [smartpqi]
     [<ffffffffc02a74f6>] pqi_scan_scsi_devices+0x36/0x90 [smartpqi]
     [<ffffffffc02a76e5>] pqi_ioctl+0x115/0x170 [smartpqi]
     [<ffffffffbb4de205>] scsi_ioctl+0x345/0x450
     [<ffffffffc0669606>] sg_ioctl+0xe6/0xdf5 [sg]
     [<ffffffffbb25b5cd>] ? do_filp_open+0x4d/0xb0
     [<ffffffffbb25d9e0>] do_vfs_ioctl+0x3a0/0x5a0
     [<ffffffffbb25a28d>] ? putname+0x3d/0x60
     [<ffffffffbb25dc81>] SyS_ioctl+0xa1/0xc0
     [<ffffffffbb78d172>] tracesys+0xa6/0xcc
    Code: ff 07 7f 05 e8 fd 0a 00 00 5d c3 0f 1f 00 83 ea 01 89 50 10 eb d9 0f 1f 44 00 00 55 48 89 e5 53 48 89 fb e8 6e 21 00 00 48 89 df <f0> ff 0f 79 05 e8 61 0b 00 00 65 48 8b 04 25 80 0e 01 00 48 89 
    RIP  [<ffffffffbb77d325>] mutex_lock+0x15/0x2f
    RSP <ffff9af5cf6a3b88>
    CR2: 0000000000000060
    

Environment

  • Red Hat Enterprise Linux 7, 8
  • HPE P408i-a SR Gen10 controller
  • smartpqi driver

Subscriber exclusive content

A Red Hat subscription provides unlimited access to our knowledgebase, tools, and much more.

Current Customers and Partners

Log in for full access

Log In