System crashed after device is offlined

Solution Unverified - Updated -

Issue

  • System crashed after device is offlined
[ 6062.027258] hpsa 0000:03:00.0: scsi 0:1:0:1: resetting logical  Direct-Access     HP       LOGICAL VOLUME   RAID-5 SSDSmartPathCap-
 En- Exp=1
[ 6063.028255] hpsa 0000:03:00.0: device is ready.
[ 6063.028416] hpsa 0000:03:00.0: scsi 0:1:0:1: reset logical  completed successfully Direct-Access     HP       LOGICAL VOLUME   RAID
-5 SSDSmartPathCap- En- Exp=1
[ 6063.028852] hpsa 0000:03:00.0: cmd_tagged_alloc: tag collision (tag=19)
[ 6063.029008] sd 0:1:0:1: [sdb] tag#8 CDB: Test Unit Ready 00 00 00 00 00 00
[ 6063.029191] sd 0:1:0:1: [sdb] tag#8 CDB: Test Unit Ready 00 00 00 00 00 00
[ 6073.027594] sd 0:1:0:1: Device offlined - not ready after error recovery
[ 6073.027753] sd 0:1:0:1: Device offlined - not ready after error recovery
[ 6073.027902] sd 0:1:0:1: Device offlined - not ready after error recovery
[ 6073.028052] sd 0:1:0:1: Device offlined - not ready after error recovery
[ 6073.028201] sd 0:1:0:1: Device offlined - not ready after error recovery
[ 6073.028349] sd 0:1:0:1: Device offlined - not ready after error recovery
[ 6073.028510] sd 0:1:0:1: [sdb] tag#0 FAILED Result: hostbyte=DID_OK driverbyte=DRIVER_TIMEOUT
[ 6073.028829] sd 0:1:0:1: [sdb] tag#0 CDB: Read(16) 88 00 00 00 00 00 01 1d 00 a0 00 00 04 00 00 00
[ 6073.029120] blk_update_request: I/O error, dev sdb, sector 18677920
[ 6073.029325] sd 0:1:0:1: rejecting I/O to offline device
[ 6073.029481] sd 0:1:0:1: [sdb] killing request
[ 6073.029633] sd 0:1:0:1: rejecting I/O to offline device
[ 6073.029642] sd 0:1:0:1: [sdb] FAILED Result: hostbyte=DID_NO_CONNECT driverbyte=DRIVER_OK
[ 6073.029644] sd 0:1:0:1: [sdb] CDB: Write(16) 8a 00 00 00 00 00 8c 21 93 80 00 00 00 08 00 00
[ 6073.029646] blk_update_request: I/O error, dev sdb, sector 2351010688
[ 6073.030505] sd 0:1:0:1: rejecting I/O to offline device
[ 6073.030662] sd 0:1:0:1: rejecting I/O to offline device

[.... ]
[ 6073.035002] sd 0:1:0:1: rejecting I/O to offline device
[ 6073.035151] sd 0:1:0:1: rejecting I/O to offline device
[ 6073.035302] sd 0:1:0:1: rejecting I/O to offline device
[ 6073.035451] sd 0:1:0:1: rejecting I/O to offline device
[ 6073.035599] sd 0:1:0:1: rejecting I/O to offline device
[ 6073.035778] sd 0:1:0:1: [sdb] tag#1 FAILED Result: hostbyte=DID_OK driverbyte=DRIVER_TIMEOUT
[ 6073.035787] XFS (dm-2): metadata I/O error: block 0x8bd9a488 ("xlog_iodone") error 5 numblks 64
[ 6073.035800] XFS (dm-2): writeback error on sector 2351006592
[ 6073.035803] XFS (dm-2): xfs_do_force_shutdown(0x2) called from line 1238 of file fs/xfs/xfs_log.c.  Return address = 0xffffffffc061c210
[ 6073.035846] XFS (dm-2): Log I/O Error Detected.  Shutting down filesystem
[ 6073.035853] XFS (dm-2): Please umount the filesystem and rectify the problem(s)
[ 6073.035873] XFS (dm-2): writeback error on sector 3527209424
[ 6073.035889] XFS (dm-2): writeback error on sector 9955096
[ 6073.035922] XFS (dm-2): metadata I/O error: block 0xd1947802 ("xfs_trans_read_buf_map") error 5 numblks 1
[ 6073.037838] sd 0:1:0:1: [sdb] tag#1 CDB: Read(16) 88 00 00 00 00 00 01 1d 04 a0 00 00 04 00 00 00
[ 6073.038124] blk_update_request: I/O error, dev sdb, sector 18678944
[ 6073.038312] sd 0:1:0:1: [sdb] tag#2 FAILED Result: hostbyte=DID_OK driverbyte=DRIVER_TIMEOUT
[ 6073.038612] sd 0:1:0:1: [sdb] tag#2 CDB: Read(16) 88 00 00 00 00 00 01 1d 08 a0 00 00 04 00 00 00
[ 6073.038905] blk_update_request: I/O error, dev sdb, sector 18679968
[ 6073.039087] sd 0:1:0:1: [sdb] tag#6 FAILED Result: hostbyte=DID_OK driverbyte=DRIVER_TIMEOUT
[ 6073.039378] sd 0:1:0:1: [sdb] tag#6 CDB: Read(16) 88 00 00 00 00 00 01 1d 0c a0 00 00 04 00 00 00
[ 6073.039678] blk_update_request: I/O error, dev sdb, sector 18680992
[ 6073.039855] sd 0:1:0:1: [sdb] tag#7 FAILED Result: hostbyte=DID_OK driverbyte=DRIVER_TIMEOUT
[ 6073.040140] sd 0:1:0:1: [sdb] tag#7 CDB: Read(16) 88 00 00 00 00 00 01 1d 10 a0 00 00 04 00 00 00
[ 6073.040427] blk_update_request: I/O error, dev sdb, sector 18682016
[ 6073.040612] sd 0:1:0:1: [sdb] tag#8 FAILED Result: hostbyte=DID_OK driverbyte=DRIVER_TIMEOUT
[ 6073.040905] sd 0:1:0:1: [sdb] tag#8 CDB: Read(16) 88 00 00 00 00 00 01 1d 14 a0 00 00 03 50 00 00
[ 6073.041190] blk_update_request: I/O error, dev sdb, sector 18683040
[ 6073.041369] hpsa 0000:03:00.0: cmd_tagged_alloc: tag collision (tag=11)
[ 6073.041524] .9.......9........... : [sdb] CDB: Read(16) 88 00 00 00 00 00 01 1d 00 a0 00 00 04 00 00 00
[ 6073.041827] sd 0:1:0:0: [sda] tag#0 CDB: Write(10) 2a 00 07 54 ab 70 00 00 08 00
[ 6076.604660] BUG: unable to handle kernel paging request at 0000000000001408
[ 6076.604821] IP: [<ffffffffb11833c2>] _raw_spin_lock_irqsave+0x22/0x40
[ 6076.604978] PGD 0 
[ 6076.605120] Oops: 0002 [#1] SMP 

Environment

  • Red Hat Enterprise Linux 7
    • kernel-3.10.0-1062.12.1.el7

Subscriber exclusive content

A Red Hat subscription provides unlimited access to our knowledgebase, tools, and much more.

Current Customers and Partners

Log in for full access

Log In

New to Red Hat?

Learn more about Red Hat subscriptions

Using a Red Hat product through a public cloud?

How to access this content