Storage issue: Abort command issued -- 1 30d1b 2002.
Environment
- Red Hat Enterprise Linux 5.5
- Qlogic HBA
Issue
- Storage related errors found in syslog.
kernel: qla2xxx 0000:27:00.0: scsi(0:0:0): Abort command issued -- 1 3190d 2002.
Resolution
- Issue was resolved by doing a "media scan" on the device. This is a type of surface and parity scan. When this scan finished, a hard disc was marked as failed. After replacing the disk, reconstructed RAID and the issue disapeared.
Please also see Abort command Issued" messages appear in /var/log/messages file.
Diagnostic Steps
Check for the error messages similar to seen below.
Oct 6 17:00:31 hostname kernel: sd 0:0:0:1: SCSI error: return code = 0x08000002
Oct 6 17:00:31 hostname kernel: sdb: Current: sense key: Aborted Command
Oct 6 17:00:31 hostname kernel: Add. Sense: Internal target failure
Oct 6 17:00:31 hostname kernel:
Oct 6 17:00:31 hostname kernel: end_request: I/O error, dev sdb, sector 104924949
Oct 6 17:00:31 hostname kernel: VxVM vxdmp V-5-0-0 i/o error occured (errno=0x0) on dmpnode 201/0x20
Oct 6 17:00:31 hostname kernel:
Oct 6 17:00:31 hostname kernel: VxVM vxio V-5-0-1266 Subdisk dm_slm01_001001-04 block 1557: Uncorrectable write error
Oct 6 17:00:31 hostname kernel: vxfs: msgcnt 4 mesg 037: V-2-37: vx_metaioerr - vx_logbuf_write - /dev/vx/dsk/dgslm01_001/spimibeamanagedvol file system meta data write error in dev/block 0/1557
Oct 6 17:00:31 hostname kernel: vxfs: msgcnt 5 mesg 031: V-2-31: vx_disable - /dev/vx/dsk/dgslm01_001/spimibeamanagedvol file system disabled
Oct 6 17:00:31 hostname kernel: vxfs: msgcnt 6 mesg 037: V-2-37: vx_metaioerr - vx_inode_iodone - /dev/vx/dsk/dgslm01_001/spimibeamanagedvol file system meta data write error in dev/block 0/35648
var/log/messages.1:Oct 1 10:55:41 hostname kernel: sd 0:0:0:1: SCSI error: return code = 0x08000002
var/log/messages.1:Oct 1 10:55:41 hostname kernel: sdb: Current: sense key: Aborted Command
var/log/messages.1:Oct 1 10:55:41 hostname kernel: Add. Sense: Internal target failure
var/log/messages.1:Oct 1 10:55:41 hostname kernel:
var/log/messages.1:Oct 1 10:55:41 hostname kernel: end_request: I/O error, dev sdb, sector 104978112
var/log/messages.1:Oct 1 10:55:41 hostname kernel: VxVM vxdmp V-5-0-0 i/o error occured (errno=0x0) on dmpnode 201/0x20
var/log/messages.1:Oct 1 10:55:41 hostname kernel:
var/log/messages.1:Oct 1 10:55:41 hostname kernel: VxVM vxio V-5-0-1266 Subdisk dm_slm01_001001-04 block 54720: Uncorrectable write error
var/log/messages.1:Oct 1 10:56:41 hostname kernel: qla2xxx 0000:27:00.0: scsi(0:0:1): Abort command issued -- 1 282cb 2002.
var/log/messages.1:Oct 1 10:56:42 hostname kernel: qla2xxx 0000:27:00.0: scsi(0:0:1): Abort command issued -- 1 282cc 2002.
var/log/messages.1:Oct 1 10:56:42 hostname kernel: qla2xxx 0000:27:00.0: scsi(0:0:1): Abort command issued -- 1 282ec 2002.
var/log/messages.1:Oct 1 10:56:42 hostname kernel: qla2xxx 0000:27:00.0: scsi(0:0:1): Abort command issued -- 1 282f2 2002.
var/log/messages.1:Oct 1 10:57:42 hostname kernel: qla2xxx 0000:27:00.0: scsi(0:0:1): Abort command issued -- 1 2830d 2002.
var/log/messages.1:Oct 1 10:57:43 hostname kernel: qla2xxx 0000:27:00.0: scsi(0:0:1): Abort command issued -- 1 2830e 2002.
var/log/messages.1:Oct 1 10:58:17 hostname kernel: INFO: task java:19697 blocked for more than 120 seconds.
var/log/messages.1:Oct 1 10:58:17 hostname kernel: "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
var/log/messages.1:Oct 1 10:58:17 hostname kernel: java D ffff81000106a220 0 19697 1 19698 19696 (NOTLB)
var/log/messages.1:Oct 1 10:58:17 hostname kernel: ffff810b0efff878 0000000000000086 ffff8101269a86e0 ffff8101269a8ef8
var/log/messages.1:Oct 1 10:58:17 hostname kernel: ffff8101269aa158 0000000000000009 ffff810b1141b0c0 ffff81102fc71080
var/log/messages.1:Oct 1 10:58:17 hostname kernel: 00025c8b93350f7a 0000000001d2471c ffff810b1141b2a8 0000000c269ad060
var/log/messages.1:Oct 1 10:58:17 hostname kernel: Call Trace:
var/log/messages.1:Oct 1 10:58:17 hostname kernel: [<ffffffff8006e1db>] do_gettimeofday+0x40/0x90
var/log/messages.1:Oct 1 10:58:17 hostname kernel: [<ffffffff800289ef>] sync_page+0x0/0x43
var/log/messages.1:Oct 1 10:58:17 hostname kernel: [<ffffffff800637ea>] io_schedule+0x3f/0x67
var/log/messages.1:Oct 1 10:58:17 hostname kernel: [<ffffffff80028a2d>] sync_page+0x3e/0x43
var/log/messages.1:Oct 1 10:58:17 hostname kernel: [<ffffffff80063a16>] __wait_on_bit+0x40/0x6e
var/log/messages.1:Oct 1 10:58:17 hostname kernel: [<ffffffff800350cd>] wait_on_page_bit+0x6c/0x72
This solution is part of Red Hat’s fast-track publication program, providing a huge library of solutions that Red Hat engineers have created while supporting our customers. To give you the knowledge you need the instant it becomes available, these articles may be presented in a raw and unedited form.
Welcome! Check out the Getting Started with Red Hat page for quick tours and guides for common tasks.
