Seeing kernel: scsi host2: hdr status = FCPIO_DATA_CNT_MISMATCH and path failures
Issue
- Cisco UCS blades and Nexus switches with mismatching QoS settings experiencing FC underruns under heavy load.
- SCSI timeouts and multipath path failures observed in
/var/log/messages. -
The UCS switch logs show:
%IPQOSMGR-2-QOSMGR_DCBXP_PFC_CMP_FAIL_MSG: Ethernet1/31 - qos config for PFC not compatible with the peer %IPQOSMGR-2-QOSMGR_DCBXP_PFC_CMP_FAIL_MSG: Ethernet1/30 - qos config for PFC not compatible with the peer -
The system will log
kernel: scsi host1: hdr status = FCPIO_DATA_CNT_MISMATCH kernel: scsi host1: hdr status = FCPIO_DATA_CNT_MISMATCH kernel: scsi host2: hdr status = FCPIO_DATA_CNT_MISMATCH kernel: scsi host2: hdr status = FCPIO_DATA_CNT_MISMATCH kernel: scsi host2: hdr status = FCPIO_DATA_CNT_MISMATCH kernel: scsi host1: hdr status = FCPIO_DATA_CNT_MISMATCH kernel: scsi host1: hdr status = FCPIO_DATA_CNT_MISMATCH kernel: scsi host2: hdr status = FCPIO_DATA_CNT_MISMATCH kernel: scsi host2: hdr status = FCPIO_DATA_CNT_MISMATCH kernel: scsi host1: hdr status = FCPIO_DATA_CNT_MISMATCH -
And:
kernel: sd 2:0:0:16: [sdcy] Medium access timeout failure. Offlining disk! kernel: device-mapper: multipath: Failing path 70:96. multipathd: checker failed path 70:96 in map mpathq multipathd: mpathq: remaining active paths: 3 kernel: host2: Assigned Port ID 7c0008 kernel: sd 2:0:0:16: [sdcy] Result: hostbyte=DID_TRANSPORT_DISRUPTED driverbyte=DRIVER_OK kernel: sd 2:0:0:16: [sdcy] CDB: Read(10): 28 00 04 96 2c 00 00 04 00 00 kernel: end_request: I/O error, dev sdcy, sector 76950528 kernel: sd 2:0:0:16: rejecting I/O to offline device kernel: sd 2:0:0:16: [sdcy] killing request kernel: sd 2:0:0:16: rejecting I/O to offline device kernel: sd 2:0:0:16: rejecting I/O to offline device kernel: sd 2:0:0:16: rejecting I/O to offline device kernel: sd 2:0:0:16: rejecting I/O to offline device kernel: sd 2:0:0:16: [sdcy] Result: hostbyte=DID_NO_CONNECT driverbyte=DRIVER_OK kernel: sd 2:0:0:16: [sdcy] CDB: Read(10): 28 00 03 2f cc 00 00 04 00 00 kernel: end_request: I/O error, dev sdcy, sector 53464064 kernel: sd 2:0:0:16: [sdcy] Result: hostbyte=DID_TRANSPORT_DISRUPTED driverbyte=DRIVER_OK kernel: sd 2:0:0:16: [sdcy] CDB: Read(10): 28 00 00 09 fc 00 00 04 00 00 kernel: end_request: I/O error, dev sdcy, sector 654336
Environment
- Red Hat Enterprise Linux 5
- Red Hat Enterprise Linux 6
- Red Hat Enterprise Linux 7
- Cisco UCS blades with fnic interfaces.
- Configured with mismatched QoS (quality of service) settings between switches.
Subscriber exclusive content
A Red Hat subscription provides unlimited access to our knowledgebase of over 48,000 articles and solutions.
Welcome! Check out the Getting Started with Red Hat page for quick tours and guides for common tasks.
