multipath reports incorrect LUN size on random reboot

Solution Verified - Updated -

Environment

  • Red Hat Enterprise Linux 8
  • Falcon IPStor Storage

Issue

  • Server lands to maintenance mode due to failing to mount SAN mount point.
  • multipath reports incorrect disk size after a random reboot for same WWID.
  • All paths of the SAN LUN are not visible.

Resolution

  • Try to verify if there are any issues present from the FC switch, FC cabling, zoning or Storage array, or SFP.
  • It would also be advised to engage the storage vendor to review the switch logs to verify if there are any error counters, CRC errors in FC switch logs.

Root Cause

  • Server was landing in maintenance mode because it was unable to form a multipath map due to incorrect device size reported by SCSI which was fixed by replacing faulty SFP.

Diagnostic Steps

  • Collect the journal logs from the maintenance mode.
  • Check the multipath errors in the journal logs:
    Aug 09 14:28:05 testbox multipathd[1880]: sdl: size = 0        <<<<<< Size reporting it as zero
    Aug 09 14:28:05 testbox multipathd[1880]: sdl: vendor = FALCON
    Aug 09 14:28:05 testbox multipathd[1880]: sdl: vpd_vendor_id = 0 "undef" (setting: multipath internal)
    [...]
    Aug 10 12:40:44 testbox multipathd[2018]: sdp: vendor = FALCON
    Aug 10 12:40:44 testbox multipath[2252]: sdq: size = 21474836480   <<<<<< Size reporting it as 21474836480 (~10Tb)
    Aug 10 12:40:44 testbox multipath[2252]: sdq: vendor = FALCON
  • Check the device size mismatch errors:
    Aug 10 12:40:34 testbox multipathd[2018]: sdg: size mismatch for mpatha, not adding path
    Aug 10 12:40:34 testbox multipathd[2018]: sdi: size mismatch for mpatha, not adding path
    Aug 10 12:40:34 testbox multipathd[2018]: sdm: size mismatch for mpatha, not adding path
  • Verify the size of the disk using sg_readcap for all the paths:
    # for i in $(lsblk | grep disk|grep sd | awk '{print $1}'); do echo /dev/$i && sg_readcap /dev/$i;done
    # for i in $(lsblk | grep disk|grep sd | awk '{print $1}'); do echo /dev/$i && /usr/lib/udev/scsi_id -g /dev/$i;done

This solution is part of Red Hat’s fast-track publication program, providing a huge library of solutions that Red Hat engineers have created while supporting our customers. To give you the knowledge you need the instant it becomes available, these articles may be presented in a raw and unedited form.

Comments