The open-iscsi initiator does not reuse iSIDs

Solution Verified - Updated -

Environment

  • RHEL6.1 and older
  • RHEL5.7 and older

  • iSCSI storage arrays implementing SCSI-3 persistent reservation implementations, based on SPC3 or SPC4

  • RHEL clusters using SCSI-PR with iSCSI storage.

Issue

  • The open-iSCSI initiator does not reuse initiator session identifiers. When using persistent reservations, this can cause a problem on iSCSI storage arrays.

Resolution

  • Red Hat is aware of this problem and is working on a fix to be released in a future version of RHEL. For further details, please contact your support
    representative.

Root Cause

  • Specification SPC3/SPC4 mandates that the device server shall apply the registration only to the I_T nexus that sent the PERSISTENT RESERVE OUT command.
  • Also, rfc 3720 specifies that the I_T_nexus identifier contain the isid which is the initiator part of the session identifier.
  • Since open-iSCSI initiator does not implement Section 9 of rfc3720 that discusses iSID reuse, targets conforming to the spec would treat each session with a unique iSID as a new I_T_nexus.

  • So keys from older logged out sessions will become orphaned and start filling up the storage database.

  • This includes but is not limited to the use of fence_scsi. 

  • The result is that the  target thinks every session is a new session from a different initiator port.

  • This can lead to the target's storage database filling up with duplicate reservation keys.

Work Around:

  • The safest method to use for clearing out stale registrations is to shut down the cluster entirely and use the following command for each device.

  • WARNING - Do NOT attempt this command until the cluster has been shut completely down.

    sg_persist -o -C -K <key> -d <device>

  • This command will remove ALL registrations/reservations -- not just the key specified in the command. It only needs to be used once per device.

This solution is part of Red Hat’s fast-track publication program, providing a huge library of solutions that Red Hat engineers have created while supporting our customers. To give you the knowledge you need the instant it becomes available, these articles may be presented in a raw and unedited form.

Comments