How many LUNs can be mapped from targetcli/LIO based targets?
Environment
- Red Hat Enterprise Linux (RHEL)
- 7
- 8
- Linux IO target (LIO)
- targetcli
Issue
-
Getting following error while exporting more than 256 LUNs using iSCSI target configured in targetcli:
/iscsi/iqn.20...0bc/tpg1/luns> create /backstores/fileio/file255 Created LUN 255. Cannot map new lun 255 into ACL iqn.1994-05.com.redhat:2012b45c62c5
-
How many LUNs can be mapped from targetcli/LIO based targets?
Resolution
- RHEL 7.5
kernel-3.10.0-862.el7
and later kernels no longer have a 256 LUNs limit. However, the limit remains in the userspace toolingtargetcli/ftslib
for creating and managing LUNs. - Red Hat is aware of a feature request to remove the limit in
targetcli/ftslib
, currently under review in RFE RHBZ#2149425. If you require this feature, open a support case in the Red Hat Customer Portal referring to this solution.
Root Cause
-
The original RFC 3720 for iSCSI describes a LUN ID field as being 64-bit identifier. It is formatted in accordance with SAM-2. So, there is no iSCSI protocol limitation which restricts the maximum number of mapped_luns to 256.
RFC 3720 10.2.1.7. LUN Some opcodes operate on a specific Logical Unit. The Logical Unit Number (LUN) field identifies which Logical Unit. If the opcode does not relate to a Logical Unit, this field is either ignored or may be used in an opcode specific way. The LUN field is 64-bits and should be formatted in accordance with [SAM2]. For example, LUN[0] from [SAM2] is BHS byte 8 and so on up to LUN[7] from [SAM2], which is BHS byte 15
-
The older LIO target code limited the max number of mapped LUNs to 'TRANSPORT_MAX_LUNS_PER_TPG' count of 256.
include/target/target_core_base.h /* Maximum Number of LUNs per Target Portal Group */ /* Don't raise above 511 or REPORT_LUNS needs to handle >1 page */ #define TRANSPORT_MAX_LUNS_PER_TPG 256 <------
Above limitation was removed through following upstream patch:
target: Remove TARGET_MAX_LUNS_PER_TRANSPORTThis patch was back-ported to RHEL 7.5 kernel-3.10.0-862.el7 through private BZ#1366062
-
While the initial 'TRANSPORT_MAX_LUNS_PER_TPG' limitation in LIO target is now removed, the targetcli code still uses MAX_LUN count of 255. So, the logical units can have values of 0-255, which results in max 256 total logical units:
rtslib-fb/rtslib/target.py class MappedLUN(CFSNode): ''' This is an interface to RTS Target Mapped LUNs. A MappedLUN is a mapping of a TPG LUN to a specific initiator node, and is part of a NodeACL. It allows the initiator to actually access the TPG LUN if ACLs are enabled for the TPG. The initial TPG LUN will then be seen by the initiator node as the MappedLUN. ''' MAX_LUN = 255 # MappedLUN private stuff [...]
-
This limit may be increased in future. If there is a requirement to export more than 256 LUNs through targetcli/LIO based target, then please reach out to your Red Hat support representative and RFE (Request for Feature Enhancement) could be created for the same.
Diagnostic Steps
-
Verify on target server by using targetcli ls command and look for below-highlighted numbers-
o- iscsi .............................................................. [Targets: 2] | o- iqn.2003-01.org.linux-iscsi.server.x8664:sn.699f8cd480bc ....... [TPGs: 1] <- 1 Target | | o- tpg1 ......................................................... [no-gen-acls, no-auth] | | o- acls ....................................................... [ACLs: 1] | | | o- iqn.1994-05.com.redhat:2012b45c62c5 .... [Mapped LUNs: 255] <- 256 mapped luns (0-255) per initiator | | o- luns ....................................................... [LUNs: 261] <- LUN per target is under 65K limit
-
To verify on the client side, use /proc/scsi/scsi command:
Host: scsi12 Channel: 00 Id: 00 Lun: 02 Vendor: LIO-ORG Model: file2 Rev: 4.0 Type: Direct-Access ANSI SCSI revision: 05 #cat /proc/scsi/scsi | grep -i "Vendor: LIO-ORG"| wc -l 255
-
Trying to map a LUN that is greater than 255 will get errors like Cannot map new lun 255 into ACL iqn.
/iscsi/iqn.20...0bc/tpg1/luns> create /backstores/fileio/file255 Created LUN 255. Cannot map new lun 255 into ACL iqn.1994-05.com.redhat:2012b45c62c5 /iscsi/iqn.20...0bc/tpg1/luns> create /backstores/fileio/file256 Created LUN 256. Cannot map new lun 256 into ACL iqn.1994-05.com.redhat:2012b45c62c5
This solution is part of Red Hat’s fast-track publication program, providing a huge library of solutions that Red Hat engineers have created while supporting our customers. To give you the knowledge you need the instant it becomes available, these articles may be presented in a raw and unedited form.
Comments