Chapter 24. Online Storage Management

⁠Chapter 24. Online Storage Management

It is often desirable to add, remove or re-size storage devices while the operating system is running, and without rebooting. This chapter outlines the procedures that may be used to reconfigure storage devices on Red Hat Enterprise Linux 7 host systems while the system is running. It covers iSCSI and Fibre Channel storage interconnects; other interconnect types may be added it the future.

This chapter focuses on adding, removing, modifying, and monitoring storage devices. It does not discuss the Fibre Channel or iSCSI protocols in detail. For more information about these protocols, refer to other documentation.

This chapter makes reference to various sysfs objects. Red Hat advises that the sysfs object names and directory structure are subject to change in major Red Hat Enterprise Linux releases. This is because the upstream Linux kernel does not provide a stable internal API. For guidelines on how to reference sysfs objects in a transportable way, refer to the document /usr/share/doc/kernel-doc-version/Documentation/sysfs-rules.txt in the kernel source tree for guidelines.

Warning

Online storage reconfiguration must be done carefully. System failures or interruptions during the process can lead to unexpected results. Red Hat advises that you reduce system load to the maximum extent possible during the change operations. This will reduce the chance of I/O errors, out-of-memory errors, or similar errors occurring in the midst of a configuration change. The following sections provide more specific guidelines regarding this.

In addition, Red Hat recommends that you back up all data before reconfiguring online storage.

⁠24.1. Target Setup

Red Hat Enterprise Linux 7 uses the targetcli shell as a front end for viewing, editing, and saving the configuration of the Linux-IO Target without the need to manipulate the kernel target's configuration files directly. The targetcli tool is a command-line interface that allows an administrator to export local storage resources, which are backed by either files, volumes, local SCSI devices, or RAM disks, to remote systems. The targetcli tool has a tree-based layout, includes built-in tab completion, and provides full auto-complete support and inline documentation.

The hierarchy of targetcli does not always match the kernel interface exactly because targetcli is simplified where possible.

Important

To ensure that the changes made in targetcli are persistent, start and enable the target service:

# systemctl start target
# systemctl enable target

⁠24.1.1. Install and Run targetcli

To install targetcli, use:

# yum install targetcli

Start the target service:

# systemctl start target

Configure target to start at boot time:

# systemctl enable target

Use the targetcli command, and then use the ls command for the layout of the tree interface:

# targetcli
:
/> ls
o- /........................................[...]
  o- backstores.............................[...]
  | o- block.................[Storage Objects: 0]           
  | o- fileio................[Storage Objects: 0]       
  | o- pscsi.................[Storage Objects: 0]         
  | o- ramdisk...............[Storage Ojbects: 0]          
  o- iscsi...........................[Targets: 0]   
  o- loopback........................[Targets: 0]

Note

In Red Hat Enterprise Linux 7.0, using the targetcli command from Bash, for example, targetcli iscsi/ create, does not work and does not return an error. Starting with Red Hat Enterprise Linux 7.1, an error status code is provided to make using targetcli with shell scripts more useful.

⁠24.1.2. Create a Backstore

Backstores enable support for different methods of storing an exported LUN's data on the local machine. Creating a storage object defines the resources the backstore uses.

Note

In Red Hat Enterprise Linux 6, the term 'backing-store' is used to refer to the mappings created. However, to avoid confusion between the various ways 'backstores' can be used, in Red Hat Enterprise Linux 7 the term 'storage objects' refers to the mappings created and 'backstores' is used to describe the different types of backing devices.

The backstore devices that LIO supports are:

FILEIO (Linux file-backed storage)

FILEIO storage objects can support either write_back or write_thru operation. The write_back enables the local file system cache. This improves performance but increases the risk of data loss. It is recommended to use write_back=false to disable write_back in favor of write_thru.

To create a fileio storage object, run the command /backstores/fileio create file_name file_location file_size write_back=false. For example:

/> /backstores/fileio create file1 /tmp/disk1.img 200M write_back=false
Created fileio file1 with size 209715200

BLOCK (Linux BLOCK devices)

The block driver allows the use of any block device that appears in the /sys/block to be used with LIO. This includes physical devices (for example, HDDs, SSDs, CDs, DVDs) and logical devices (for example, software or hardware RAID volumes, or LVM volumes).

Note

BLOCK backstores usually provide the best performance.

To create a BLOCK backstore using the /dev/sdb block device, use the following command:

/> /backstores/block create name=block_backend dev=/dev/sdb
Generating a wwn serial.
Created block storage object block_backend using /dev/sdb.

PSCSI (Linux pass-through SCSI devices)

Any storage object that supports direct pass-through of SCSI commands without SCSI emulation, and with an underlying SCSI device that appears with lsscsi in /proc/scsi/scsi (such as a SAS hard drive) can be configured as a backstore. SCSI-3 and higher is supported with this subsystem.

Warning

PSCSI should only be used by advanced users. Advanced SCSI commands such as for Aysmmetric Logical Unit Assignment (ALUAs) or Persistent Reservations (for example, those used by VMware ESX, and vSphere) are usually not implemented in the device firmware and can cause malfunctions or crashes. When in doubt, use BLOCK for production setups instead.

To create a PSCSI backstore for a physical SCSI device, a TYPE_ROM device using /dev/sr0 in this example, use:

/> backstores/pscsi/ create name=pscsi_backend dev=/dev/sr0
Generating a wwn serial.
Created pscsi storage object pscsi_backend using /dev/sr0

Memory Copy RAM disk (Linux RAMDISK_MCP)

Memory Copy RAM disks (ramdisk) provide RAM disks with full SCSI emulation and separate memory mappings using memory copy for initiators. This provides capability for multi-sessions and is particularly useful for fast, volatile mass storage for production purposes.

To create a 1GB RAM disk backstore, use the following command:

/> backstores/ramdisk/ create name=rd_backend size=1GB
Generating a wwn serial.
Created rd_mcp ramdisk rd_backend with size 1GB.

⁠24.1.3. Create an iSCSI Target

To create an iSCSI target:

⁠

Procedure 24.1. Create an iSCSI target

Run targetcli.
Move into the iSCSI configuration path:
```
/> iscsi/
```
Note
The cd command is also accepted to change directories, as well as simply listing the path to move into.

Create an iSCSI target using a default target name.

/iscsi> create 
Created target 
iqn.2003-01.org.linux-iscsi.hostname.x8664:sn.78b473f296ff
Created TPG1

Or create an iSCSI target using a specified name.

/iscsi > create iqn.2006-04.com.example:444
Created target iqn.2006-04.com.example:444
Created TPG1

Verify that the newly created target is visible when targets are listed with ls.

/iscsi > ls
o- iscsi.......................................[1 Target]
    o- iqn.2006-04.com.example:444................[1 TPG] 
        o- tpg1...........................[enabled, auth]
            o- acls...............................[0 ACL]
            o- luns...............................[0 LUN]
            o- portals.........................[0 Portal]

Note

As of Red Hat Enterprise Linux 7.1, whenever a target is created, a default portal is also created.

⁠24.1.4. Configure an iSCSI Portal

To configure an iSCSI portal, an iSCSI target must first be created and associated with a TPG. For instructions on how to do this, refer to Section 24.1.3, “Create an iSCSI Target”.

Note

As of Red Hat Enterprise Linux 7.1 when an iSCSI target is created, a default portal is created as well. This portal is set to listen on all IP addresses with the default port number (that is, 0.0.0.0:3260). To remove this and add only specified portals, use /iscsi/iqn-name/tpg1/portals delete ip_address=0.0.0.0 ip_port=3260 then create a new portal with the required information.

⁠

Procedure 24.2. Create an iSCSI Portal

Move into the TPG.
```
/iscsi> iqn.2006-04.example:444/tpg1/
```
There are two ways to create a portal: create a default portal, or create a portal specifying what IP address to listen to.
Creating a default portal uses the default iSCSI port 3260 and allows the target to listen on all IP addresses on that port.
```
/iscsi/iqn.20...mple:444/tpg1> portals/ create
Using default IP port 3260
Binding to INADDR_Any (0.0.0.0)
Created network portal 0.0.0.0:3260
```
To create a portal specifying what IP address to listen to, use the following command.
```
/iscsi/iqn.20...mple:444/tpg1> portals/ create 192.168.122.137
Using default IP port 3260
Created network portal 192.168.122.137:3260
```

Verify that the newly created portal is visible with the ls command.

/iscsi/iqn.20...mple:444/tpg1> ls
o- tpg.................................. [enambled, auth]  
    o- acls ......................................[0 ACL]
    o- luns ......................................[0 LUN]
    o- portals ................................[1 Portal]
        o- 192.168.122.137:3260......................[OK]

⁠24.1.5. Configure LUNs

To configure LUNs, first create storage objects. See Section 24.1.2, “Create a Backstore” for more information.

⁠

Procedure 24.3. Configure LUNs

Create LUNs of already created storage objects.

/iscsi/iqn.20...mple:444/tpg1> luns/ create /backstores/ramdisk/ramdisk1
Created LUN 0.

/iscsi/iqn.20...mple:444/tpg1> luns/ create /backstores/block/block1
Created LUN 1.

/iscsi/iqn.20...mple:444/tpg1> luns/ create /backstores/fileio/file1
Created LUN 2.

Show the changes.

/iscsi/iqn.20...mple:444/tpg1> ls
o- tpg.................................. [enambled, auth]
    o- acls ......................................[0 ACL]
    o- luns .....................................[3 LUNs]
    |  o- lun0.........................[ramdisk/ramdisk1]
    |  o- lun1.................[block/block1 (/dev/vdb1)]
    |  o- lun2...................[fileio/file1 (/foo.img)]
    o- portals ................................[1 Portal]
        o- 192.168.122.137:3260......................[OK]

Note

Be aware that the default LUN name starts at 0, as opposed to 1 as was the case when using tgtd in Red Hat Enterprise Linux 6.

Important

By default, LUNs are created with read-write permissions. In the event that a new LUN is added after ACLs have been created that LUN will be automatically mapped to all available ACLs. This can cause a security risk. Use the following procedure to create a LUN as read-only.

⁠

Procedure 24.4. Create a read-only LUN

To create a LUN with read-only permissions, first use the following command:
```
/> set global auto_add_mapped_luns=false
Parameter auto_add_mapped_luns is now 'false'.
```
This prevents the auto mapping of LUNs to existing ACLs allowing the manual mapping of LUNs.

Next, manually create the LUN with the command

iscsi/target_iqn_name/tpg1/acls/initiator_iqn_name/ create mapped_lun=next_sequential_LUN_number tpg_lun_or_backstore=backstore write_protect=1

/> iscsi/iqn.2015-06.com.redhat:target/tpg1/acls/iqn.2015-06.com.redhat:initiator/ create mapped_lun=1 tpg_lun_or_backstore=/backstores/block/block2 write_protect=1
Created LUN 1.
Created Mapped LUN 1.
/> ls
o- / ...................................................... [...]
  o- backstores ........................................... [...]
  <snip>
  o- iscsi ......................................... [Targets: 1]
  | o- iqn.2015-06.com.redhat:target .................. [TPGs: 1]
  |   o- tpg1 ............................ [no-gen-acls, no-auth]
  |     o- acls ....................................... [ACLs: 2]
  |     | o- iqn.2015-06.com.redhat:initiator .. [Mapped LUNs: 2]
  |     | | o- mapped_lun0 .............. [lun0 block/disk1 (rw)]
  |     | | o- mapped_lun1 .............. [lun1 block/disk2 (ro)]
  |     o- luns ....................................... [LUNs: 2]
  |     | o- lun0 ...................... [block/disk1 (/dev/vdb)]
  |     | o- lun1 ...................... [block/disk2 (/dev/vdc)]
  <snip>

The mapped_lun1 line now has (ro) at the end (unlike mapped_lun0's (rw)) stating that it is read-only.

⁠24.1.6. Configure ACLs

Create an ACL for each initiator that will be connecting. This enforces authentication when that initiator connects, allowing only LUNs to be exposed to each initiator. Usually each initator has exclusive access to a LUN. Both targets and initiators have unique identifying names. The initiator's unique name must be known to configure ACLs. For open-iscsi initiators, this can be found in /etc/iscsi/initiatorname.iscsi.

⁠

Procedure 24.5. Configure ACLs

Move into the acls directory.
```
/iscsi/iqn.20...mple:444/tpg1> acls/
```
Create an ACL. Either use the initiator name found in /etc/iscsi/initiatorname.iscsi on the initiator, or if using a name that is easier to remember, refer to Section 24.2, “Creating an iSCSI Initiator” to ensure ACL matches the initiator. For example:
```
/iscsi/iqn.20...444/tpg1/acls> create iqn.2006-04.com.example.foo:888
Created Node ACL for iqn.2006-04.com.example.foo:888
Created mapped LUN 2.
Created mapped LUN 1.
Created mapped LUN 0.
```
Note
The above example's behavior depends on the setting used. In this case, the global setting auto_add_mapped_luns is used. This automatically maps LUNs to any created ACL.

Show the changes.

/iscsi/iqn.20...444/tpg1/acls> ls
o- acls .................................................[1 ACL]
    o- iqn.2006-04.com.example.foo:888 ....[3 Mapped LUNs, auth]
        o- mapped_lun0 .............[lun0 ramdisk/ramdisk1 (rw)]
        o- mapped_lun1 .................[lun1 block/block1 (rw)]
        o- mapped_lun2 .................[lun2 fileio/file1 (rw)]

⁠24.1.7. Fibre Channel over Ethernet (FCoE) Target Setup

In addition to mounting LUNs over FCoE, as described in Section 24.4, “Configuring a Fibre Channel over Ethernet Interface”, exporting LUNs to other machines over FCoE is also supported with the aid of targetcli.

Important

Before proceeding, refer to Section 24.4, “Configuring a Fibre Channel over Ethernet Interface” and verify that basic FCoE setup is completed, and that fcoeadm -i displays configured FCoE interfaces.

⁠

Procedure 24.6. Configure FCoE target

Setting up an FCoE target requires the installation of the targetcli package, along with its dependencies. Refer to Section 24.1, “Target Setup” for more information on targetcli basics and set up.
Create an FCoE target instance on an FCoE interface.
```
/> tcm_fc/ create 00:11:22:33:44:55:66:77
```
If FCoE interfaces are present on the system, tab-completing after create will list available interfaces. If not, ensure fcoeadm -i shows active interfaces.
Map a backstore to the target instance.
⁠
Example 24.1. Example of mapping a backstore to the target instance
```
/> tcm_fc/00:11:22:33:44:55:66:77
```
```
/> luns/ create /backstores/fileio/example2
```
Allow access to the LUN from an FCoE initiator.
```
/> acls/ create 00:99:88:77:66:55:44:33
```
The LUN should now be accessible to that initiator.
To make the changes persistant across reboots, use the saveconfig command and type yes when prompted. If this is not done the configuration will be lost after rebooting.
Exit targetcli by typing exit or entering ctrl+D.

⁠24.1.8. Remove objects with `targetcli`

To remove an backstore use the command:

/> /backstores/backstore-type/backstore-name

To remove parts of an iSCSI target, such as an ACL, use the following command:

/> /iscsi/iqn-name/tpg/acls/ delete iqn-name

To remove the entire target, including all ACLs, LUNs, and portals, use the following command:

/> /iscsi delete iqn-name

⁠24.1.9. `targetcli` References

For more information on targetcli, refer to the following resources:

man targetcli: The targetcli man page. It includes an example walk through.
The Linux SCSI Target Wiki: http://linux-iscsi.org/wiki/Targetcli
Screencast by Andy Grover: https://www.youtube.com/watch?v=BkBGTBadOO8
Note
This was uploaded on February 28, 2012. As such, the service name has changed from targetcli to target.