Procedure

1. Start the interactive parted shell:

   # parted block-device

   • Replace block-device with the path to the device you want to examine: for example, /dev/sda.

2. View the partition table:

   (parted) print

3. Optionally, use the following command to switch to another device you want to examine next:

   (parted) select block-device

Procedure

1. Start the interactive parted shell:

   # parted block-device

   • Replace block-device with the path to the device where you want to create a partition table: for example, /dev/sda.

2. Determine if there already is a partition table on the device:

   (parted) print

   If the device already contains partitions, they will be deleted in the next steps.

3. Create the new partition table:

   (parted) mklabel table-type

   • Replace table-type with with the intended partition table type:

     • msdos for MBR
     • gpt for GPT

   Example 3.2. Creating a GPT table

   For example, to create a GPT table on the disk, use:

   (parted) mklabel gpt

   The changes start taking place as soon as you enter this command, so review it before executing it.

4. View the partition table to confirm that the partition table exists:

   (parted) print

5. Exit the parted shell:

   (parted) quit

Procedure

1. Start the interactive parted shell:

   # parted block-device

   • Replace block-device with the path to the device where you want to create a partition: for example, /dev/sda.

2. View the current partition table to determine if there is enough free space:

   (parted) print

   • If there is not enough free space, you can resize an existing partition. For more information, see Section 3.5, “Resizing a partition”.

   • From the partition table, determine:

     • The start and end points of the new partition
     • On MBR, what partition type it should be.

3. Create the new partition:

   (parted) mkpart part-type name fs-type start end

   • Replace part-type with with primary, logical, or extended based on what you decided from the partition table. This applies only to the MBR partition table.
   • Replace name with an arbitrary partition name. This is required for GPT partition tables.
   • Replace fs-type with any one of xfs, ext2, ext3, ext4, fat16, fat32, hfs, hfs+, linux-swap, ntfs, or reiserfs. The fs-type parameter is optional. Note that parted does not create the file system on the partition.
   • Replace start and end with the sizes that determine the starting and ending points of the partition, counting from the beginning of the disk. You can use size suffixes, such as 512MiB, 20GiB, or 1.5TiB. The default size megabytes.

   Example 3.3. Creating a small primary partition

   For example, to create a primary partition from 1024MiB until 2048MiB on an MBR table, use:

   (parted) mkpart primary 1024MiB 2048MiB

   The changes start taking place as soon as you enter this command, so review it before executing it.

4. View the partition table to confirm that the created partition is in the partition table with the correct partition type, file system type, and size:

   (parted) print

5. Exit the parted shell:

   (parted) quit

6. Use the following command to wait for the system to register the new device node:

   # udevadm settle

7. Verify that the kernel recognizes the new partition:

   # cat /proc/partitions

Procedure

1. Start the interactive fdisk shell:

   # fdisk block-device

   • Replace block-device with the path to the device where you want to set a partition type: for example, /dev/sda.

2. View the current partition table to determine the minor partition number:

   Command (m for help): print

   You can see the current partition type in the Type column and its corresponding type ID in the Id column.

3. Enter the partition type command and select a partition using its minor number:

   Command (m for help): type
   Partition number (1,2,3 default 3): 2

4. Optionally, list the available hexadecimal codes:

   Hex code (type L to list all codes): L

5. Set the partition type:

   Hex code (type L to list all codes): 8e

6. Write your changes and exit the fdisk shell:

   Command (m for help): write
   The partition table has been altered.
   Syncing disks.

7. Verify your changes:

   # fdisk --list block-device

Procedure

1. Start the interactive parted shell:

   # parted block-device

   • Replace block-device with the path to the device where you want to remove a partition: for example, /dev/sda.

2. View the current partition table to determine the minor number of the partition to remove:

   (parted) print

3. Remove the partition:

   (parted) rm minor-number

   • Replace minor-number with the minor number of the partition you want to remove: for example, 3.

   The changes start taking place as soon as you enter this command, so review it before executing it.

4. Confirm that the partition is removed from the partition table:

   (parted) print

5. Exit the parted shell:

   (parted) quit

6. Verify that the kernel knows the partition is removed:

   # cat /proc/partitions

7. Remove the partition from the /etc/fstab file if it is present. Find the line that declares the removed partition, and remove it from the file.

8. Regenerate mount units so that your system registers the new /etc/fstab configuration:

   # systemctl daemon-reload

9. If you have deleted a swap partition or removed pieces of LVM, remove all references to the partition from the kernel command line in the /etc/default/grub file and regenerate GRUB configuration:

   • On a BIOS-based system:

     # grub2-mkconfig --output=/etc/grub2.cfg

   • On a UEFI-based system:

     # grub2-mkconfig --output=/etc/grub2-efi.cfg

10. To register the changes in the early boot system, rebuild the initramfs file system:

    # dracut --force --verbose

Procedure

1. If you want to shrink the partition, shrink the file system on it first so that it is not larger than the resized partition. Note that XFS does not support shrinking.

2. Start the interactive parted shell:

   # parted block-device

   • Replace block-device with the path to the device where you want to resize a partition: for example, /dev/sda.

3. View the current partition table:

   (parted) print

   From the partition table, determine:

   • The minor number of the partition
   • The location of the existing partition and its new ending point after resizing

4. Resize the partition:

   (parted) resizepart minor-number new-end

   • Replace minor-number with the minor number of the partition that you are resizing: for example, 3.
   • Replace new-end with the size that determines the new ending point of the resized partition, counting from the beginning of the disk. You can use size suffixes, such as 512MiB, 20GiB, or 1.5TiB. The default size megabytes.

   Example 3.4. Extending a partition

   For example, to extend a partition located at the beginning of the disk to be 2GiB in size, use:

   (parted) resizepart 1 2GiB

   The changes start taking place as soon as you enter this command, so review it before executing it.

5. View the partition table to confirm that the resized partition is in the partition table with the correct size:

   (parted) print

6. Exit the parted shell:

   (parted) quit

7. Verify that the kernel recognizes the new partition:

   # cat /proc/partitions

8. If you extended the partition, extend the file system on it as well. See (reference) for details.

Procedure

1. Reconfigure the selected namespace to sector mode:

   # ndctl create-namespace \
           --force \
           --reconfig=namespace-ID \
           --mode=sector

   Example 5.1. Reconfiguring namespace1.0 in sector mode

   To reconfigure the namespace1.0 namespace to use sector mode:

   # ndctl create-namespace \
           --force \
           --reconfig=namespace1.0 \
           --mode=sector
   
   {
     "dev":"namespace1.0",
     "mode":"sector",
     "size":"11.99 GiB (12.87 GB)",
     "uuid":"5805480e-90e6-407e-96a4-23e1cde2ed78",
     "raw_uuid":"879d9e9f-fd43-4ed5-b64f-3bcd0781391a",
     "sector_size":4096,
     "blockdev":"pmem1s",
     "numa_node":1
   }

2. The reconfigured namespace is now available under the /dev directory as /dev/pmemNs.

Procedure

1. List the pmem regions on your system that have available space. In the following example, space is available in the region5 and region4 regions:

   # ndctl list --regions
   
   [
     {
       "dev":"region5",
       "size":270582939648,
       "available_size":270582939648,
       "type":"pmem",
       "iset_id":-7337419320239190016
     },
     {
       "dev":"region4",
       "size":270582939648,
       "available_size":270582939648,
       "type":"pmem",
       "iset_id":-137289417188962304
     }
   ]

2. On any of the available regions, allocate one or more namespaces:

   # ndctl create-namespace \
           --mode=sector \
           --region=regionN \
           --size=namespace-size

   Example 5.2. Creating a namespace on a region

   The following command creates a 36-GiB sector namespace on region4:

   # ndctl create-namespace \
           --mode=sector \
           --region=region4 \
           --size=36G

3. The new namespace is now available under the /dev directory as /dev/pmemNs.

Procedure

1. List all namespaces on your system:

   # ndctl list --namespaces --idle
   
   [
     {
       "dev":"namespace1.0",
       "mode":"raw",
       "size":34359738368,
       "state":"disabled",
       "numa_node":1
     },
     {
       "dev":"namespace0.0",
       "mode":"raw",
       "size":34359738368,
       "state":"disabled",
       "numa_node":0
     }
   ]

2. Reconfigure any namespace:

   # ndctl create-namespace \
           --force \
           --mode=devdax \
           --reconfig=namespace-ID

   Example 5.3. Reconfiguring a namespace as device DAX

   The following command reconfigures namespace0.0 for data storage that supports DAX. It is aligned to a 2-MiB fault granularity to ensure that the operating system faults in 2-MiB pages at a time:

   # ndctl create-namespace \
           --force \
           --mode=devdax \
           --align=2M \
           --reconfig=namespace0.0

3. The namespace is now available at the /dev/daxN.M path.

Procedure

1. List the pmem regions on your system that have available space. In the following example, space is available in the region5 and region4 regions:

   # ndctl list --regions
   
   [
     {
       "dev":"region5",
       "size":270582939648,
       "available_size":270582939648,
       "type":"pmem",
       "iset_id":-7337419320239190016
     },
     {
       "dev":"region4",
       "size":270582939648,
       "available_size":270582939648,
       "type":"pmem",
       "iset_id":-137289417188962304
     }
   ]

2. On any of the available regions, allocate one or more namespaces:

   # ndctl create-namespace \
           --mode=devdax \
           --region=regionN \
           --size=namespace-size

   Example 5.4. Creating a namespace on a region

   The following command creates a 36-GiB device DAX namespace on region4. It is aligned to a 2-MiB fault granularity to ensure that the operating system faults in 2-MiB pages at a time:

   # ndctl create-namespace \
           --mode=devdax \
           --region=region4 \
           --align=2M \
           --size=36G
   
   {
     "dev":"namespace1.2",
     "mode":"devdax",
     "map":"dev",
     "size":"35.44 GiB (38.05 GB)",
     "uuid":"5ae01b9c-1ebf-4fb6-bc0c-6085f73d31ee",
     "raw_uuid":"4c8be2b0-0842-4bcb-8a26-4bbd3b44add2",
     "daxregion":{
       "id":1,
       "size":"35.44 GiB (38.05 GB)",
       "align":2097152,
       "devices":[
         {
           "chardev":"dax1.2",
           "size":"35.44 GiB (38.05 GB)"
         }
       ]
     },
     "numa_node":1
   }

3. The namespace is now available at the /dev/daxN.M path.

Procedure

1. List all namespaces on your system:

   # ndctl list --namespaces --idle
   
   [
     {
       "dev":"namespace1.0",
       "mode":"raw",
       "size":34359738368,
       "state":"disabled",
       "numa_node":1
     },
     {
       "dev":"namespace0.0",
       "mode":"raw",
       "size":34359738368,
       "state":"disabled",
       "numa_node":0
     }
   ]

2. Reconfigure any namespace:

   # ndctl create-namespace \
           --force \
           --mode=fsdax \
           --reconfig=namespace-ID

   Example 5.5. Reconfiguring a namespace as file system DAX

   To use namespace0.0 for a file system that supports DAX, use the following command:

   # ndctl create-namespace \
           --force \
           --mode=fsdax \
           --reconfig=namespace0.0
   
   {
     "dev":"namespace0.0",
     "mode":"fsdax",
     "size":"32.00 GiB (34.36 GB)",
     "uuid":"ab91cc8f-4c3e-482e-a86f-78d177ac655d",
     "blockdev":"pmem0",
     "numa_node":0
   }

3. The namespace is now available at the /dev/pmemN path.

Procedure

1. List the pmem regions on your system that have available space. In the following example, space is available in the region5 and region4 regions:

   # ndctl list --regions
   
   [
     {
       "dev":"region5",
       "size":270582939648,
       "available_size":270582939648,
       "type":"pmem",
       "iset_id":-7337419320239190016
     },
     {
       "dev":"region4",
       "size":270582939648,
       "available_size":270582939648,
       "type":"pmem",
       "iset_id":-137289417188962304
     }
   ]

2. On any of the available regions, allocate one or more namespaces:

   # ndctl create-namespace \
           --mode=fsdax \
           --region=regionN \
           --size=namespace-size

   Example 5.6. Creating a namespace on a region

   The following command creates a 36-GiB file system DAX namespace on region4:

   # ndctl create-namespace \
           --mode=fsdax \
           --region=region4 \
           --size=36G
   
   {
     "dev":"namespace4.0",
     "mode":"fsdax",
     "size":"35.44 GiB (38.05 GB)",
     "uuid":"9c5330b5-dc90-4f7a-bccd-5b558fa881fe",
     "blockdev":"pmem4",
     "numa_node":0
   }

3. The namespace is now available at the /dev/pmemN path.

Procedure

1. Optionally, create a partition on the file system DAX device. See Section 3.3, “Creating a partition”.

   By default, the parted tool aligns partitions on 1 MiB boundaries. For the first partition, specify 2 MiB as the start of the partition. If the size of the partition is a multiple of 2 MiB, all other partitions are also aligned.

2. Create an XFS or ext4 file system on the partition or the NVDIMM device.

   For XFS, disable shared copy-on-write data extents, because they are incompatible with the dax mount option. Additionally, in order to increase the likelihood of large page mappings, set the stripe unit and stripe width.

   # mkfs.xfs -m reflink=0 -d su=2m,sw=1 fsdax-partition-or-device

3. Mount the file system with the -o dax mount option:

   # mount -o dax fsdax-partition-or-device mount-point

4. Applications can now use persistent memory and create files in the mount-point directory, open the files, and use the mmap operation to map the files for direct access.

1. Detect which NVDIMM device is failing
2. Back up data stored on it
3. Physically replace the device

Procedure

1. To detect the broken device, use the following command:

   # ndctl list --dimms --regions --health --media-errors --human

   The badblocks field shows which NVDIMM is broken. Note its name in the dev field.

   Example 5.7. Health status of NVDIMM devices

   In the following example, the NVDIMM named nmem0 is broken:

   # ndctl list --dimms --regions --health --media-errors --human
   
   ...
     "regions":[
       {
         "dev":"region0",
         "size":"250.00 GiB (268.44 GB)",
         "available_size":0,
         "type":"pmem",
         "numa_node":0,
         "iset_id":"0xXXXXXXXXXXXXXXXX",
         "mappings":[
           {
             "dimm":"nmem1",
             "offset":"0x10000000",
             "length":"0x1f40000000",
             "position":1
           },
           {
             "dimm":"nmem0",
             "offset":"0x10000000",
             "length":"0x1f40000000",
             "position":0
           }
         ],
         "badblock_count":1,
         "badblocks":[
           {
             "offset":65536,
             "length":1,
             "dimms":[
               "nmem0"
             ]
           }
         ],
         "persistence_domain":"memory_controller"
       }
     ]
   }

2. Use the following command to find the phys_id attribute of the broken NVDIMM:

   # ndctl list --dimms --human

   From the previous example, you know that nmem0 is the broken NVDIMM. Therefore, find the phys_id attribute of nmem0.

   Example 5.8. The phys_id attributes of NVDIMMs

   In the following example, the phys_id is 0x10:

   # ndctl list --dimms --human
   
   [
     {
       "dev":"nmem1",
       "id":"XXXX-XX-XXXX-XXXXXXXX",
       "handle":"0x120",
       "phys_id":"0x1c"
     },
     {
       "dev":"nmem0",
       "id":"XXXX-XX-XXXX-XXXXXXXX",
       "handle":"0x20",
       "phys_id":"0x10",
       "flag_failed_flush":true,
       "flag_smart_event":true
     }
   ]

3. Use the following command to find the memory slot of the broken NVDIMM:

   # dmidecode

   In the output, find the entry where the Handle identifier matches the phys_id attribute of the broken NVDIMM. The Locator field lists the memory slot used by the broken NVDIMM.

   Example 5.9. NVDIMM Memory Slot Listing

   In the following example, the nmem0 device matches the 0x0010 identifier and uses the DIMM-XXX-YYYY memory slot:

   # dmidecode
   
   ...
   Handle 0x0010, DMI type 17, 40 bytes
   Memory Device
           Array Handle: 0x0004
           Error Information Handle: Not Provided
           Total Width: 72 bits
           Data Width: 64 bits
           Size: 125 GB
           Form Factor: DIMM
           Set: 1
           Locator: DIMM-XXX-YYYY
           Bank Locator: Bank0
           Type: Other
           Type Detail: Non-Volatile Registered (Buffered)
   ...

4. Back up all data in the namespaces on the NVDIMM. If you do not back up the data before replacing the NVDIMM, the data will be lost when you remove the NVDIMM from your system.

   Warning

   In some cases, such as when the NVDIMM is completely broken, the backup might fail.

   To prevent this, regularly monitor your NVDIMM devices using S.M.A.R.T. as described in Section 5.8.2, “Monitoring NVDIMM health using S.M.A.R.T.” and replace failing NVDIMMs before they break.

   Use the following command to list the namespaces on the NVDIMM:

   # ndctl list --namespaces --dimm=DIMM-ID-number

   Example 5.10. NVDIMM namespaces listing

   In the following example, the nmem0 device contains the namespace0.0 and namespace0.2 namespaces, which you need to back up:

   # ndctl list --namespaces --dimm=0
   
   [
     {
       "dev":"namespace0.2",
       "mode":"sector",
       "size":67042312192,
       "uuid":"XXXXXXXX-XXXX-XXXX-XXXX-XXXXXXXXXXXX",
       "raw_uuid":"XXXXXXXX-XXXX-XXXX-XXXX-XXXXXXXXXXXX",
       "sector_size":4096,
       "blockdev":"pmem0.2s",
       "numa_node":0
     },
     {
       "dev":"namespace0.0",
       "mode":"sector",
       "size":67042312192,
       "uuid":"XXXXXXXX-XXXX-XXXX-XXXX-XXXXXXXXXXXX",
       "raw_uuid":"XXXXXXXX-XXXX-XXXX-XXXX-XXXXXXXXXXXX",
       "sector_size":4096,
       "blockdev":"pmem0s",
       "numa_node":0
     }
   ]

5. Replace the broken NVDIMM physically.

Procedure

1. Install targetcli:

   # yum install targetcli

2. Start the target service:

   # systemctl start target

3. Configure target to start at boot time:

   # systemctl enable target

4. Open port 3260 in the firewall and reload the firewall configuration:

   # firewall-cmd --permanent --add-port=3260/tcp
   Success
   
   # firewall-cmd --reload
   Success

5. View the targetcli layout:

   # targetcli
   /> ls
   o- /........................................[...]
     o- backstores.............................[...]
     | o- block.................[Storage Objects: 0]
     | o- fileio................[Storage Objects: 0]
     | o- pscsi.................[Storage Objects: 0]
     | o- ramdisk...............[Storage Objects: 0]
     o- iscsi...........................[Targets: 0]
     o- loopback........................[Targets: 0]

Procedure

1. Navigate to the iSCSI directory:

   /> iscsi/

   Note

   The cd command is used to change directories as well as to list the path to move into.

2. Use one of the following options to create an iSCSI target:

   1. Creating an iSCSI target using a default target name:

      /iscsi> create
      
      Created target
      iqn.2003-01.org.linux-iscsi.hostname.x8664:sn.78b473f296ff
      Created TPG1

   2. Creating an iSCSI target using a specific name:

      /iscsi> create iqn.2006-04.com.example:444
      
      Created target iqn.2006-04.com.example:444
      Created TPG1
      Here iqn.2006-04.com.example:444 is target_iqn_name

      Replace iqn.2006-04.com.example:444 with the specific target name.

3. Verify the newly created target:

   /iscsi> ls
   
   o- iscsi.......................................[1 Target]
       o- iqn.2006-04.com.example:444................[1 TPG]
           o- tpg1...........................[enabled, auth]
              o- acls...............................[0 ACL]
               o- luns...............................[0 LUN]
              o- portals.........................[0 Portal]

Procedure

1. Navigate to the backstores directory:

   /> backstores/

2. Create a fileio storage object:

   /> backstores/fileio create file1 /tmp/disk1.img 200M write_back=false
   
   Created fileio file1 with size 209715200

3. Verify the created fileio storage object:

   /backstores> ls

Procedure

1. Navigate to the backstores directory:

   /> backstores/

2. Create a block backstore:

   /> backstores/block create name=block_backend dev=/dev/sdb
   
   Generating a wwn serial.
   Created block storage object block_backend using /dev/vdb.

3. Verify the created block storage object:

   /backstores> ls

   Note

   You can also create a block backstore on a logical volume.

Procedure

1. Navigate to the backstores directory:

   /> backstores/

2. Create a pscsi backstore for a physical SCSI device, a TYPE_ROM device using /dev/sr0 in this example:

   /> backstores/pscsi/ create name=pscsi_backend dev=/dev/sr0
   
   Generating a wwn serial.
   Created pscsi storage object pscsi_backend using /dev/sr0

3. Verify the created pscsi storage object:

   /backstores> ls

Procedure

1. Navigate to the backstores directory:

   /> backstores/

2. Create a 1GB RAM disk backstore:

   /> backstores/ramdisk/ create name=rd_backend size=1GB
   
   Generating a wwn serial.
   Created rd_mcp ramdisk rd_backend with size 1GB.

3. Verify the created ramdisk storage object:

   /backstores> ls

Procedure

1. Navigate to the TPG directory:

   /iscsi> iqn.2006-04.example:444/tpg1/

2. Use one of the following options to create an iSCSI portal:

   1. Creating a default portal uses the default iSCSI port 3260 and allows the target to listen to all IP addresses on that port:

      /iscsi/iqn.20...mple:444/tpg1> portals/ create
      
      Using default IP port 3260
      Binding to INADDR_Any (0.0.0.0)
      Created network portal 0.0.0.0:3260

      Note

      When an iSCSI target is created, a default portal is also created. This portal is set to listen to all IP addresses with the default port number that is: 0.0.0.0:3260.

      To remove the default portal:

      /iscsi/iqn-name/tpg1/portals delete ip_address=0.0.0.0 ip_port=3260

   2. Creating a portal using a specific IP address:

      /iscsi/iqn.20...mple:444/tpg1> portals/ create 192.168.122.137
      
      Using default IP port 3260
      Created network portal 192.168.122.137:3260

3. Verify the newly created portal:

   /iscsi/iqn.20...mple:444/tpg1> ls
   
   o- tpg.................................. [enambled, auth]
       o- acls ......................................[0 ACL]
       o- luns ......................................[0 LUN]
       o- portals ................................[1 Portal]
          o- 192.168.122.137:3260......................[OK]

Procedure

1. Create LUNs of already created storage objects:

   /iscsi/iqn.20...mple:444/tpg1> luns/ create /backstores/ramdisk/rd_backend
   Created LUN 0.
   
   /iscsi/iqn.20...mple:444/tpg1> luns/ create /backstores/block/block_backend
   Created LUN 1.
   
   /iscsi/iqn.20...mple:444/tpg1> luns/ create /backstores/fileio/file1
   Created LUN 2.

2. Verify the created LUNs:

   /iscsi/iqn.20...mple:444/tpg1> ls
   
   o- tpg.................................. [enambled, auth]
       o- acls ......................................[0 ACL]
       o- luns .....................................[3 LUNs]
       |  o- lun0.........................[ramdisk/ramdisk1]
       |  o- lun1.................[block/block1 (/dev/vdb1)]
       |  o- lun2...................[fileio/file1 (/foo.img)]
       o- portals ................................[1 Portal]
           o- 192.168.122.137:3260......................[OK]

   Default LUN name starts at 0.

   Important

   By default, LUNs are created with read-write permissions. If a new LUN is added after ACLs are created, LUN automatically maps to all available ACLs and can cause a security risk. To create a LUN with read-only permissions, see Section 7.1.10, “Creating a read-only iSCSI LUN”.

3. Configure ACLs. For more information, see Section 7.1.11, “Creating an iSCSI ACL”.

Procedure

1. Set read-only permissions:

   /> set global auto_add_mapped_luns=false
   
   Parameter auto_add_mapped_luns is now 'false'.

   This prevents the auto mapping of LUNs to existing ACLs allowing the manual mapping of LUNs.

2. Create the LUN:

   /> iscsi/target_iqn_name/tpg1/acls/initiator_iqn_name/ create mapped_lun=next_sequential_LUN_number tpg_lun_or_backstore=backstore write_protect=1

   Example:

   /> iscsi/iqn.2006-04.example:444/tpg1/acls/2006-04.com.example.foo:888/ create mapped_lun=1 tpg_lun_or_backstore=/backstores/block/block2 write_protect=1
   
   Created LUN 1.
   Created Mapped LUN 1.

3. Verify the created LUN:

   /> ls
   
   o- / ...................................................... [...]
     o- backstores ........................................... [...]
     <snip>
     o- iscsi ......................................... [Targets: 1]
     | o- iqn.2006-04.example:444 .................. [TPGs: 1]
     |   o- tpg1 ............................ [no-gen-acls, no-auth]
     |     o- acls ....................................... [ACLs: 2]
     |     | o- 2006-04.com.example.foo:888 .. [Mapped LUNs: 2]
     |     | | o- mapped_lun0 .............. [lun0 block/disk1 (rw)]
     |     | | o- mapped_lun1 .............. [lun1 block/disk2 (ro)]
     |     o- luns ....................................... [LUNs: 2]
     |     | o- lun0 ...................... [block/disk1 (/dev/vdb)]
     |     | o- lun1 ...................... [block/disk2 (/dev/vdc)]
     <snip>

   The mapped_lun1 line now has (ro) at the end (unlike mapped_lun0’s (rw)) stating that it is read-only.

4. Configure ACLs. For more information, see Section 7.1.11, “Creating an iSCSI ACL”.

Procedure

1. Navigate to the acls directory

   /iscsi/iqn.20...mple:444/tpg1> acls/

2. Use one of the following options to create an ACL :

   1. Using the initiator name from /etc/iscsi/initiatorname.iscsi file on the initiator.

   2. Using a name that is easier to remember, see section Section 7.1.12, “Creating an iSCSI initiator” to ensure ACL matches the initiator.

      /iscsi/iqn.20...444/tpg1/acls> create iqn.2006-04.com.example.foo:888
      
      Created Node ACL for iqn.2006-04.com.example.foo:888
      Created mapped LUN 2.
      Created mapped LUN 1.
      Created mapped LUN 0.

      Note

      The global setting auto_add_mapped_luns used in the preceding example, automatically maps LUNs to any created ACL.

      You can set user-created ACLs within the TPG node on the target server:

      /iscsi/iqn.20...scsi:444/tpg1> set attribute generate_node_acls=1

3. Verify the created ACL:

   /iscsi/iqn.20...444/tpg1/acls> ls
   
   o- acls .................................................[1 ACL]
       o- iqn.2006-04.com.example.foo:888 ....[3 Mapped LUNs, auth]
           o- mapped_lun0 .............[lun0 ramdisk/ramdisk1 (rw)]
           o- mapped_lun1 .................[lun1 block/block1 (rw)]
           o- mapped_lun2 .................[lun2 fileio/file1 (rw)]

Procedure

1. Install iscsi-initiator-utils on client machine:

   # yum install iscsi-initiator-utils

2. Check the initiator name:

   # cat /etc/iscsi/initiatorname.iscsi
   
   InitiatorName=2006-04.com.example.foo:888

3. If the ACL was given a custom name in Section 7.1.11, “Creating an iSCSI ACL”, modify the /etc/iscsi/initiatorname.iscsi file accordingly.

   # vi /etc/iscsi/initiatorname.iscsi

4. Discover the target and log in to the target with the displayed target IQN:

   # iscsiadm -m discovery -t st -p 10.64.24.179
       10.64.24.179:3260,1 iqn.2006-04.example:444
   
   # iscsiadm -m node -T iqn.2006-04.example:444 -l
       Logging in to [iface: default, target: iqn.2006-04.example:444, portal: 10.64.24.179,3260] (multiple)
       Login to [iface: default, target: iqn.2006-04.example:444, portal: 10.64.24.179,3260] successful.

   Replace 10.64.24.179 with the target-ip-address.

   You can use this procedure for any number of initiators connected to the same target if their respective initiator names are added to the ACL as described in the Section 7.1.11, “Creating an iSCSI ACL”.

5. Find the iSCSI disk name and create a file system on this iSCSI disk:

   # grep "Attached SCSI" /var/log/messages
   
   # mkfs.ext4 /dev/disk_name

   Replace disk_name with the iSCSI disk name displayed in the /var/log/messages file.

6. Mount the file system:

   # mkdir /mount/point
   
   # mount /dev/disk_name /mount/point

   Replace /mount/point with the mount point of the partition.

7. Edit the /etc/fstab file to mount the file system automatically when the system boots:

   # vi /etc/fstab
   
   /dev/disk_name /mount/point ext4 _netdev 0 0

   Replace disk_name with the iSCSI disk name and /mount/point with the mount point of the partition.

Procedure

1. Set attribute authentication:

   /iscsi/iqn.20...mple:444/tpg1> set attribute authentication=1
   
   Parameter authentication is now '1'.

2. Set userid and password:

   /tpg1> set auth userid=redhat
   Parameter userid is now 'redhat'.
   
   /iscsi/iqn.20...689dcbb3/tpg1> set auth password=redhat_passwd
   Parameter password is now 'redhat_passwd'.

Procedure

1. Enable CHAP authentication in the iscsid.conf file:

   # vi /etc/iscsi/iscsid.conf
   
   node.session.auth.authmethod = CHAP

   By default, the node.session.auth.authmethod is set to None

2. Add target username and password in the iscsid.conf file:

   node.session.auth.username = redhat
   node.session.auth.password = redhat_passwd

3. Start the iscsid daemon:

   # systemctl start iscsid.service

Procedure

1. Find information about the running sessions:

   # iscsiadm -m session -P 3

   This command displays the session or device state, session ID (sid), some negotiated parameters, and the SCSI devices accessible through the session.

   • For shorter output, for example, to display only the sid-to-node mapping, run:

     # iscsiadm -m session -P 0
             or
     # iscsiadm -m session
     
     tcp [2] 10.15.84.19:3260,2 iqn.1992-08.com.netapp:sn.33615311
     tcp [3] 10.15.85.19:3260,3 iqn.1992-08.com.netapp:sn.33615311

     These commands print the list of running sessions in the following format: driver [sid] target_ip:port,target_portal_group_tag proper_target_name.

Procedure

1. Log off from the target:

   # iscsiadm -m node -T iqn.2006-04.example:444 -u

   For more information on how to log in to the target, see Section 7.1.12, “Creating an iSCSI initiator”.

2. Remove the entire target, including all ACLs, LUNs, and portals:

   /> iscsi/ delete iqn.2006-04.com.example:444

   Replace iqn.2006-04.com.example:444 with the target_iqn_name.

   • To remove an iSCSI backstore:

     /> backstores/backstore-type/ delete block_backend

     • Replace backstore-type with either fileio, block, pscsi, or ramdisk.
     • Replace block_backend with the backstore-name you want to delete.

   • To remove parts of an iSCSI target, such as an ACL:

     /> /iscsi/iqn-name/tpg/acls/ delete iqn.2006-04.com.example:444

3. View the changes:

   /> iscsi/ ls

Procedure

1. Disable swapping for the associated logical volume:

   # swapoff -v /dev/VolGroup00/LogVol01

2. Resize the LVM2 logical volume by 2 GB:

   # lvresize /dev/VolGroup00/LogVol01 -L +2G

3. Format the new swap space:

   # mkswap /dev/VolGroup00/LogVol01

4. Enable the extended logical volume:

   # swapon -v /dev/VolGroup00/LogVol01

5. To test if the swap logical volume was successfully extended and activated, inspect active swap space:

   $ cat /proc/swaps
   $ free -h

Procedure

1. Create the LVM2 logical volume of size 2 GB:

   # lvcreate VolGroup00 -n LogVol02 -L 2G

2. Format the new swap space:

   # mkswap /dev/VolGroup00/LogVol02

3. Add the following entry to the /etc/fstab file:

   /dev/VolGroup00/LogVol02 swap swap defaults 0 0

4. Regenerate mount units so that your system registers the new configuration:

   # systemctl daemon-reload

5. Activate swap on the logical volume:

   # swapon -v /dev/VolGroup00/LogVol02

6. To test if the swap logical volume was successfully created and activated, inspect active swap space:

   $ cat /proc/swaps
   $ free -h

Procedure

1. Determine the size of the new swap file in megabytes and multiply by 1024 to determine the number of blocks. For example, the block size of a 64 MB swap file is 65536.

2. Create an empty file:

   # dd if=/dev/zero of=/swapfile bs=1024 count=65536

   Replace count with the value equal to the desired block size.

3. Set up the swap file with the command:

   # mkswap /swapfile

4. Change the security of the swap file so it is not world readable.

   # chmod 0600 /swapfile

5. To enable the swap file at boot time, edit /etc/fstab as root to include the following entry:

   /swapfile swap swap defaults 0 0

   The next time the system boots, it activates the new swap file.

6. Regenerate mount units so that your system registers the new /etc/fstab configuration:

   # systemctl daemon-reload

7. To activate the swap file immediately:

   # swapon /swapfile

8. To test if the new swap file was successfully created and activated, inspect active swap space:

   $ cat /proc/swaps
   $ free -h

Procedure

1. Disable swapping for the associated logical volume:

   # swapoff -v /dev/VolGroup00/LogVol01

2. Reduce the LVM2 logical volume by 512 MB:

   # lvreduce /dev/VolGroup00/LogVol01 -L -512M

3. Format the new swap space:

   # mkswap /dev/VolGroup00/LogVol01

4. Activate swap on the logical volume:

   # swapon -v /dev/VolGroup00/LogVol01

5. To test if the swap logical volume was successfully reduced, inspect active swap space:

   $ cat /proc/swaps
   $ free -h

Procedure

1. Disable swapping for the associated logical volume:

   # swapoff -v /dev/VolGroup00/LogVol02

2. Remove the LVM2 logical volume:

   # lvremove /dev/VolGroup00/LogVol02

3. Remove the following associated entry from the /etc/fstab file:

   /dev/VolGroup00/LogVol02 swap swap defaults 0 0

4. Regenerate mount units so that your system registers the new configuration:

   # systemctl daemon-reload

5. To test if the logical volume was successfully removed, inspect active swap space:

   $ cat /proc/swaps
   $ free -h

Procedure

1. At a shell prompt, execute the following command to disable the swap file (where /swapfile is the swap file):

   # swapoff -v /swapfile

2. Remove its entry from the /etc/fstab file accordingly.

3. Regenerate mount units so that your system registers the new configuration:

   # systemctl daemon-reload

4. Remove the actual file:

   # rm /swapfile

Procedure

1. Create the nvmet-rdma subsystem:

   # modprobe nvmet-rdma
   
   # mkdir /sys/kernel/config/nvmet/subsystems/testnqn
   
   # cd /sys/kernel/config/nvmet/subsystems/testnqn

   Replace testnqn with the subsystem name.

2. Allow any host to connect to this target:

   # echo 1 > attr_allow_any_host

3. Configure a namespace:

   # mkdir namespaces/10
   
   # cd namespaces/10

   Replace 10 with the namespace number

4. Set a path to the NVMe device:

   #echo -n /dev/nvme0n1 > device_path

5. Enable the namespace:

   # echo 1 > enable

6. Create a directory with an NVMe port:

   # mkdir /sys/kernel/config/nvmet/ports/1
   
   # cd /sys/kernel/config/nvmet/ports/1

7. Display the IP address of mlx5_ib0:

   # ip addr show mlx5_ib0
   
   8: mlx5_ib0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 4092 qdisc mq state UP group default qlen 256
       link/infiniband 00:00:06:2f:fe:80:00:00:00:00:00:00:e4:1d:2d:03:00:e7:0f:f6 brd 00:ff:ff:ff:ff:12:40:1b:ff:ff:00:00:00:00:00:00:ff:ff:ff:ff
       inet 172.31.0.202/24 brd 172.31.0.255 scope global noprefixroute mlx5_ib0
          valid_lft forever preferred_lft forever
       inet6 fe80::e61d:2d03:e7:ff6/64 scope link noprefixroute
          valid_lft forever preferred_lft forever

8. Set the transport address for the target:

   # echo -n 172.31.0.202 > addr_traddr

9. Set RDMA as the transport type:

   # echo rdma > addr_trtype
   
   # echo 4420 > addr_trsvcid

10. Set the address family for the port:

    # echo ipv4 > addr_adrfam

11. Create a soft link:

    # ln -s /sys/kernel/config/nvmet/subsystems/testnqn   /sys/kernel/config/nvmet/ports/1/subsystems/testnqn

Procedure

1. Install the nvmetcli package:

   # yum install nvmetcli

2. Download the rdma.json file

   # wget http://git.infradead.org/users/hch/nvmetcli.git/blob_plain/0a6b088db2dc2e5de11e6f23f1e890e4b54fee64:/rdma.json

3. Edit the rdma.json file and change the traddr value to 172.31.0.202.

4. Setup the target by loading the NVMe target configuration file:

   # nvmetcli restore rdma.json

Procedure

1. Install the nvme-cli tool:

   # yum install nvme-cli

2. Load the nvme-rdma module if its not loaded:

   # modprobe nvme-rdma

3. Discover available subsystems on the NVMe target:

   # nvme discover -t rdma -a 172.31.0.202 -s 4420
   
   Discovery Log Number of Records 1, Generation counter 2
   =====Discovery Log Entry 0======
   trtype:  rdma
   adrfam:  ipv4
   subtype: nvme subsystem
   treq:    not specified, sq flow control disable supported
   portid:  1
   trsvcid: 4420
   subnqn:  testnqn
   traddr:  172.31.0.202
   rdma_prtype: not specified
   rdma_qptype: connected
   rdma_cms:    rdma-cm
   rdma_pkey: 0x0000

4. Connect to the discovered subsystems:

   # nvme connect -t rdma -n testnqn -a 172.31.0.202 -s 4420
   
   # lsblk
   NAME                         MAJ:MIN RM   SIZE RO TYPE MOUNTPOINT
   sda                            8:0    0 465.8G  0 disk
   ├─sda1                         8:1    0     1G  0 part /boot
   └─sda2                         8:2    0 464.8G  0 part
     ├─rhel_rdma--virt--03-root 253:0    0    50G  0 lvm  /
     ├─rhel_rdma--virt--03-swap 253:1    0     4G  0 lvm  [SWAP]
     └─rhel_rdma--virt--03-home 253:2    0 410.8G  0 lvm  /home
   nvme0n1
   
   #cat /sys/class/nvme/nvme0/transport
   rdma

   Replace testnqn with the NVMe subsystem name.

   Replace 172.31.0.202 with the target IP address.

   Replace 4420 with the port number.

Procedure

1. Install the nvme-cli tool:

   # yum install nvme-cli

   This creates the hostnqn file in the /etc/nvme/ directory. The hostnqn file identifies the NVMe host.

   To generate a new hostnqn:

   #nvme gen-hostnqn

2. Find the WWNN and WWPN of the local and remote ports and use the output to find the subsystem NQN:

   # cat /sys/class/scsi_host/host*/nvme_info
   
   NVME Initiator Enabled
   XRI Dist lpfc0 Total 6144 IO 5894 ELS 250
   NVME LPORT lpfc0 WWPN x10000090fae0b5f5 WWNN x20000090fae0b5f5 DID x010f00 ONLINE
   NVME RPORT       WWPN x204700a098cbcac6 WWNN x204600a098cbcac6 DID x01050e TARGET DISCSRVC ONLINE
   
   NVME Statistics
   LS: Xmt 000000000e Cmpl 000000000e Abort 00000000
   LS XMIT: Err 00000000  CMPL: xb 00000000 Err 00000000
   Total FCP Cmpl 00000000000008ea Issue 00000000000008ec OutIO 0000000000000002
       abort 00000000 noxri 00000000 nondlp 00000000 qdepth 00000000 wqerr 00000000 err 00000000
   FCP CMPL: xb 00000000 Err 00000000

   # nvme discover --transport fc --traddr nn-0x204600a098cbcac6:pn-0x204700a098cbcac6 --host-traddr nn-0x20000090fae0b5f5:pn-0x10000090fae0b5f5
   
   Discovery Log Number of Records 2, Generation counter 49530
   =====Discovery Log Entry 0======
   trtype:  fc
   adrfam:  fibre-channel
   subtype: nvme subsystem
   treq:    not specified
   portid:  0
   trsvcid: none
   subnqn:  nqn.1992-08.com.netapp:sn.e18bfca87d5e11e98c0800a098cbcac6:subsystem.st14_nvme_ss_1_1
   traddr:  nn-0x204600a098cbcac6:pn-0x204700a098cbcac6

   Replace nn-0x204600a098cbcac6:pn-0x204700a098cbcac6 with the traddr.

   Replace nn-0x20000090fae0b5f5:pn-0x10000090fae0b5f5 with the host-traddr.

3. Connect to the NVMe target using the nvme-cli:

   # nvme connect --transport fc --traddr nn-0x204600a098cbcac6:pn-0x204700a098cbcac6 --host-traddr nn-0x20000090fae0b5f5:pn-0x10000090fae0b5f5 -n nqn.1992-08.com.netapp:sn.e18bfca87d5e11e98c0800a098cbcac6:subsystem.st14_nvme_ss_1_1

   Replace nn-0x204600a098cbcac6:pn-0x204700a098cbcac6 with the traddr.

   Replace nn-0x20000090fae0b5f5:pn-0x10000090fae0b5f5 with the host-traddr.

   Replace nqn.1992-08.com.netapp:sn.e18bfca87d5e11e98c0800a098cbcac6:subsystem.st14_nvme_ss_1_1 with the subnqn.

Procedure

1. Install the nvme-cli tool:

   # yum install nvme-cli

   This creates the hostnqn file in the /etc/nvme/ directory. The hostnqn file identifies the NVMe host.

   To generate a new hostnqn:

   #nvme gen-hostnqn

2. Remove and reload the qla2xxx module:

   # rmmod qla2xxx
   # modprobe qla2xxx

3. Find the WWNN and WWPN of the local and remote ports:

   # dmesg |grep traddr
   
   [    6.139862] qla2xxx [0000:04:00.0]-ffff:0: register_localport: host-traddr=nn-0x20000024ff19bb62:pn-0x21000024ff19bb62 on portID:10700
   [    6.241762] qla2xxx [0000:04:00.0]-2102:0: qla_nvme_register_remote: traddr=nn-0x203b00a098cbcac6:pn-0x203d00a098cbcac6 PortID:01050d

   Using these host-traddr and traddr values, find the subsystem NQN:

   nvme discover --transport fc --traddr nn-0x203b00a098cbcac6:pn-0x203d00a098cbcac6 --host-traddr nn-0x20000024ff19bb62:pn-0x21000024ff19bb62
   
   Discovery Log Number of Records 2, Generation counter 49530
   =====Discovery Log Entry 0======
   trtype:  fc
   adrfam:  fibre-channel
   subtype: nvme subsystem
   treq:    not specified
   portid:  0
   trsvcid: none
   subnqn:  nqn.1992-08.com.netapp:sn.c9ecc9187b1111e98c0800a098cbcac6:subsystem.vs_nvme_multipath_1_subsystem_468
   traddr:  nn-0x203b00a098cbcac6:pn-0x203d00a098cbcac6

   Replace nn-0x203b00a098cbcac6:pn-0x203d00a098cbcac6 with the traddr.

   Replace nn-0x20000024ff19bb62:pn-0x21000024ff19bb62 with the host-traddr.

4. Connect to the NVMe target using the nvme-cli tool:

   # nvme connect  --transport fc --traddr nn-0x203b00a098cbcac6:pn-0x203d00a098cbcac6 --host-traddr nn-0x20000024ff19bb62:pn-0x21000024ff19bb62 -n nqn.1992-08.com.netapp:sn.c9ecc9187b1111e98c0800a098cbcac6:subsystem.vs_nvme_multipath_1_subsystem_468

   Replace nn-0x203b00a098cbcac6:pn-0x203d00a098cbcac6 with the traddr.

   Replace nn-0x20000024ff19bb62:pn-0x21000024ff19bb62 with the host-traddr.

   Replace nqn.1992-08.com.netapp:sn.c9ecc9187b1111e98c0800a098cbcac6:subsystem.vs_nvme_multipath_1_subsystem_468 with the subnqn.

1. Check if the DHCP service is enabled on the server.

   • Check if the dhcp.service is running:

     # systemctl status dhcpd.service

   • If the dhcp.service is inactive, you must enable and start it:

     # systemctl enable dhcpd.service
     # systemctl start dhcpd.service

     Reboot the diskless client.

2. If the problem remains, check the DHCP configurational file /etc/dhcp/dhcpd.conf on a server. For more information, see Section 15.3, “Configuring DHCP server for diskless clients”.

3. Check if the Firewall ports are opened.

   • Check if the tftp.service is listed in active services:

     # firewall-cmd --get-active-zones
     # firewall-cmd --info-zone=public

   • If the tftp.service is not listed in active services, add it to the list:

     # firewall-cmd --add-service=tftp

   • Check if the nfs.service is listed in active services:

     # firewall-cmd --get-active-zones
     # firewall-cmd --info-zone=public

   • If the nfs.service is not listed in active services, add it to the list:

     # firewall-cmd --add-service=nfs

1. Check if the file is in place. The location on a server /var/lib/tftpboot/.

2. If the file is in place, check its permissions:

   # chmod 644 pxelinux.0

3. Check if the Firewall ports are opened.

1. Check if NFS service is enabled on a server.

   • Check if nfs.service is running:

     # systemctl status nfs.service

   • If the nfs.service is inactive, you must enable and start it:

     # systemctl enable nfs.service
     # systemctl start nfs.service

2. Check if the parameters are correct in pxelinux.cfg. For more details, see Section 15.4, “Configuring an exported file system for diskless clients”.
3. Check if the Firewall ports are opened.

Procedure

1. Ensure which disk is failing. To do so, enter the following command:

   # journalctl -k -f

   You will find a message showing you which disk has failed:

   md/raid:md0: Disk failure on sdg, disabling device.
   md/raid:md0: Operation continuing on 5 devices.

2. Press Ctrl+C on your keyboard to exit the journalctl program.

3. Add a new disk to the array. To do so, enter the following command:

   # mdadm --manage /dev/md0 --add /dev/sdh

4. Mark the failed disk as faulty. To do so, enter the following command:

   # mdadm --manage /dev/md0 --fail /dev/sdg

5. Check if the faulty disk was masked correctly by using the following command:

   # mdadm --detail /dev/md0

   At the end of the last command output you will see information about RAID disks similar to this where disk /dev/sdg has a faulty status:

       Number   Major   Minor   RaidDevice State
          0       8       16        0      active sync   /dev/sdb
          1       8       32        1      active sync   /dev/sdc
          2       8       48        2      active sync   /dev/sdd
          3       8       64        3      active sync   /dev/sde
          4       8       80        4      active sync   /dev/sdf
          6       8      112        5      active sync   /dev/sdh
   
          5       8       96        -      faulty   /dev/sdg

6. Finally, remove the faulty disk from the array. To do so, enter the following command:

   # mdadm --manage /dev/md0 --remove /dev/sdg

7. Check RAID details by using following command:

   # mdadm --detail /dev/md0

   At the end of the last command output you will see information about RAID disks similar to this:

       Number   Major   Minor   RaidDevice State
          0       8       16        0      active sync   /dev/sdb
          1       8       32        1      active sync   /dev/sdc
          2       8       48        2      active sync   /dev/sdd
          3       8       64        3      active sync   /dev/sde
          4       8       80        4      active sync   /dev/sdf
          6       8      112        5      active sync   /dev/sdh

Procedure

1. Check the log message by using the following command:

   # journalctl -k -f

   You will find a message showing you which disk has failed:

   md/raid:md0: Disk failure on sdb, disabling device.
   md/raid:md0: Operation continuing on 5 devices.

2. Press Ctrl+C on your keyboard to exit the journalctl program.

3. Add the new disk to the array as a spare one. To do so, enter the following command:

   # mdadm --manage /dev/md0 --add /dev/sdi

4. Mark the broken disk as faulty. To do so, enter the following command:

   # mdadm --manage /dev/md0 --fail /dev/sdb

5. Remove the faulty disk from the array. To do so, enter the following command:

   # mdadm --manage /dev/md0 --remove /dev/sdb

6. Check the status of the array by using the following command:

   # mdadm --detail /dev/md0

   At the end of the last command output you will see information about RAID disks similar to this:

       Number   Major   Minor   RaidDevice State
          7       8      128        0      active sync   /dev/sdi
          1       8       32        1      active sync   /dev/sdc
          2       8       48        2      active sync   /dev/sdd
          3       8       64        3      active sync   /dev/sde
          4       8       80        4      active sync   /dev/sdf
          6       8      112        5      active sync   /dev/sdh

Procedure

1. To check the array for the failed disks behavior, enter the following command:

   # echo check > /sys/block/md0/md/sync_action

   That action will check the array and write the result into the /sys/block/md0/md/sync_action file.

2. Open file /sys/block/md0/md/sync_action with the text editor of your choice and see if there is any message about disk synchronization failures.

3. To resynchronize the disks in the array, enter the following command:

   # echo repair > /sys/block/md0/md/sync_action

   This action will resynchronize the disks in the array and write the result into the /sys/block/md0/md/sync_action file.

4. To view the synchronization progress, enter the following command:

   # cat /proc/mdstat