Chapter 5. Managing multipathed volumes

DM-Multipath provides a variety of tools and commands you can use to manage multipath volumes.

5.1. The multipath command

The multipath command is used to detect and coalesce multiple paths to devices. It provides a variety of option you can use to administer your multipathed devices.

Table 5.1, “Useful multipath Command Options” describes some options of the multipath command that you may find useful.

Table 5.1. Useful multipath Command Options

Option	Description
`-l`	Display the current multipath configuration gathered from `sysfs` and the device mapper.
`-ll`	Display the current multipath configuration gathered from `sysfs`, the device mapper, and all other available components on the system.
`-f device`	Remove the named multipath device.
`-F`	Remove all unused multipath devices.
`-w device`	Remove the `wwid` of the specified device from the `wwids` file.
`-W`	Reset the `wwids` file to include only the current multipath devices.

5.1.1. Multipath command output

When you create, modify, or list a multipath device, you get a display of the current device setup. The format is as follows.

For each multipath device:

action_if_any: alias (wwid_if_different_from_alias) dm_device_name_if_known vendor,product size=size features='features' hwhandler='hardware_handler' wp=write_permission_if_known

For each path group:

-+- policy='scheduling_policy' prio=prio_if_known status=path_group_status_if_known

For each path:

 `- host:channel:id:lun devnode major:minor dm_status_if_known path_status online_status

For example, the output of a multipath command might appear as follows:

3600d0230000000000e13955cc3757800 dm-1 WINSYS,SF2372
size=269G features='0' hwhandler='0' wp=rw
|-+- policy='round-robin 0' prio=1 status=active
| `- 6:0:0:0 sdb 8:16  active ready  running
`-+- policy='round-robin 0' prio=1 status=enabled
  `- 7:0:0:0 sdf 8:80  active ready  running

If the path is up and ready for I/O, the status of the path is ready or ghost. If the path is down, the status is faulty or shaky. The path status is updated periodically by the multipathd daemon based on the polling interval defined in the /etc/multipath.conf file.

Additional possible path status values are as follows.

i/o pending: The checker is actively checking this path, and the state will be updated shortly.
i/o timeout: This is the same as faulty. It lets the user know that the checker did not return either success or failure before the timeout period.
removed: The path has been removed from the system, and will shortly be removed from the multipath device. It is treated the same as faulty.
wild: multipathd was unable to run the path checker, because of an internal error or configuration issue. This is roughly the same as faulty, except multipath will skip many actions on the path.
unchecked: The path checker has not run on this path, either because it has just been discovered, it does not have an assigned path checker, or the path checker encountered an error. This is treated the same as wild.
delayed: The path checker returned that the path is up, but multipath is delaying the reinstatement of the path because the path has recently failed multiple times and multipath has been configured to delay paths in this case.

The dm status is similar to the path status, but from the kernel’s point of view. The active dm state covers the ready and ghost path states. The pending path state has no equivalent dm state. All other path states map to the failed dm state. The dm state will retain its current status until the path checker has completed.

The possible values for online_status are running and offline. A status of offline means that this SCSI device has been disabled.

Note

When a multipath device is being created or modified, the path group status, the dm device name, the write permissions, and the dm status are not known. Also, the features are not always correct.

5.1.2. Displaying multipath configuration

You can use the -l and -ll options of the multipath command to display the current multipath configuration. The -l option displays multipath topology gathered from information in sysfs and the device mapper. The -ll option displays the information the -l option displays in addition to all other available components of the system.

When displaying the multipath configuration, you can specify a verbosity level with the -v option of the multipath command. Specifying -v0 yields no output. Specifying -v1 outputs the created or updated multipath names only, which you can then feed to other tools such as kpartx. Specifying -v2 prints all detected paths, multipaths, and device maps. For even more detailed information, you can also specify -v3, -v4, or -v5.

The following example shows the output of a multipath -l command.

# multipath -l
3600d0230000000000e13955cc3757800 dm-1 WINSYS,SF2372
size=269G features='0' hwhandler='0' wp=rw
|-- policy='round-robin 0' prio=1 status=active
| `- 6:0:0:0 sdb 8:16  active ready  running
`-- policy='round-robin 0' prio=1 status=enabled
  `- 7:0:0:0 sdf 8:80  active ready  running

The following example shows the output of a multipath -ll command.

# multipath -ll
3600d0230000000000e13955cc3757801 dm-10 WINSYS,SF2372
size=269G features='0' hwhandler='0' wp=rw
|-- policy='round-robin 0' prio=1 status=enabled
| `- 19:0:0:1 sdc 8:32  active ready  running
`-- policy='round-robin 0' prio=1 status=enabled
  `- 18:0:0:1 sdh 8:112 active ready  running
3600d0230000000000e13955cc3757803 dm-2 WINSYS,SF2372
size=125G features='0' hwhandler='0' wp=rw
`-+- policy='round-robin 0' prio=1 status=active
  |- 19:0:0:3 sde 8:64  active ready  running
  `- 18:0:0:3 sdj 8:144 active ready  running

5.2. Resizing an online multipath device

If you need to resize an online multipath device, use the following procedure.

Resize your physical device.
Execute the following command to find the paths to the LUN:
```
# multipath -l
```
Resize your paths. For SCSI devices, writing a 1 to the rescan file for the device causes the SCSI driver to rescan, as in the following command:
```
# echo 1 > /sys/block/path_device/device/rescan
```
Ensure that you run this command for each of the path devices. For example, if your path devices are sda, sdb, sde, and sdf, you would run the following commands:
```
# echo 1 > /sys/block/sda/device/rescan
# echo 1 > /sys/block/sdb/device/rescan
# echo 1 > /sys/block/sde/device/rescan
# echo 1 > /sys/block/sdf/device/rescan
```
Resize your multipath device by executing the multipathd resize command:
```
# multipathd resize map multipath_device
```
Resize the file system (assuming no LVM or DOS partitions are used):
```
# resize2fs /dev/mapper/mpatha
```

5.3. Moving a root file system from a single path device to a multipath device

If you have installed your system on a single-path device and later add another path to the root file system, you will need to move your root file system to a multipathed device. This section documents the procedure for moving from a single-path to a multipathed device.

After ensuring that you have installed the device-mapper-multipath package, perform the following procedure:

Execute the following command to create the /etc/multipath.conf configuration file, load the multipath module, and set chkconfig for the multipathd to on:
```
# mpathconf --enable
```
If the find_multipaths configuration parameter is not set to yes, edit the blacklist and blacklist_exceptions sections of the /etc/multipath.conf file, as described in Section 4.3, “Blacklisting devices from DM Multipath”.
In order for multipath to build a multipath device on top of the root device as soon as it is discovered, enter the following command. This command also ensures that find_multipaths will allow the device, even if it only has one path.
```
# multipath -a root_devname
```
For example, if the root device is /dev/sdb, enter the following command.
```
# multipath -a /dev/sdb
wwid '3600d02300069c9ce09d41c4ac9c53200' added
```
To confirm that your configuration file is set up correctly, you can enter the multipath command and search the output for a line of the following format. This indicates that the command failed to create the multipath device.
```
date  wwid: ignoring map
```
For example, if the WWID if the device is 3600d02300069c9ce09d41c4ac9c53200, you would see a line in the output such as the following:
```
# multipath
Oct 21 09:37:19 | 3600d02300069c9ce09d41c4ac9c53200: ignoring map
```
To rebuild the initramfs file system with multipath, execute the dracut command with the following options:
```
# dracut --force -H --add multipath
```
Shut the machine down.
Configure the FC switch so that other paths are visible to the machine.
Boot the machine.
Check whether the root file system ('/') is on the multipathed device.

5.4. Moving a swap file system from a single path device to a multipath device

By default, swap devices are set up as logical volumes. This does not require any special procedure for configuring them as multipath devices as long as you set up multipathing on the physical volumes that constitute the logical volume group. If your swap device is not an LVM volume, however, and it is mounted by device name, you may need to edit the /etc/fstab file to switch to the appropriate multipath device name.

Determine the WWID number of the swap device by running the /sbin/multipath command with the -v3 option. The output from the command should show the swap device in the paths list.
You should look in the command output for a line of the following format, showing the swap device:
```
WWID  H:B:T:L devname MAJOR:MINOR
```
For example, if your swap file system is set up on sda or one of its partitions, you would see a line in the output such as the following:
```
===== paths list =====
...
1ATA     WDC WD800JD-75MSA3                           WD-WMAM9F 1:0:0:0 sda 8:0
...
```

Set up an alias for the swap device in the /etc/multipath.conf file:

multipaths {
    multipath {
        wwid WWID_of_swap_device
        alias swapdev
    }
}

Edit the /etc/fstab file and replace the old device path to the root device with the multipath device.
For example, if you had the following entry in the /etc/fstab file:
```
/dev/sda2 swap                    swap    defaults        0 0
```
You would change the entry to the following:
```
/dev/mapper/swapdev swap          swap    defaults        0 0
```

5.5. Determining device mapper entries with the dmsetup command

You can use the dmsetup command to find out which device mapper entries match the multipathed devices.

The following command displays all the device mapper devices and their major and minor numbers. The minor numbers determine the name of the dm device. For example, a minor number of 3 corresponds to the multipathed device /dev/dm-3.

# dmsetup ls
mpathd  (253:4)
mpathep1        (253:12)
mpathfp1        (253:11)
mpathb  (253:3)
mpathgp1        (253:14)
mpathhp1        (253:13)
mpatha  (253:2)
mpathh  (253:9)
mpathg  (253:8)
VolGroup00-LogVol01     (253:1)
mpathf  (253:7)
VolGroup00-LogVol00     (253:0)
mpathe  (253:6)
mpathbp1        (253:10)
mpathd  (253:5)

5.6. Administering the multipathd daemon

The multipathd commands can be used to administer the multipathd daemon. For information on the available multipathd commands, see the multipathd(8) man page.

The following command shows the standard default format for the output of the multipathd show maps command.

# multipathd show maps
name sysfs uuid
mpathc dm-0 360a98000324669436c2b45666c567942

Some multipathd commands include a format option followed by a wildcard. You can display a list of available wildcards with the following command.

# multipathd show wildcards

The multipathd command supports format commands that show the status of multipath devices and paths in "raw" format versions. In raw format, no headers are printed and the fields are not padded to align the columns with the headers. Instead, the fields print exactly as specified in the format string. This output can then be more easily used for scripting. You can display the wildcards used in the format string with the multipathd show wildcards command.

The following multipathd commands show the multipath devices that multipathd is monitoring, using a format string with multipath wildcards, in regular and raw format.

list|show maps|multipaths format $format
list|show maps|multipaths raw format $format

The following multipathd commands show the paths that multipathd is monitoring, using a format string with multipath wildcards, in regular and raw format.

list|show paths format $format
list|show paths raw format $format

The following commands show the difference between the non-raw and raw formats for the multipathd show maps. Note that in raw format there are no headers and only a single space between the columns.

# multipathd show maps format "%n %w %d %s"
name   uuid                              sysfs vend/prod/rev
mpathc 360a98000324669436c2b45666c567942 dm-0  NETAPP,LUN

# multipathd show maps raw format "%n %w %d %s"
mpathc 360a98000324669436c2b45666c567942 dm-0 NETAPP,LUN

5.7. Cleaning up multipath files on package removal

If you should have occasion to remove the device-mapper-multipath rpm. file, note that this does not remove the /etc/multipath.conf, /etc/multipath/bindings, and /etc/multipath/wwids files. You may need to remove those files manually on subsequent installations of the device-mapper-multipath package.

Log in to Your Red Hat Account

Red Hat Account

Customer Portal

Select Your Language

Infrastructure and Management

Cloud Computing

Storage

Runtimes

Integration and Automation

Services

Tools

Red Hat Insights

Red Hat Product Security Center

Security Updates

Resources

Customer Portal Community

Customer Events

Stories

Chapter 5. Managing multipathed volumes

5.1. The multipath command

5.1.1. Multipath command output

5.1.2. Displaying multipath configuration

5.2. Resizing an online multipath device

5.3. Moving a root file system from a single path device to a multipath device

5.4. Moving a swap file system from a single path device to a multipath device

5.5. Determining device mapper entries with the dmsetup command

5.6. Administering the multipathd daemon

5.7. Cleaning up multipath files on package removal