Chapter 5. Managing multipathed volumes
DM-Multipath provides a variety of tools and commands you can use to manage multipath volumes.
5.1. The multipath command
The multipath
command is used to detect and coalesce multiple paths to devices. It provides a variety of option you can use to administer your multipathed devices.
Table 5.1, “Useful multipath
Command Options” describes some options of the multipath
command that you may find useful.
Table 5.1. Useful multipath
Command Options
Option | Description |
---|---|
|
Display the current multipath configuration gathered from |
|
Display the current multipath configuration gathered from |
| Remove the named multipath device. |
| Remove all unused multipath devices. |
|
Remove the |
|
Reset the |
5.1.1. Multipath command output
When you create, modify, or list a multipath device, you get a display of the current device setup. The format is as follows.
For each multipath device:
action_if_any: alias (wwid_if_different_from_alias) dm_device_name_if_known vendor,product size=size features='features' hwhandler='hardware_handler' wp=write_permission_if_known
For each path group:
-+- policy='scheduling_policy' prio=prio_if_known status=path_group_status_if_known
For each path:
`- host:channel:id:lun devnode major:minor dm_status_if_known path_status online_status
For example, the output of a multipath command might appear as follows:
3600d0230000000000e13955cc3757800 dm-1 WINSYS,SF2372 size=269G features='0' hwhandler='0' wp=rw |-+- policy='round-robin 0' prio=1 status=active | `- 6:0:0:0 sdb 8:16 active ready running `-+- policy='round-robin 0' prio=1 status=enabled `- 7:0:0:0 sdf 8:80 active ready running
If the path is up and ready for I/O, the status of the path is ready
or ghost
. If the path is down, the status is faulty
or shaky
. The path status is updated periodically by the multipathd
daemon based on the polling interval defined in the /etc/multipath.conf
file.
Additional possible path status values are as follows.
-
i/o pending
: The checker is actively checking this path, and the state will be updated shortly. -
i/o timeout
: This is the same asfaulty
. It lets the user know that the checker did not return either success or failure before the timeout period. -
removed
: The path has been removed from the system, and will shortly be removed from the multipath device. It is treated the same asfaulty
. -
wild
:multipathd
was unable to run the path checker, because of an internal error or configuration issue. This is roughly the same asfaulty
, except multipath will skip many actions on the path. -
unchecked
: The path checker has not run on this path, either because it has just been discovered, it does not have an assigned path checker, or the path checker encountered an error. This is treated the same aswild
. -
delayed
: The path checker returned that the path is up, but multipath is delaying the reinstatement of the path because the path has recently failed multiple times and multipath has been configured to delay paths in this case.
The dm status is similar to the path status, but from the kernel’s point of view. The active
dm state covers the ready
and ghost
path states. The pending
path state has no equivalent dm state. All other path states map to the failed
dm state. The dm state will retain its current status until the path checker has completed.
The possible values for online_status
are running
and offline
. A status of offline
means that this SCSI device has been disabled.
When a multipath device is being created or modified, the path group status, the dm device name, the write permissions, and the dm status are not known. Also, the features are not always correct.
5.1.2. Displaying multipath configuration
You can use the -l
and -ll
options of the multipath
command to display the current multipath configuration. The -l
option displays multipath topology gathered from information in sysfs
and the device mapper. The -ll
option displays the information the -l
option displays in addition to all other available components of the system.
When displaying the multipath configuration, you can specify a verbosity level with the -v
option of the multipath
command. Specifying -v0
yields no output. Specifying -v1
outputs the created or updated multipath names only, which you can then feed to other tools such as kpartx
. Specifying -v2
prints all detected paths, multipaths, and device maps. For even more detailed information, you can also specify -v3,
-v4
, or -v5
.
The following example shows the output of a multipath -l
command.
# multipath -l
3600d0230000000000e13955cc3757800 dm-1 WINSYS,SF2372
size=269G features='0' hwhandler='0' wp=rw
|-- policy='round-robin 0' prio=1 status=active
| `- 6:0:0:0 sdb 8:16 active ready running
`-- policy='round-robin 0' prio=1 status=enabled
`- 7:0:0:0 sdf 8:80 active ready running
The following example shows the output of a multipath -ll
command.
# multipath -ll
3600d0230000000000e13955cc3757801 dm-10 WINSYS,SF2372
size=269G features='0' hwhandler='0' wp=rw
|-- policy='round-robin 0' prio=1 status=enabled
| `- 19:0:0:1 sdc 8:32 active ready running
`-- policy='round-robin 0' prio=1 status=enabled
`- 18:0:0:1 sdh 8:112 active ready running
3600d0230000000000e13955cc3757803 dm-2 WINSYS,SF2372
size=125G features='0' hwhandler='0' wp=rw
`-+- policy='round-robin 0' prio=1 status=active
|- 19:0:0:3 sde 8:64 active ready running
`- 18:0:0:3 sdj 8:144 active ready running
5.2. Resizing an online multipath device
If you need to resize an online multipath device, use the following procedure.
- Resize your physical device.
Execute the following command to find the paths to the LUN:
#
multipath -l
Resize your paths. For SCSI devices, writing a 1 to the
rescan
file for the device causes the SCSI driver to rescan, as in the following command:#
echo 1 > /sys/block/path_device/device/rescan
Ensure that you run this command for each of the path devices. For example, if your path devices are
sda
,sdb
,sde
, andsdf
, you would run the following commands:#
echo 1 > /sys/block/sda/device/rescan
#echo 1 > /sys/block/sdb/device/rescan
#echo 1 > /sys/block/sde/device/rescan
#echo 1 > /sys/block/sdf/device/rescan
Resize your multipath device by executing the
multipathd resize
command:#
multipathd resize map multipath_device
Resize the file system (assuming no LVM or DOS partitions are used):
#
resize2fs /dev/mapper/mpatha
5.3. Moving a root file system from a single path device to a multipath device
If you have installed your system on a single-path device and later add another path to the root file system, you will need to move your root file system to a multipathed device. This section documents the procedure for moving from a single-path to a multipathed device.
After ensuring that you have installed the device-mapper-multipath
package, perform the following procedure:
Execute the following command to create the
/etc/multipath.conf
configuration file, load the multipath module, and setchkconfig
for themultipathd
toon
:#
mpathconf --enable
-
If the
find_multipaths
configuration parameter is not set toyes
, edit theblacklist
andblacklist_exceptions
sections of the/etc/multipath.conf
file, as described in Section 4.3, “Blacklisting devices from DM Multipath”. In order for multipath to build a multipath device on top of the root device as soon as it is discovered, enter the following command. This command also ensures that
find_multipaths
will allow the device, even if it only has one path.#
multipath -a root_devname
For example, if the root device is
/dev/sdb
, enter the following command.#
multipath -a /dev/sdb
wwid '3600d02300069c9ce09d41c4ac9c53200' addedTo confirm that your configuration file is set up correctly, you can enter the
multipath
command and search the output for a line of the following format. This indicates that the command failed to create the multipath device.date wwid: ignoring map
For example, if the WWID if the device is 3600d02300069c9ce09d41c4ac9c53200, you would see a line in the output such as the following:
#
multipath
Oct 21 09:37:19 | 3600d02300069c9ce09d41c4ac9c53200: ignoring mapTo rebuild the
initramfs
file system withmultipath
, execute thedracut
command with the following options:#
dracut --force -H --add multipath
- Shut the machine down.
- Configure the FC switch so that other paths are visible to the machine.
- Boot the machine.
- Check whether the root file system ('/') is on the multipathed device.
5.4. Moving a swap file system from a single path device to a multipath device
By default, swap devices are set up as logical volumes. This does not require any special procedure for configuring them as multipath devices as long as you set up multipathing on the physical volumes that constitute the logical volume group. If your swap device is not an LVM volume, however, and it is mounted by device name, you may need to edit the /etc/fstab
file to switch to the appropriate multipath device name.
Determine the WWID number of the swap device by running the
/sbin/multipath
command with the-v3
option. The output from the command should show the swap device in the paths list.You should look in the command output for a line of the following format, showing the swap device:
WWID H:B:T:L devname MAJOR:MINOR
For example, if your swap file system is set up on
sda
or one of its partitions, you would see a line in the output such as the following:===== paths list ===== ... 1ATA WDC WD800JD-75MSA3 WD-WMAM9F 1:0:0:0 sda 8:0 ...
Set up an alias for the swap device in the
/etc/multipath.conf
file:multipaths { multipath { wwid WWID_of_swap_device alias swapdev } }
Edit the
/etc/fstab
file and replace the old device path to the root device with the multipath device.For example, if you had the following entry in the
/etc/fstab
file:/dev/sda2 swap swap defaults 0 0
You would change the entry to the following:
/dev/mapper/swapdev swap swap defaults 0 0
5.5. Determining device mapper entries with the dmsetup command
You can use the dmsetup
command to find out which device mapper entries match the multipathed devices.
The following command displays all the device mapper devices and their major and minor numbers. The minor numbers determine the name of the dm device. For example, a minor number of 3 corresponds to the multipathed device /dev/dm-3
.
# dmsetup ls
mpathd (253:4)
mpathep1 (253:12)
mpathfp1 (253:11)
mpathb (253:3)
mpathgp1 (253:14)
mpathhp1 (253:13)
mpatha (253:2)
mpathh (253:9)
mpathg (253:8)
VolGroup00-LogVol01 (253:1)
mpathf (253:7)
VolGroup00-LogVol00 (253:0)
mpathe (253:6)
mpathbp1 (253:10)
mpathd (253:5)
5.6. Administering the multipathd daemon
The multipathd
commands can be used to administer the multipathd
daemon. For information on the available multipathd
commands, see the multipathd
(8) man page.
The following command shows the standard default format for the output of the multipathd show maps
command.
# multipathd show maps
name sysfs uuid
mpathc dm-0 360a98000324669436c2b45666c567942
Some multipathd
commands include a format
option followed by a wildcard. You can display a list of available wildcards with the following command.
# multipathd show wildcards
The multipathd
command supports format commands that show the status of multipath devices and paths in "raw" format versions. In raw format, no headers are printed and the fields are not padded to align the columns with the headers. Instead, the fields print exactly as specified in the format string. This output can then be more easily used for scripting. You can display the wildcards used in the format string with the multipathd show wildcards
command.
The following multipathd
commands show the multipath devices that multipathd
is monitoring, using a format string with multipath wildcards, in regular and raw format.
list|show maps|multipaths format $format list|show maps|multipaths raw format $format
The following multipathd
commands show the paths that multipathd
is monitoring, using a format string with multipath wildcards, in regular and raw format.
list|show paths format $format list|show paths raw format $format
The following commands show the difference between the non-raw and raw formats for the multipathd show maps
. Note that in raw
format there are no headers and only a single space between the columns.
#multipathd show maps format "%n %w %d %s"
name uuid sysfs vend/prod/rev mpathc 360a98000324669436c2b45666c567942 dm-0 NETAPP,LUN #multipathd show maps raw format "%n %w %d %s"
mpathc 360a98000324669436c2b45666c567942 dm-0 NETAPP,LUN
5.7. Cleaning up multipath files on package removal
If you should have occasion to remove the device-mapper-multipath
rpm
. file, note that this does not remove the /etc/multipath.conf
, /etc/multipath/bindings
, and /etc/multipath/wwids
files. You may need to remove those files manually on subsequent installations of the device-mapper-multipath
package.