Red Hat Training

A Red Hat training course is available for RHEL 8

Chapter 1. Overview of device mapper multipathing

DM Multipath provides:

Redundancy
DM Multipath can provide failover in an active/passive configuration. In an active/passive configuration, only a subset of the paths is used at any time for I/O. If any element of an I/O path such as the cable, switch, or controller fails, DM Multipath switches to an alternate path.
Note

The number of paths is dependent on the setup. Usually, DM Multipath setups have 2, 4, or 8 paths to the storage, but this is a common setup and other numbers are possible for the paths.

Improved Performance
DM Multipath can be configured in an active/active mode, where I/O is spread over the paths in a round-robin fashion. In some configurations, DM Multipath can detect loading on the I/O paths and dynamically rebalance the load.

1.1. Active/Passive multipath configuration with one RAID device

In this configuration, there are two Host Bus Adapters (HBAs) on the server, two SAN switches, and two RAID controllers. Following are the possible failure in this configuration:

  • HBA failure
  • Fibre Channel cable failure
  • SAN switch failure
  • Array controller port failure

With DM Multipath configured, a failure at any of these points causes DM Multipath to switch to the alternate I/O path. The following image describes the configuration with two I/O paths from the server to a RAID device. Here, there is one I/O path that goes through hba1, SAN1, and cntrlr1 and a second I/O path that goes through hba2, SAN2, and cntrlr2.

Figure 1.1. Active/Passive multipath configuration with one RAID device

Active/Passive multipath configuration with one RAID device

1.2. Active/Passive multipath configuration with two RAID devices

In this configuration, there are two HBAs on the server, two SAN switches, and two RAID devices with two RAID controllers each. With DM Multipath configured, a failure at any of the points of the I/O path to either of the RAID devices causes DM Multipath to switch to the alternate I/O path for that device. The following image describes the configuration with two I/O paths to each RAID device. Here, there are two I/O paths to each RAID device.

Figure 1.2. Active/Passive multipath configuration with two RAID device

Active/Passive multipath configuration with two RAID device

1.3. Active/Active multipath configuration with one RAID device

In this configuration, there are two HBAs on the server, two SAN switches, and two RAID controllers. The following image describes the configuration with two I/O paths from the server to a storage device. Here, I/O can be spread among these two paths.

Figure 1.3. Active/Active multipath configuration with one RAID device

Active/Active multipath configuration with one RAID device

1.4. DM Multipath components

The following table describes the DM Multipath components.

Table 1.1. Components of DM Multipath

Component

Description

dm_multipath kernel module

Reroutes I/O and supports failover for paths and path groups.

mpathconf utility

Configures and enables device mapper multipathing.

multipath command

Lists and configures the multipath devices. It is also executed by udev whenever a block device is added, to determine if the device should be part of a multipath device or not.

multipathd daemon

Automatically creates and removes multipath devices and monitors paths; as paths fail and come back, it may update the multipath device. Allows interactive changes to multipath devices. Reload the service if there are any changes to the /etc/multipath.conf file.

kpartx command

Creates device mapper devices for the partitions on a device. This command is automatically executed by udev when multipath devices are created to create partition devices on top of them. The kpartx command is provided in its own package, but the device-mapper-multipath package depends on it.

mpathpersist

Sets up SCSI-3 persistent reservations on multipath devices. This command works similarly to the way sg_persist works for SCSI devices that are not multipathed, but it handles setting persistent reservations on all paths of a multipath device. It coordinates with multipathd to ensure that the reservations are set up correctly on paths that are added later. To use this functionality, the reservation_key attribute must be defined in the /etc/multipath.conf file. Otherwise the multipathd daemon will not check for persistent reservations for newly discovered paths or reinstated paths.

1.5. The multipath command

The multipath command is used to detect and combine multiple paths to devices. It provides a variety of options you can use to administer your multipathed devices.

The following table describes some options of the multipath command that you may find useful.

Table 1.2. Useful multipath command options

OptionDescription

-l

Display the current multipath topology gathered from sysfs and the device mapper.

-ll

Display the current multipath topology gathered from sysfs, the device mapper, and all other available components on the system.

-f device

Remove the named multipath device.

-F

Remove all unused multipath devices.

-w device

Remove the wwid of the specified device from the wwids file.

-W

Reset the wwids file to include only the current multipath devices.

-r

Force reload of the multipath device.

1.6. Displaying multipath topology

To effectively monitor paths, troubleshoot multipath issues, or check whether the multipath configurations are set correctly, you can display the multipath topology.

Procedure

  1. Display the multipath device topology:

    # multipath -ll
    mpatha (3600d0230000000000e13954ed5f89300) dm-4 WINSYS,SF2372
    size=233G features='1 queue_if_no_path' hwhandler='0' wp=rw
    `-+- policy='service-time 0' prio=1 status=active
      `- 6:0:0:0 sdf 8:80 active ready running

    The output can be split into three parts. Each part displays information for the following group:

    • Multipath device information:

      • mpatha (3600d0230000000000e13954ed5f89300): alias (wwid if it’s different from the alias)
      • dm-4: dm device name
      • WINSYS,SF2372: vendor, product
      • size=233G: size
      • features='1 queue_if_no_path': features
      • hwhandler='0': hardware handler
      • wp=rw: write permissions
    • Path group information:

      • policy='service-time 0': scheduling policy
      • prio=1: path group priority
      • status=active: path group status
    • Path information:

      • 6:0:0:0: host:channel:id:lun
      • sdf: devnode
      • 8:80: major:minor numbers
      • active: dm status
      • ready: path status
      • running: online status

        For more information about the dm, path and online status, see Path status.

Other multipath commands, which are used to list, create, or reload multipath devices, also display the device topology. However, some information might be unknown and shown as undef in the output. This is normal behavior. Use the multipath -ll command to view the correct state.

Note

In certain cases, such as creating a multipath device, the multipath topology displays a parameter, which represents if any action was taken. For example, the following command output shows the create: parameter to represent that a multipath device was created:

create: mpatha (3600d0230000000000e13954ed5f89300) undef WINSYS,SF2372
size=233G features='1 queue_if_no_path' hwhandler='0' wp=undef
`-+- policy='service-time 0' prio=1 status=undef
  `- 6:0:0:0 sdf 8:80 undef ready running

1.7. Path status

The path status is updated periodically by the multipathd daemon based on the polling interval defined in the /etc/multipath.conf file. In terms of the kernel, the dm status is similar to the path status. The dm state will retain its current status until the path checker has completed.

Path status
ready, ghost
The path is up and ready for I/O.
faulty, shaky
The path is down.
i/o pending
The checker is actively checking this path, and the state will be updated shortly.
i/o timeout
The checker did not return success/failure before the timeout period. This is treated the same as faulty.
removed
The path has been removed from the system, and will shortly be removed from the multipath device. This is treated the same as faulty.
wild
multipathd was unable to run the path checker, because of an internal error or configuration issue. This is treated the same as faulty, except multipath will skip many actions on the path.
unchecked
The path checker has not run on this path, either because it has just been discovered, it does not have an assigned path checker, or the path checker encountered an error. This is treated the same as wild.
delayed
The path checker returns that the path is up, but multipath is delaying the reinstatement of the path because the path has recently failed multiple times and multipath has been configured to delay paths in this case. This is treated the same as faulty.
Dm status
Active
Maps to the ready and ghost path status.
Failed
Maps to all other path status, except i/o pending that does not have an equivalent dm state.
Online status
Running
The device is enabled.
Offline
The device has been disabled.

1.8. Additional resources

  • multipath(8) and multipathd(8) man pages
  • /etc/multipath.conf file