When rebooting my RHEL 5 server, not all multipath device partitions are created

Solution Verified - Updated -

Environment

  • Red Hat Enterprise Linux (RHEL) 5
  • device-mapper-multipath

Issue

  • When rebooting my server a previously configured partition on an mpath device (such as mpath1p1) does not reappear after reboot.
  • A file system backed by a partition on a multipath device is not being mounted automatically on boot
  • One or more partitions on multipath devices are not being mapped by device-mapper after boot

Resolution

Update to at least device-mapper-multipath/kpartx 0.4.7-34.el5_5.6 or later.

If the issue is not resolved by the above mentioned update, then the system may be susceptible to a separate but similar problem during bootup. See the Diagnostic Steps below for more information. This issue can be worked around by manually running kpartx -a <device> after the system has booted. For information on a more permanent fix, please contact Red Hat Global Support Services.

Root Cause

Due to a race condition between udev and device-mapper-multipath's kpartx utility, kpartx may have attempted to create partitions on a multipath device before it was completely initialized, preventing mpath device partitions from being created. With the above mentioned update, udev now waits for the multipath device to be completely initialized before attempting to create partitions on it.

A separate problem was discovered later which may result in similar symptoms. This is caused by a timing issue in /etc/rc.sysinit in which devices may not be fully initialized by the time kpartx is run.

The original issue that caused this was addressed by Red Hat Engineering via Bugzilla #634903 and #568127. The later-discovered timing problem in /etc/rc.sysinit is currently being tracked by BZ #853149.

Diagnostic Steps

  • If multipath partition-mapping issues still exist after updating to the above mentioned release, try adding udevsettle to the /etc/rc.sysinit script as shown below, which may resolve a separate timing issue that exists during bootup.
/sbin/multipath.static -v 0
/sbin/udevsettle   ## ADD THIS LINE ##
if [ -x /sbin/kpartx ]; then
        /sbin/dmsetup ls --target multipath --exec "/sbin/kpartx -a -p p"
fi
  • NOTE: This change will be overwritten with any update to the initscripts package.

This solution is part of Red Hat’s fast-track publication program, providing a huge library of solutions that Red Hat engineers have created while supporting our customers. To give you the knowledge you need the instant it becomes available, these articles may be presented in a raw and unedited form.

Comments