How to troubleshoot device-mapper: table: 253:7: multipath: error getting device?
version：redhat 6.4
problem：
[root@j1 ~]#dmesg
[Hardware Error]: Machine check events logged
[Hardware Error]: Machine check events logged
[Hardware Error]: Machine check events logged
[Hardware Error]: Machine check events logged
ISO 9660 Extensions: Microsoft Joliet Level 3
ISO 9660 Extensions: RRIP_1991A
device-mapper: table: 253:7: multipath: error getting device
device-mapper: ioctl: error adding target to table
device-mapper: table: 253:7: multipath: error getting device
device-mapper: ioctl: error adding target to table
device-mapper: table: 253:7: multipath: error getting device
device-mapper: ioctl: error adding target to table
device-mapper: table: 253:7: multipath: error getting device
device-mapper: ioctl: error adding target to table
[root@j1 ~]# dmsetup info -C
Name Maj Min Stat Open Targ Event UUID
vg00-LogVol01 253 5 L--w 1 1 0 LVM-eJV2Rjx11AacJLzet6tcb2z2O9PTUku64U1mtUXXTcl72B6Iu6W5V6Hin7uTNOkw
vg00-LogVol00 253 0 L--w 1 1 0 LVM-eJV2Rjx11AacJLzet6tcb2z2O9PTUku6EJrCeqVbJoyKw44HELUYjF9N2dVlHc4Q
mpathd 253 4 L--w 33 1 1 mpath-36001438009b064580000400000710000
mpathc 253 3 L--w 9 1 1 mpath-36001438009b064580000400000650000
mpathb 253 2 L--w 63 1 1 mpath-36001438009b0645800004000006b0000
vg00-LogVol03 253 1 L--w 1 1 0 LVM-eJV2Rjx11AacJLzet6tcb2z2O9PTUku6HogLq442U3F5A2Kj4PDAOm56R1Krex2j
vg00-LogVol02 253 6 L--w 1 1 0 LVM-eJV2Rjx11AacJLzet6tcb2z2O9PTUku6wQS2Nd3n04mFad51Lx1Y8VBdqIxHq0KV
[root@j1 ~]# vgs
VG #PV #LV #SN Attr VSize VFree
vg00 1 4 0 wz--n- 558.68g 369.23g
[root@j1 ~]# pvs
PV VG Fmt Attr PSize PFree
/dev/sda2 vg00 lvm2 a-- 558.68g 369.23g
[root@j1 ~]# lvs
LV VG Attr LSize Pool Origin Data% Move Log Cpy%Sync Convert
LogVol00 vg00 -wi-ao--- 97.66g
LogVol01 vg00 -wi-ao--- 9.77g
LogVol02 vg00 -wi-ao--- 19.53g
LogVol03 vg00 -wi-ao--- 62.50g
[root@j1 ~]# lsmod |grep dm_multipath
dm_multipath 17756 4 dm_round_robin
dm_mod 82839 22 dm_multipath,dm_mirror,dm_log
root@j1 ~]# grep -v ^$ /etc/multipath.conf
defaults {
udev_dir /dev
polling_interval 10
path_selector "round-robin 0"
path_grouping_policy failover
getuid_callout "/lib/udev/scsi_id --whitelisted --device=/dev/%n"
prio alua
path_checker tur
rr_min_io 100
rr_min_io_rq 1
rr_weight uniform
failback immediate
no_path_retry 12
user_friendly_names yes
}
blacklist {
devnode "^(ram|raw|loop|fd|md|dm-|sr|scd|st)[0-9]"
devnode "^hd[a-z][[0-9]]"
devnode "^cciss!c[0-9]d[0-9]"
devnode "^sda[0-9]"
}
devices {
device {
vendor "HP"
product "OPEN-."
path_grouping_policy multibus
getuid_callout "/lib/udev/scsi_id --whitelisted --device=/dev/%n"
path_selector "round-robin 0"
path_checker tur
features "0"
hardware_handler "0"
prio const
failback immediate
rr_weight uniform
no_path_retry queue
rr_min_io 1000
rr_min_io_rq 1
}
}
Responses
Red Hat
Guru
3240 points
Hi,
Please refer to Why do I see "multipath: error getting device" and "device-mapper: ioctl: error adding target to table" during bootup in RHEL? to see whether the provided solution works for you.
Red Hat
Guru
3240 points
Also, it's better to put output and config files between ~~~ to preserve the formatting (for readability).
(I added the output you provided into the original question.)
Is your operating system installed on a multipathed device?
If so, then you should have installed RHEL6 using a special installation boot option mpath, otherwise multipathing is not started within initramfs: if your root filesystem is on a multipathed disk, it causes LVM to grab one of the individual paths and lock onto it. When the dm-multipath is started a bit later, it cannot get exclusive access to all individual paths of the multipathed system disk because LVM is already locked onto one of the individual paths.
If this is your problem, it might be better to reinstall the OS unless you're very familiar with multipathing and LVM: it can be fixed without reinstallation, but the procedure is a bit tricky.
The first step is to edit /etc/sysconfig/mkinitrd/multipath to contain:
MULTIPATH=yes
and rebuilding your initrd using the mkinitrd command.
This will cause dm-multipath to be started earlier in the boot sequence, when the system is still running on initrd, before LVM volume groups are activated. As a result, dm-multipath gets to claim the individual paths of the multipath devices first, then LVM can lock on to the multipathed device instead of one of the individual /dev/sd* paths.
For LVM this is not a problem, as it will scan all the disk devices it can find anyway.
But for plain old disk partitions (think /boot!), this will make the original device name (typically /dev/sda1) inaccessible, as you should now be using the equivalent multipathed device name, i.e. /dev/mapper/mpath*p1, or the UUID= syntax in /etc/fstab to specify the filesystem. As a result, the system will probably drop into single user mode because it cannot mount the /boot filesystem.
So be aware of this, and be prepared to get on the system console to fix your /etc/fstab to match the new situation the first time you boot the system after making this change.
Welcome! Check out the Getting Started with Red Hat page for quick tours and guides for common tasks.