Array information is not recognised in multipath after adding new LUNs.

Solution Verified - Updated -

Issue

  • System taking long time to boot
  • Array information is not recognized in multipath after adding new LUNs.
    mpathlb (eui.14xxyy30123456789a123456789a19760) dm-324 ##,##
    size=50G features='0' hwhandler='0' wp=rw
    `-+- policy='service-time 0' prio=0 status=active
      |- #:#:#:#        nvme2n314 259:610 active faulty running
      `- #:#:#:#        nvme1n314 259:658 active faulty running
    mpathdz (eui.14260123456789a123456789a19760) dm-140 ##,##
    size=50G features='0' hwhandler='0' wp=rw
    `-+- policy='service-time 0' prio=0 status=active
      |- #:#:#:#        nvme2n121 259:236 active faulty running
      `- #:#:#:#        nvme1n121 259:261 active faulty running
    mpathak (eui.142650123456789a123456789a0019760) dm-47 ##,##
    size=50G features='0' hwhandler='0' wp=rw
    `-+- policy='service-time 0' prio=0 status=active
      |- #:#:#:#        nvme2n64 259:129 active faulty running
      `- #:#:#:#        nvme1n64 259:144 active faulty running
    mpathr (eui.1426501xyza6789a123456789a019760) dm-20 NVME,EMC PowerMax_8000
    size=100G features='0' hwhandler='0' wp=rw
    `-+- policy='service-time 0' prio=50 status=active
      |- 2:2177:19:8531 nvme2n19 259:25 active ready running
      `- 1:3268:19:8531 nvme1n19 259:47 active ready running
  • udevd fails to iterate over all the attached devices and following logs could be seen
Aug  3 02:45:36 hostname systemd-udevd[2083]: seq 10137 '/devices/virtual/nvme-fabrics/ctl/nvme1/nvme1n217' is taking a long time
Aug  3 02:45:36 hostname systemd-udevd[2083]: seq 10141 '/devices/virtual/nvme-fabrics/ctl/nvme1/nvme1n218' is taking a long time
Aug  3 02:45:36 hostname systemd-udevd[2083]: seq 10146 '/devices/virtual/nvme-fabrics/ctl/nvme0/nvme0n249' is taking a long time
Aug  3 02:45:36 hostname systemd-udevd[2083]: seq 10148 '/devices/virtual/nvme-fabrics/ctl/nvme1/nvme1n219' is taking a long time
Aug  3 02:45:36 hostname systemd-udevd[2083]: seq 10143 '/devices/virtual/nvme-fabrics/ctl/nvme0/nvme0n248' is taking a long time
Aug  3 02:45:36 hostname systemd-udevd[2083]: seq 10149 '/devices/virtual/nvme-fabrics/ctl/nvme0/nvme0n250' is taking a long time
Aug  3 02:45:36 hostname systemd-udevd[2083]: seq 10139 '/devices/virtual/nvme-fabrics/ctl/nvme0/nvme0n247' is taking a long time


Jun 10 15:05:07 hostname systemd-udevd[793]: seq 19855 '/devices/pci0000:c0/0000:c0:03.1/0000:c2:00.0/host0/port-0:0/expander-0:0/port-0:0:1/expander-0:1/port-0:1:2/end_device-0:1:2/target0:0:3/0:0:3:0/block/sdc' is taking a long time
Jun 10 15:05:07 hostname systemd-udevd[793]: seq 19853 '/devices/pci0000:c0/0000:c0:03.1/0000:c2:00.0/host0/port-0:0/expander-0:0/port-0:0:1/expander-0:1/port-0:1:9/end_device-0:1:9/target0:0:10/0:0:10:0/block/sdj' is taking a long time

We may encounter udevd symlinks failure notification

Jun 10 15:06:05 hostname systemd-udevd[932]: Failed to update device symlinks: Too many levels of symbolic links
Jun 10 15:06:06 hostname systemd-udevd[1010]: Failed to update device symlinks: Too many levels of symbolic links
Jun 10 15:06:06 hostname systemd-udevd[925]: Failed to update device symlinks: Too many levels of symbolic links

This may result in multipathd failure messages such as

Jun 10 15:10:07 hostname multipathd[3120]: sdej: failed to get udev uid: Invalid argument
Jun 10 15:10:07 hostname multipathd[3120]: sdej: failed to get unknown uid: Invalid argument
Jun 10 15:10:07 hostname multipathd[3120]: sdej: failed to get path uid
Jun 10 15:10:07 hostname multipathd[3120]: sdee: failed to get udev uid: 
  • udevadm info does not report complete information for all the devices
P: /devices/virtual/nvme-subsystem/nvme-subsys0/nvme0n1
N: nvme0n1
S: disk/by-id/nvme-EMC_PowerMax_8000_00000197601426
S: disk/by-id/nvme-eui.142xxx303230303000yyyy6000019760
E: DEVLINKS=/dev/disk/by-id/nvme-EMC_PowerMax_8000_00000197601426 /dev/disk/by-id/nvme-eui.142xxx303230303000yyyy6000019760
E: DEVNAME=/dev/nvme0n1
E: DEVPATH=/devices/virtual/nvme-subsystem/nvme-subsys0/nvme0n1
E: DEVTYPE=disk
E: DM_MULTIPATH_DEVICE_PATH=0
E: ID_MODEL=EMC PowerMax_8000
E: ID_SERIAL=EMC PowerMax_8000_00000197601426
E: ID_SERIAL_SHORT=00000197601426
E: ID_WWN=eui.142xxx303230303000yyyy6000019760
E: ID_WWN_WITH_EXTENSION=eui.142xxx303230303000yyyy6000019760
E: MAJOR=259
E: MINOR=1
E: MPATH_SBIN_PATH=/sbin
E: SUBSYSTEM=block
E: TAGS=:systemd:
E: USEC_INITIALIZED=61301362

------------------- 

P: /devices/virtual/nvme-subsystem/nvme-subsys0/nvme0n433
N: nvme0n433
E: DEVNAME=/dev/nvme0n433
E: DEVPATH=/devices/virtual/nvme-subsystem/nvme-subsys0/nvme0n433
E: DEVTYPE=disk
E: MAJOR=259
E: MINOR=1002
E: SUBSYSTEM=block

Environment

  • Red Hat Enterprise Linux 8.4
    • udev

Subscriber exclusive content

A Red Hat subscription provides unlimited access to our knowledgebase, tools, and much more.

Current Customers and Partners

Log in for full access

Log In

New to Red Hat?

Learn more about Red Hat subscriptions

Using a Red Hat product through a public cloud?

How to access this content