systemd-udevd failing on multipath devices

Solution Verified - Updated -

Issue

  • systemd-udevd process is failing on multipath devices and shows below errors.
    During following timestamps the CPU utilization on systems was also reached up to 90-100%

    Aug 11 15:22:08 host1 systemd-udevd: timeout: killing '/sbin/kpartx -a /dev/dm-173' [7491]
    Aug 11 15:22:08 host1 systemd-udevd: timeout: killing '/sbin/kpartx -a /dev/dm-131' [7484]
    Aug 11 15:22:08 host1 systemd-udevd: timeout: killing '/sbin/kpartx -a /dev/dm-185' [7521]
    Aug 11 15:22:08 host1 systemd-udevd: timeout: killing '/sbin/kpartx -a /dev/dm-179' [7520]
    Aug 11 15:22:08 host1 systemd-udevd: timeout: killing '/sbin/kpartx -a /dev/dm-174' [7522]
    Aug 11 15:22:09 host1 systemd-udevd: timeout: killing '/sbin/kpartx -a /dev/dm-129' [7492]
    Aug 11 15:22:09 host1 systemd-udevd: timeout: killing '/sbin/kpartx -a /dev/dm-173' [7491]
    Aug 11 15:22:09 host1 systemd-udevd: timeout: killing '/sbin/kpartx -a /dev/dm-131' [7484]
    Aug 11 15:22:09 host1 systemd-udevd: timeout: killing '/sbin/kpartx -a /dev/dm-185' [7521]
    Aug 11 15:22:09 host1 systemd-udevd: timeout: killing '/sbin/kpartx -a /dev/dm-179' [7520]
    Aug 11 15:22:09 host1 systemd-udevd: timeout: killing '/sbin/kpartx -a /dev/dm-174' [7522]
    Aug 11 15:22:10 host1 systemd-udevd: timeout: killing '/sbin/kpartx -a /dev/dm-129' [7492]
    
  • The ps aux output also shows lots of udev processes stuck in D (Uninterruptible) state

    $ ps aux|grep -i udev|grep D
    root      1178  0.0  0.0      0     0 ?        D    08:17   0:00 [systemd-udevd]
    root      1179  0.0  0.0  44928  3296 ?        D    08:17   0:01 /usr/lib/systemd/systemd-udevd
    root      1182  0.0  0.0      0     0 ?        D    08:17   0:00 [systemd-udevd]
    root      1183  0.0  0.0      0     0 ?        D    08:17   0:01 [systemd-udevd]
    root      1184  0.0  0.0  44788  3340 ?        D    08:17   0:01 /usr/lib/systemd/systemd-udevd
    root      1185  0.0  0.0  44920  3376 ?        D    08:17   0:01 /usr/lib/systemd/systemd-udevd
    root      1186  0.0  0.0      0     0 ?        D    08:17   0:00 [systemd-udevd]
    root      1187  0.0  0.0  44788  3344 ?        D    08:17   0:00 /usr/lib/systemd/systemd-udevd
    root      1188  0.0  0.0      0     0 ?        D    08:17   0:00 [systemd-udevd]
    root      1189  0.0  0.0  44788  3192 ?        D    08:17   0:00 /usr/lib/systemd/systemd-udevd
    root      1190  0.0  0.0  44788  3152 ?        D    08:17   0:01 /usr/lib/systemd/systemd-udevd
    root      1193  0.0  0.0      0     0 ?        D    08:17   0:00 [systemd-udevd]
    root      1198  0.0  0.0  44788  3248 ?        D    08:17   0:00 /usr/lib/systemd/systemd-udevd
    root      1199  0.0  0.0  44788  3352 ?        D    08:17   0:00 /usr/lib/systemd/systemd-udevd
    root      1201  0.0  0.0  44788  3280 ?        D    08:17   0:00 /usr/lib/systemd/systemd-udevd
    root      1202  0.0  0.0  44788  3116 ?        D    08:17   0:00 /usr/lib/systemd/systemd-udevd
    root      1203  0.0  0.0  44788  3248 ?        D    08:17   0:00 /usr/lib/systemd/systemd-udevd
    root      1207  0.0  0.0  44788  3240 ?        D    08:17   0:00 /usr/lib/systemd/systemd-udevd
    root      1208  0.0  0.0  44788  3220 ?        D    08:17   0:00 /usr/lib/systemd/systemd-udevd
    root      1209  0.0  0.0  44788  3208 ?        D    08:17   0:00 /usr/lib/systemd/systemd-udevd
    root      1210  0.0  0.0  44788  3232 ?        D    08:17   0:00 /usr/lib/systemd/systemd-udevd
    

Environment

  • Red Hat Enterprise Linux 7
  • System with large number of SAN devices

Subscriber exclusive content

A Red Hat subscription provides unlimited access to our knowledgebase, tools, and much more.

Current Customers and Partners

Log in for full access

Log In

New to Red Hat?

Learn more about Red Hat subscriptions

Using a Red Hat product through a public cloud?

How to access this content