Multiple udevd processes causing high load average
Issue
- High CPU load while running
udevadm trigger
, storage stack is impacted. -
While
udevadm trigger
is running,pvs
command has a long delay to return, and database is affected (/etc/udev/rules.d/99-udev-oracle.rules
content):SUBSYSTEM=="block",KERNEL=="emcpower*",PROGRAM=="/lib/udev/scsi_id -gu /dev/%P",RESULT=="360000960000597000004512030521234",OWNER="grid",GROUP="dba",MODE="0660" SUBSYSTEM=="block",KERNEL=="emcpower*",PROGRAM=="/lib/udev/scsi_id -gu /dev/%P",RESULT=="36000096000059700000452103062543",OWNER="grid",GROUP="dba",MODE="0660" [...]
-
When running
udevadm trigger
, system logs are filled with the output:Jan 14 11:12:13 localhost udevd[5913]: worker [34311] unexpectedly returned with status 0x0100 Jan 14 11:12:13 localhost udevd[5913]: worker [34311] failed while handling '/devices/virtual/block/emcpowerx/emcpowerx1' Jan 14 11:12:13 localhost udevd[5913]: worker [34318] unexpectedly returned with status 0x0100 Jan 14 11:12:13 localhost udevd[5913]: worker [34318] failed while handling '/devices/virtual/block/emcpowery/emcpowery1' Jan 14 11:12:13 localhost udevd[5913]: worker [34322] unexpectedly returned with status 0x0100 Jan 14 11:12:13 localhost udevd[5913]: worker [34322] failed while handling '/devices/virtual/block/emcpowere/emcpowere1' Jan 14 11:12:14 localhost udevd[5913]: worker [34328] unexpectedly returned with status 0x0100 Jan 14 11:12:14 localhost udevd[5913]: worker [34328] failed while handling '/devices/virtual/block/emcpowerf/emcpowerf1'
-
There are large number of
/sbin/udev -d
processes running. - I spot checked a few other oracle servers and saw between 1 and 3 udev processes running.
Environment
- Red Hat Enterprise Linux (RHEL) 6,7
- EMC PowerPath
- Oracle Database
Subscriber exclusive content
A Red Hat subscription provides unlimited access to our knowledgebase, tools, and much more.