Multipathd service hangs during startup and gets killed by systemd when there are non responding paths
Issue
-
Starting or restarting multipathd fails with systemd killing the process after approximately 3 minutes.
-
The commands
systemctl start multipathd
andsystemctl restart multipathd
hang. After approximately 3 minutes the following output reports their failure:Job for multipathd.service failed because a timeout was exceeded. See "systemctl status multipathd.service" and "journalctl -xe" for details.
-
Messages similar to the following appear in the system logs:
Mar 27 10:15:42 hostname systemd[1]: Starting Device-Mapper Multipath Device Controller... Mar 27 10:15:42 hostname multipathd[308729]: --------start up-------- Mar 27 10:15:42 hostname multipathd[308729]: read /etc/multipath.conf Mar 27 10:15:42 hostname multipathd[308729]: path checkers start up Mar 27 10:17:12 hostname systemd[1]: multipathd.service: Start operation timed out. Terminating. Mar 27 10:18:42 hostname systemd[1]: multipathd.service: State 'stop-sigterm' timed out. Killing. Mar 27 10:18:42 hostname systemd[1]: multipathd.service: Killing process 308729 (multipathd) with signal SIGKILL. Mar 27 10:18:55 hostname systemd[1]: multipathd.service: Main process exited, code=killed, status=9/KILL Mar 27 10:18:55 hostname systemd[1]: multipathd.service: Failed with result 'timeout'. Mar 27 10:18:55 hostname systemd[1]: Failed to start Device-Mapper Multipath Device Controller.
-
Multipath commands are also hanging on the system.
-
Depending on the state of multipath maps, IO operations to multipath maps may not be hanging: If the hanging paths have already been failed by earlier errors or timeouts, then IO to the multipath maps will not be hanging.
Environment
-
Observed in Red Hat Enterprise Linux 8, but it is expected to appear in other versions as well.
-
SCSI devices that are not responding (or take too long to respond) to SCSI requests are presented to the system.
Subscriber exclusive content
A Red Hat subscription provides unlimited access to our knowledgebase, tools, and much more.