system is failing to boot in systemd in Red Hat Enterprise Linux 7
Environment
- Red Hat Enterprise Linux 7
- Systemd
- grub
Issue
- RHEL7 system is failing to boot in systemd with following point
[[32m OK [0m] Started Tell Plymouth To Write Out Runtime Data.
[[32m OK [0m] Started Update UTMP about System Boot/Shutdown.
[[32m OK [0m] Reached target System Initialization.
[[32m OK [0m] Reached target Timers.
[[32m OK [0m] Listening on D-Bus System Message Bus Socket.
[[32m OK [0m] Reached target Sockets.
[[32m OK [0m] Reached target Basic System.
[[32m OK [0m] Reached target Multi-User System.
Starting Dump dmesg to /var/log/dmesg...
[[32m OK [0m] Started Dump dmesg to /var/log/dmesg.
- How to debug Red Hat Enterprise Linux 7 boot issue with systemd
Resolution
Run the following commands in the back ground shell with Ctrl-Alt-F9, which can enabled by systemctl enable debug-shell.service
# rm /etc/systemd/system/default.target
# systemctl set-default multi-user.target
# systemctl get-default <==== This should return 'multi-user.target'
This recreates /etc/systemd/system/default.target as a symbolic link to multi-user.target
.
Root Cause
/etc/systemd/system/default.target was a normal file (copied from multi-user.target). It should be a symbolic link to /lib/systemd/system/multi-user.target.
Otherwise, systemd cannot resolve properly the dependency for default.target, and many services are not activated.
Diagnostic Steps
- Reproduce the boot problem with a boot parameter "systemd.log_level=debug systemd.log_target=kmsg log_buf_len=1M" added to the kernel line
- As grub is in use not grub2 from RHEL7, edit /etc/grub.conf to add them
- Enable debug-shell.service
# systemctl enable debug-shell.service
Reproduce the problem, and run the following command in the background shell with Ctrl-Alt-F9:
# systemctl list-jobs > /tmp/systemctl-listjobs.log
# ps -ef > /tmp/psef.log
# lsof > /tmp/lsof.log
# systemctl get-default > /tmp/systemctl-getdefault.log
# systemctl list-units > /tmp/systemctl-listunits.log
# systemctl list-units --all > /tmp/systemctl-listunitsall.log
# systemctl get-default > /tmp/systemctl-getdefault.log
# journalctl > /tmp/journal.log
# tar zcvf /tmp/systemd-dbg.tar.gz /tmp/systemctl-listjobs.log /tmp/systemctl-getdefault.log /tmp/systemctl-listunits.log /tmp/systemctl-listunitsall.log /tmp/journal.log /tmp/psef.log /tmp/lsof.log /etc/systemd/system /lib/systemd/system
systemctl get-default
returneddefault.target
(notmulti-user.target
)
This solution is part of Red Hat’s fast-track publication program, providing a huge library of solutions that Red Hat engineers have created while supporting our customers. To give you the knowledge you need the instant it becomes available, these articles may be presented in a raw and unedited form.
Comments