Why kdump service fails to build initramfs if "path" is not set in /etc/kdump.conf file.
Environment
- Red Hat Enterprise Linux 7.1
- kexec-tools
Issue
- kdump service fails to build initramfs if "path" is not set in /etc/kdump.conf file.
-
Kdump fails with below call traces
# systemctl status kdump.service kdump.service - Crash recovery kernel arming Loaded: loaded (/usr/lib/systemd/system/kdump.service; enabled) Active: failed (Result: exit-code) since Tue 2015-07-28 04:00:54 PDT; 6s ago Process: 30341 ExecStop=/usr/bin/kdumpctl stop (code=exited, status=0/SUCCESS) Process: 30350 ExecStart=/usr/bin/kdumpctl start (code=exited, status=1/FAILURE) Main PID: 30350 (code=exited, status=1/FAILURE) Jul 28 04:00:53 hostname kdumpctl[30350]: Detected change(s) in the following file(s): Jul 28 04:00:53 hostname kdumpctl[30350]: /etc/kdump.conf Jul 28 04:00:53 hostname kdumpctl[30350]: Rebuilding /boot/initramfs-3.10.0-229.el7.x86_64kdump.img Jul 28 04:00:54 hostname kdumpctl[30350]: Dump path /var/crash//var/crash/ does not exist. Jul 28 04:00:54 hostname kdumpctl[30350]: mkdumprd: failed to make kdump initrd Jul 28 04:00:54 hostname kdumpctl[30350]: Starting kdump: [FAILED] Jul 28 04:00:54 hostname systemd[1]: kdump.service: main process exited, code=exited, status=1/FAILURE Jul 28 04:00:54 hostname systemd[1]: Failed to start Crash recovery kernel arming. Jul 28 04:00:54 hostname systemd[1]: Unit kdump.service entered failed state.
Resolution
-
This is an expected behavior.
- If kdump service is configured to dump the vmcore on NFS share then it is necessary to set the "path" variable in /etc.kdump.conf file else "mkdumprd" command will fail to generate a initramfs file for kdump kernel.
- This behavior is introduced in /usr/sbin/mkdumprd bash script provided by "kexec-tools" package shipped with RHEL-7.1.
- In older version of kexec-tools package if /var/crash directory is not present on NFS share then kdump service will automatically create it without any error message.
-
This is documented in /usr/share/doc/kexec-tools-2.0.7/kexec-kdump-howto.txt
path /var/crash/ (NFS mounted on /var) Say foo.com:/export/tmp is mounted on /var. In this case dump target is nfs server and path will be adjusted to "/crash" and dump will be saved to foo.com:/export/tmp/crash/ directory.Resolution:
- Set "path" to "/var/crash" in /etc/kdump.conf file if NFS share is mounted on "/var".
OR - Set "path" to "/" in /etc/kdump.conf file if NFS share is mounted on "/var/crash".
- Set "path" to "/var/crash" in /etc/kdump.conf file if NFS share is mounted on "/var".
Root Cause
-
In older version of kexec-tools package if /var/crash directory is not present on NFS
share then kdump service will automatically create it without any error message.
RHEL-7.0 [ /usr/sbin/mkdumprd ]ext[234]|xfs|btrfs|minix|nfs) <<<--- if ! findmnt $config_val >/dev/null; then perror_exit "Dump target $config_val is probably not mounted." fi if [ "$config_opt" = "nfs" ]; then add_dracut_module "nfs" fi add_mount "$config_val" mkdir_save_path_fs $config_val <<<--- check_size fs $config_val ;; -
In newer version of kexec-tools package We need to set the "path" in /etc/kdump.conf file
to dump the vmcore on NFS share else kdump service will fail with an error message.
RHEL-7.1 [ /usr/sbin/mkdumprd ]ext[234]|xfs|btrfs|minix|nfs) <<<--- if ! findmnt $config_val >/dev/null; then perror_exit "Dump target $config_val is probably not mounted." fi if [ "$config_opt" = "nfs" ]; then add_dracut_module "nfs" fi add_mount "$config_val" check_save_path_fs $(make_absolute_save_path $config_val) <<<--- check_size fs $config_val ;; check_save_path_fs() { local _path=$1 if [ ! -d $_path ]; then perror_exit "Dump path $_path does not exist." <<<--- fi } -
This is documented in /usr/share/doc/kexec-tools-2.0.7/kexec-kdump-howto.txt
path /var/crash/ (NFS mounted on /var) ---------------------------------------------- Say foo.com:/export/tmp is mounted on /var. In this case dump target is nfs server and path will be adjusted to "/crash" and dump will be saved to foo.com:/export/tmp/crash/ directory.
Diagnostic Steps
- Check kdump Configuration.
This solution is part of Red Hat’s fast-track publication program, providing a huge library of solutions that Red Hat engineers have created while supporting our customers. To give you the knowledge you need the instant it becomes available, these articles may be presented in a raw and unedited form.
Welcome! Check out the Getting Started with Red Hat page for quick tours and guides for common tasks.
