Kdump Helper

Updated -

Kdump is a reliable kernel crash-dumping mechanism that captures crash dumps for troubleshooting issues like kernel crashes, hangs, and reboots. Setting up Kdump usually requires a series of steps and configurations. The Kdump Helper app is designed to simplify the process and reduce the effort required to set up Kdump on your machines.

Input a minimum amount of information and this app will generate an all-in-one script for you to set up Kdump to dump data in memory into a dump file called a vmcore. It has two modes: guided mode and manual mode . If you just want to input minimum information and use default or recommended parameters whenever possible, simply use the guided mode (the default mode), which then presents you a few steps with clear instructions to help you choose or input a proper parameter. If you have a very good understanding of Kdump and want to control as many options as possible, the manual mode will be your choice. It allows you to adjust all parameters.

Once you click the "Generate" button, the app will generate a script for you to set up Kdump based on the information you provided. Just run it on your server, and after it finishes, your Kdump service will be ready to capture crash dumps.

Announcement

Blog Post

Articles

Solutions

Was this helpful?

Yes No
We appreciate your feedback. Leave a comment if you would like to provide more detail.
It looks like we have some work to do. Leave a comment to let us know how we could improve.

Get notified when this content is updated

Follow

Share

Red Hat Customer Portal Twitter Facebook

6 Comments

Subscriber exclusive content

An active Red Hat subscription is required to participate.

Log In
Richard Beldin's picture Expert 973 points
18 July 2014 7:54 PM

BZ 990302 is assigned state, talking about how crashkernel=auto does not work for large physical memory machines. Does this tool take that into account? Thanks, Rick Beldin (rick.beldin@hp.com)

GL Red Hat Community Member 25 points
21 July 2014 2:37 AM

Yes, on RHEL6.x we set the crashkernel by the follow rule.

RHEL6 crashkernel compute

   https://access.redhat.com/site/solutions/59432

    ram size    crashkernel parameter   ram / crashkernel factor
    >0GB              128MB                         15
    >2GB              256MB                         23
    >6GB              512MB                         15
    >8GB              768MB                         31

We use crashkernel=auto for RHEL7 only, because starting with RHEL7 kernels crashkernel=auto should be used. The kernel will automatically reserve an appropriate amount of memory for the kdump kernel.

https://access.redhat.com/solutions/916043

BR
Gengsheng(gliu@redhat.com)

vv Active Contributor 237 points
25 September 2014 1:24 AM

The new UI looks awesome, we are using it as well.
Gengsheng

rh Red Hat Community Member 25 points
13 October 2014 9:46 PM

In the future will this also include the ability to panic on different NMI?

gs
3 February 2015 1:50 AM

what's the most recent version of kexec-tools for RHEL6.5

SS Red Hat Community Member 90 points
8 April 2015 7:56 AM

You should not modify /etc/sysctl.conf directly to add or change configurations. Instead you can make custom sysctl configuration files in /etc/sysctl.d/ since RHEL 5. This is the preferable way to configure sysctl.

RK
17 June 2015 2:50 PM

Team,
we have a server with 16 CPU, 125 GB memory, running 5.11, the server has about 500 file systems on it, in the kdump helper script I do not see the offset and crashkernel values for memory more than 32 GB.

any recommendations ?

Thank you!
Ravi

YZ Red Hat Newbie 5 points
18 June 2015 5:01 AM

Hi Ravi,
Thank you for contacting us.
For RAM size greater than 24G:
Try crashkernel memory 768M and RAM/crashkernel offset of 32, which looks like 768M@32M.
If you get an Out-Of-Memory error message, then try with increasing the crashkernel parameter to 896M
More details, please read this solution:
https://access.redhat.com/solutions/106193

Thanks!
Best regards,
Yunfeng Zhao

UT Community Member 48 points
26 September 2015 2:33 PM

Hi ,

We have a RAM size of 384GB on RHEL6.7.What shoukld be the memory reservation for this amount of RAM?

-Aravinda

HF Newbie 7 points
19 May 2016 12:16 PM

Hi,

When using option 2, mem_total seems to not be initialized in the generated script updateGrub.sh which generates bash errors like : line 19 '[' -le 2 ']'
unary operator expected. Thanks Homi FARIS (homi.faris@hpe.com)

YZ Red Hat Newbie 5 points
20 May 2016 6:57 AM

Hi Homi, Thank you for reporting the issue. We have already fixed it. Thanks.

Best regards, Yunfeng Zhao

Felipe Aranda's picture Red Hat Active Contributor 119 points
8 March 2017 3:56 PM

The man kdump.conf specifies the "net" option is deprecated by "nfs" for nfs dumps.

nfs <nfs mount> | nfs4 <nfs mount> | net <nfs mount>
Will  mount  fs and copy /proc/vmcore to <mnt>/<path>/%HOST-%DATE/, supports DNS. Note that a fqdn should be used as the server name in the mount point. By default, NFSv3 is used, nfs4 will use NFSv4. "net" is deprecated by "nfs"

The kdump helper script put the "net" option instead "nfs". Customer ask me if it is a fully supported configuration ?

Thanks you!

Felipe Aranda G.

DZ Red Hat Community Member 41 points
9 March 2017 2:09 AM

Hi Felipe,

Thanks for reporting this issue. we will fix it.

Dong Zhao

DP Newbie 17 points
28 August 2020 7:38 PM

RHEL 7.5 system having issues running script generated.

I added a 100GB disk (system has 32GB RAM) to systemvg and created a filesystem (100GB) to mount as /var/crash.

When I execute the kdumpconfig.sh script it fails with the following

[root@hostname ~]# ./kdumpconfig.sh
Kdump Helper is starting to configure kdump service
Your total memory is 31 G
Detected Legacy-boot
backup /boot/grub2/grub.cfg to /boot/grub2/grub.cfg.kdumphelper.20-08-28-12:11:09
crashkernel=auto is set in /boot/grub2/grub.cfg
backup /etc/kdump.conf to /etc/kdump.conf.kdumphelper.20-08-28-12:11:09
==== The dump directory is mounted to a separate device. Your vmcore will be dumped to that device ====
enable kdump service...
● kdump.service                                                                                                  loaded    failed   failed    Crash recovery kernel arming
Job for kdump.service failed because the control process exited with error code. See "systemctl status kdump.service" and "journalctl -xe" for details.
Starting to Configure extra diagnostic options
backup /etc/sysctl.conf to /etc/sysctl.conf.kdumphelper.20-08-28-12:11:11
kernel.sysrq=1 set in /etc/sysctl.conf
kernel.unknown_nmi_panic=1  set in /etc/sysctl.conf

Checking the kdump service I get

[root@hostname collectl]# systemctl status kdump.service -l
● kdump.service - Crash recovery kernel arming
   Loaded: loaded (/usr/lib/systemd/system/kdump.service; enabled; vendor preset: enabled)
   Active: failed (Result: exit-code) since Fri 2020-08-28 12:09:24 EDT; 3s ago
  Process: 14111 ExecStart=/usr/bin/kdumpctl start (code=exited, status=1/FAILURE)
 Main PID: 14111 (code=exited, status=1/FAILURE)

Aug 28 12:09:22 hostname kdumpctl[14111]: /etc/fstab
Aug 28 12:09:22 hostname kdumpctl[14111]: Rebuilding /boot/initramfs-3.10.0-862.6.3.el7.x86_64kdump.img
Aug 28 12:09:24 hostname kdumpctl[14111]: df: ‘/var/crash/var/crash/’: No such file or directory
Aug 28 12:09:24 hostname kdumpctl[14111]: Dump path /var/crash/var/crash/ does not exist.
Aug 28 12:09:24 hostname kdumpctl[14111]: mkdumprd: failed to make kdump initrd
Aug 28 12:09:24 hostname kdumpctl[14111]: Starting kdump: [FAILED]
Aug 28 12:09:24 hostname systemd[1]: kdump.service: main process exited, code=exited, status=1/FAILURE
Aug 28 12:09:24 hostname systemd[1]: Failed to start Crash recovery kernel arming.
Aug 28 12:09:24 hostname systemd[1]: Unit kdump.service entered failed state.
Aug 28 12:09:24 hostname systemd[1]: kdump.service failed.

Why is it trying to mount /var/crash/var/crash? I checked the /etc/kdump.conf

[root@hostname ~]# cat /etc/kdump.conf
path /var/crash
core_collector makedumpfile -c --message-level 1 -d 1
default reboot
xfs UUID="8bab1418-1cfa-4921-9a0d-cdd4c58d5768"

Nothing looks like it would be forcing it to add the path onto itself.

I tried remounting the 100GB filesystem as /crash in case it was not happy with me using the default location for some reason.

[root@hostname ~]# ./kdumpconfig.sh
Kdump Helper is starting to configure kdump service
Your total memory is 31 G
Detected Legacy-boot
backup /boot/grub2/grub.cfg to /boot/grub2/grub.cfg.kdumphelper.20-08-28-12:38:27
crashkernel=auto is set in /boot/grub2/grub.cfg
backup /etc/kdump.conf to /etc/kdump.conf.kdumphelper.20-08-28-12:38:27
==== The dump directory is mounted to a separate device. Your vmcore will be dumped to that device ====
enable kdump service...
● kdump.service                                                                                                  loaded    failed   failed    Crash recovery kernel arming
Job for kdump.service failed because the control process exited with error code. See "systemctl status kdump.service" and "journalctl -xe" for details.
Starting to Configure extra diagnostic options
backup /etc/sysctl.conf to /etc/sysctl.conf.kdumphelper.20-08-28-12:38:30
kernel.sysrq=1 set in /etc/sysctl.conf
kernel.unknown_nmi_panic=1  set in /etc/sysctl.conf
[root@hostname ~]# systemctl status kdump.service -l
● kdump.service - Crash recovery kernel arming
   Loaded: loaded (/usr/lib/systemd/system/kdump.service; enabled; vendor preset: enabled)
   Active: failed (Result: exit-code) since Fri 2020-08-28 12:38:30 EDT; 13s ago
  Process: 41736 ExecStart=/usr/bin/kdumpctl start (code=exited, status=1/FAILURE)
 Main PID: 41736 (code=exited, status=1/FAILURE)

Aug 28 12:38:28 hostname kdumpctl[41736]: /etc/fstab
Aug 28 12:38:28 hostname kdumpctl[41736]: Rebuilding /boot/initramfs-3.10.0-862.6.3.el7.x86_64kdump.img
Aug 28 12:38:30 hostname kdumpctl[41736]: df: ‘/crash/crash’: No such file or directory
Aug 28 12:38:30 hostname kdumpctl[41736]: Dump path /crash/crash does not exist.
Aug 28 12:38:30 hostname kdumpctl[41736]: mkdumprd: failed to make kdump initrd
Aug 28 12:38:30 hostname kdumpctl[41736]: Starting kdump: [FAILED]
Aug 28 12:38:30 hostname systemd[1]: kdump.service: main process exited, code=exited, status=1/FAILURE
Aug 28 12:38:30 hostname systemd[1]: Failed to start Crash recovery kernel arming.
Aug 28 12:38:30 hostname systemd[1]: Unit kdump.service entered failed state.
Aug 28 12:38:30 hostname systemd[1]: kdump.service failed.

If I specified just / for the location things would go off the rails.

Deleting the line also didn't help.

Had to mkdir /crash/crash to get the service to start with /crash as the directory defined.

YZ Red Hat Newbie 5 points
31 August 2020 1:50 AM

Hi Demian,

Thank you very much for reaching out us.

Would you like to provide the script you generate in this Bugzilla?

Thanks.