3.2. Using kdump and kexec with the Red Hat Enterprise Linux for Real Time Kernel

kdump is a reliable kernel crash dumping mechanism because the crash dump is captured from the context of a freshly booted kernel and not from the context of the crashed kernel. kdump uses a mechanism called kexec to boot into a second kernel whenever the system crashes. This second kernel, often called the crash kernel, boots with very little memory and captures the dump image.
If kdump is enabled on your system, the standard boot kernel will reserve a small section of system RAM and load the kdump kernel into the reserved space. When a kernel panic or other fatal error occurs, kexec is used to boot into the kdump kernel without going through BIOS. The system reboots to the kdump kernel that is confined to the memory space reserved by the standard boot kernel, and this kernel writes a copy or image of the system memory to the storage mechanism defined in the configuration files. Because kexec does not go through the BIOS, the memory of the original boot is retained, and the crash dump is much more detailed. Once this is done, the kernel reboots, which resets the machine and brings the boot kernel back up.
There are three required procedures for enabling kdump under Red Hat Enterprise Linux 7. First, ensure that the required RPM packages are installed on the system. Second, create the minimum configuration and modifies the GRUB command line using the rt-setup-kdump tool. Third, use a graphical system configuration tool called system-config-kdump to create and enable a detailed kdump configuration.
  1. Installing Required kdump Packages

    The rt-setup-kdump tool is part of the rt-setup package. You also need kexec-tools and system-config-kdump:
    ~]# yum install rt-setup kexec-tools system-config-kdump
  2. Creating a Basic kdump Kernel with rt-setup-kdump

    1. Run the rt-setup-kdump tool as root:
      ~]# rt-setup-kdump --grub
      The --grub parameter adds the necessary changes to all the real-time kernel entries listed in the GRUB configuration.
    2. Restart the system to set up the reserved memory space. You can then turn on the kdump init script and start the kdump service:
      ~]# systemctl enable kdump
      ~]# systemctl start kdump
  3. Enabling kdump with system-config-kdump

    1. Select the Kernel crash dumps system tool from the ApplicationsSystem Tools menu, or use the following command at the shell prompt:
      ~]# system-config-kdump
    2. The Kernel Dump Configuration window displays. On the toolbar, click the button labeled Enable. The Red Hat Enterprise Linux for Real Time kernel supports the crashkernel=auto parameter which automatically calculates the amount of memory necessary to accommodate the kdump kernel.
      By design, on Red Hat Enterprise Linux 7 systems with less than 4GB of RAM, the crashkernel=auto does not reserve any memory for the kdump kernel. In this case, it is necessary to manually set the amount of memory required. You can do so by entering your required value in the New kdump memory field on the Basic Settings tab:
      Kdump System Tool - Basic Settings

      Note

      An alternative way of allocating memory for the kdump kernel is by manually setting the crashkernel=<value> parameter in the GRUB configuration.
    3. Click the Target Settings tab, and specify the target location for your dump file. It can be either stored as a file in a local file system, written directly to a device, or sent over a network using the NFS (Network File System) or SSH (Secure Shell) protocol.
      Kdump System Tool - Target Settings
      To save your settings, click the Apply button on the toolbar.
    4. Reboot the system to ensure that kdump is properly started. If you want to check that the kdump is working correctly, you can simulate a panic using sysrq:
      ~]# echo c > /proc/sysrq-trigger
      This will cause the kernel to panic, and the system will boot into the kdump kernel. Once your system has been brought back up, you can check the log file at the specified location.

Note

Some hardware needs to be reset during the configuration of the kdump kernel. If you have any problems getting the kdump kernel to work, edit the /etc/sysconfig/kdump file and add reset_devices=1 to the KDUMP_COMMANDLINE_APPEND variable.

Important

On IBM LS21 machines, the following warning message can occur when attempting to boot the kdump kernel:
irq 9: nobody cared (try booting with the "irqpoll" option) handlers:
[<ffffffff811660a0>] (acpi_irq+0x0/0x1b)
turning off IO-APIC fast mode.
Some systems will recover from this error and continue booting, while some will freeze after displaying the message. This is a known issue. If you see this error, add the line acpi=noirq as a boot parameter to the kdump kernel. Only add this line if this error occurs as it can cause boot problems on machines not affected by this issue.
Related Manual Pages

For more information, or for further reading, the following man page is related to the information given in this section.

  • kexec(8)
  • /etc/kdump.conf