46.2. Configuring the kdump Service

There are three common means of configuring the kdump service: you can enable and configure it at the first boot, use the Kernel Dump Configuration utility for the graphical user interface, or do so manually on the command line.

Important

A limitation in the current implementation of the Intel IOMMU driver can occasionally prevent the kdump service from capturing the core dump image. To use kdump on Intel architectures reliably, it is advised that the IOMMU support is disabled.

Warning

It is known that the kdump service does not work reliably on certain combinations of HP Smart Array devices and system boards from the same vendor. Consequent to this, users are strongly advised to test the configuration before using it in production environment, and if necessary, configure kdump to store the kernel crash dump to a remote machine over a network. For more information on how to test the kdump configuration, refer to Section 46.2.4, “Testing the Configuration”.

46.2.1. Configuring kdump at First Boot

When the system boots for the first time, the firstboot application is launched to guide the user through the initial configuration of the freshly installed system. To configure kdump, navigate to the Kdump page and follow the instructions below.

Important

Unless the system has enough memory, the Kdump page will not be available. For information on minimum memory requirements, refer to the Red Hat Enterprise Linux comparison chart. When the kdump crash recovery is enabled, the minimum memory requirements increase by the amount of memory reserved for it. This value is determined by the user and on x86, AMD64, and Intel 64 architectures, it defaults to 128 MB plus 64 MB for each TB of physical memory (that is, a total of 192 MB for a system with 1 TB of physical memory).
The kdump configuration screen

Figure 46.1. The kdump configuration screen

46.2.1.1. Enabling the Service

To start the kdump daemon at boot time, select the Enable kdump? checkbox. This will enable the service for runlevels 2, 3, 4, and 5, and start it for the current session. Similarly, clearing the checkbox will disable it for all runlevels and stop the service immediately.

46.2.1.2. Configuring the Memory Usage

To configure the amount of memory that is reserved for the kdump kernel, click the up and down arrow buttons next to the Kdump Memory field to increase or decrease the value. Notice that the Usable System Memory field changes accordingly showing you the remaining memory that will be available to the system.

46.2.2. Using the Kernel Dump Configuration Utility

To start the Kernel Dump Configuration utility, select Applications → System Tools → Kdump from the panel, or type system-config-kdump at a shell prompt. Unless you are already authenticated, you will be prompted to enter the root password.
The Kernel Dump Configuration utility

Figure 46.2. The Kernel Dump Configuration utility

The utility allows you to configure kdump as well as to enable or disable starting the service at boot time. When you are done, click OK to save the changes. The system reboot will be requested.

Important

Unless the system has enough memory, the Kernel Dump Configuration utility will not start and you will be presented with an error message. For information on minimum memory requirements, refer to the Red Hat Enterprise Linux comparison chart. When the kdump crash recovery is enabled, the minimum memory requirements increase by the amount of memory reserved for it. This value is determined by the user and on x86, AMD64, and Intel 64 architectures, it defaults to 128 MB plus 64 MB for each TB of physical memory (that is, a total of 192 MB for a system with 1 TB of physical memory).

46.2.2.1. Enabling the Service

To start the kdump daemon at boot time, select the Enable kdump checkbox. This will enable the service for runlevels 2, 3, 4, and 5, and start it for the current session. Similarly, clearing the checkbox will disable it for all runlevels and stop the service immediately.

46.2.2.2. Configuring the Memory Usage

To configure the amount of memory that is reserved for the kdump kernel, click the up and down arrow buttons next to the New kdump Memory field to increase or decrease the value. Notice that the Usable Memory field changes accordingly showing you the remaining memory that will be available to the system.

46.2.2.3. Configuring the Target Type

When a kernel crash is captured, the core dump can be either stored as a file in a local file system, written directly to a device, or sent over a network using the NFS (Network File System) or SSH (Secure Shell) protocol. To change this, click the Edit Location button, and select a location type as described below.
The Edit Location dialog

Figure 46.3. The Edit Location dialog

To save the dump to the local file system, select file from the pulldown list. Optionally, if you wish to write the file to a different partition, select ext3 or ext2 from the pulldown list according to the file system you are using, and enter a valid device name to the Enter location field. Note that after clicking OK, you can then customize the destination directory by changing the value in the Path field at the bottom.
To write the dump directly to a device, select raw from the pulldown list, and enter a valid device name (for example, /dev/sdb1). When you are done, click OK to confirm your choice.
To store the dump to a remote machine using the NFS protocol, select nfs from the pulldown list, and enter a valid target in the hostname:directory form (for example, penguin.example.com:/export). Clicking the OK button will confirm your changes. Finally, edit the value of the Path field to customize the destination directory (for instance, cores).
To store the dump to a remote machine using the SSH protocol, select ssh from the pulldown list, and enter a valid username and hostname in the username@hostname form (for example, john@penguin.example.com). Clicking the OK button will confirm your changes. Finally, edit the value of the Path field to customize the destination directory (for instance, /export/cores).
Refer to Chapter 20, OpenSSH for information on how to configure an SSH server, and how to set up a key-based authentication.

46.2.2.4. Configuring the Core Collector

To reduce the size of the vmcore dump file, kdump allows you to specify an external application (that is, a core collector) to compress the data, and optionally leave out all irrelevant information. Currently, the only fully supported core collector is makedumpfile.
To enable the dump file compression, make sure the -c parameter is listed after the makedumpfile command in the Core Collector field (for example, makedumpfile -c).
To remove certain pages from the dump, add the -d value parameter after the makedumpfile command in the Core Collector field. The value is a sum of values of pages you want to omit as described in Table 46.1, “Supported filtering levels”. For example, to remove both zero and free pages, use makedumpfile -d 17.
Refer to the manual page for makedumpfile for a complete list of available options.

46.2.2.5. Changing the Default Action

To choose what action to perform when kdump fails to create a core dump, select the appropriate option from the Default Action pulldown list. Available options are mount rootfs and run /sbin/init (the default action), reboot (to reboot the system), shell (to present a user with an interactive shell prompt), and halt (to halt the system).

46.2.3. Configuring kdump on the Command Line

46.2.3.1. Configuring the Memory Usage

To configure the amount of memory that is reserved for the kdump kernel on x86, AMD64, and Intel 64 architectures, open the /boot/grub/grub.conf file as root and add the crashkernel=<size>M@16M parameter to the list of kernel options as shown in Example 46.1, “Sample /boot/grub/grub.conf file”.

Important

Unless the system has enough memory, the kdump crash recovery service will not be operational. For information on minimum memory requirements, refer to the Red Hat Enterprise Linux comparison chart. When kdump is enabled, the minimum memory requirements increase by the amount of memory reserved for it. This value is determined by the user and on x86, AMD64, and Intel 64 architectures, it defaults to 128 MB plus 64 MB for each TB of physical memory (that is, a total of 192 MB for a system with 1 TB of physical memory).

Example 46.1. Sample /boot/grub/grub.conf file

# grub.conf generated by anaconda
#
# Note that you do not have to rerun grub after making changes to this file
# NOTICE:  You have a /boot partition.  This means that
#          all kernel and initrd paths are relative to /boot/, eg.
#          root (hd0,0)
#          kernel /vmlinuz-version ro root=/dev/sda3
#          initrd /initrd-version.img
#boot=/dev/sda
default=0
timeout=5
splashimage=(hd0,0)/grub/splash.xpm.gz
hiddenmenu
title Red Hat Enterprise Linux Server (2.6.18-274.3.1.el5)
        root (hd0,0)
        kernel /vmlinuz-2.6.18-274.3.1.el5 ro root=/dev/sda3 crashkernel=128M@16M
        initrd /initrd-2.6.18-274.3.1.el5.img

46.2.3.2. Configuring the Target Type

When a kernel crash is captured, the core dump can be either stored as a file in a local file system, written directly to a device, or sent over a network using the NFS (Network File System) or SSH (Secure Shell) protocol. Note that only one of these options can be set at the moment. The default option is to store the vmcore file in the /var/crash/ directory of the local file system. To change this, open the /etc/kdump.conf configuration file as root and edit the options as described below.
To change the local directory in which the core dump is to be saved, remove the hash sign (“#”) from the beginning of the #path /var/crash line, and replace the value with a desired directory path. Optionally, if you wish to write the file to a different partition, follow the same procedure with the #ext3 /dev/sda3 line as well, and change both the file system type and the device (a device name, a file system label, and UUID are all supported) accordingly. For example: 
ext3 /dev/sda4
path /usr/local/cores
To write the dump directly to a device, remove the hash sign (“#”) from the beginning of the #raw /dev/sda5 line, and replace the value with a desired device name. For example: 
raw /dev/sdb1
To store the dump to a remote machine using the NFS protocol, remove the hash sign (“#”) from the beginning of the #net my.server.com:/export/tmp line, and replace the value with a valid hostname and directory path. For example: 
net penguin.example.com:/export/cores
To store the dump to a remote machine using the SSH protocol, remove the hash sign (“#”) from the beginning of the #net user@my.server.com line, and replace the value with a valid username and hostname. For example: 
net john@penguin.example.com
Refer to Chapter 20, OpenSSH for information on how to configure an SSH server, and how to set up a key-based authentication.

46.2.3.3. Configuring the Core Collector

To reduce the size of the vmcore dump file, kdump allows you to specify an external application (that is, a core collector) to compress the data, and optionally leave out all irrelevant information. Currently, the only fully supported core collector is makedumpfile.
To enable the core collector, open the /etc/kdump.conf configuration file as root, remove the hash sign (“#”) from the beginning of the #core_collector makedumpfile -c --message-level 1 line, and edit the command line options as described below.
To enable the dump file compression, add the -c parameter. For example: 
core_collector makedumpfile -c
To remove certain pages from the dump, add the -d value parameter, where value is a sum of values of pages you want to omit as described in Table 46.1, “Supported filtering levels”. For example, to remove both zero and free pages, use the following: 
core_collector makedumpfile -d 17 -c
Refer to the manual page for makedumpfile for a complete list of available options.

Table 46.1. Supported filtering levels

Option Description
1 Zero pages
2 Cache pages
4 Cache private
8 User pages
16 Free pages

46.2.3.4. Changing the Default Action

By default, when kdump fails to create a core dump, the root file system is mounted and /sbin/init is run. To change this behavior, open the /etc/kdump.conf configuration file as root, remove the hash sign (“#”) from the beginning of the #default shell line, and replace the value with a desired action as described in Table 46.2, “Supported actions”. For example: 
default halt

Table 46.2. Supported actions

Option Action
reboot Reboot the system, losing the core in the process.
halt After failing to capture a core, halt the system.
shell Run the msh session from within the initramfs, allowing a user to record the core manually.

46.2.3.5. Enabling the Service

To start the kdump daemon at boot time, type the following at a shell prompt as root: 
~]# chkconfig kdump on
This will enable the service for runlevels 2, 3, 4, and 5. Similarly, typing chkconfig kdump off will disable it for all runlevels. To start the service in the current session, use the following command as root: 
~]# service kdump start
No kdump initial ramdisk found.                            [WARNING]
Rebuilding /boot/initrd-2.6.18-194.8.1.el5kdump.img
Starting kdump:                                            [  OK  ]
For more information on runlevels and configuring services in general, refer to Chapter 18, Controlling Access to Services.

46.2.4. Testing the Configuration

Warning

The commands below will cause the kernel to crash. Use caution when following these steps, and by no means use them on a production machine.
To test the configuration, reboot the system with kdump enabled, and as root, make sure that the service is running: 
~]# service kdump status
Kdump is operational
Then type the following commands at a shell prompt as root: 
~]# echo 1 > /proc/sys/kernel/sysrq
~]# echo c > /proc/sysrq-trigger
This will force the Linux kernel to crash, and the YYYY-MM-DD-HH:MM/vmcore file will be copied to the location you have selected in the configuration (that is, to /var/crash/ by default).