• The blue points in this graph represent the system response time (in microseconds) of machines running a tuned RHEL 9 kernel.
• The green points in the graph represent the system response time of machines running a tuned real-time kernel.

Prerequisites

• The latest version of RHEL 9 is installed on an AMD64 or Intel64 system. The real-time kernel runs on AMD64 and Intel 64 (also known as x86_64) server platforms that are certified to run Red Hat Enterprise Linux.
• Your machine is registered and RHEL is attached to a RHEL for Real Time subscription.

• Ensure that the base platform is properly tuned and the system BIOS parameters are adjusted.

  Note

  Failure to perform the prerequisite tasks before installing the real-time kernel might prevent a consistent performance from a RHEL for Real Time kernel deployment.

Verification steps

• Check the installation location and verify that the components have been installed successfully.

  # rpm -ql realtime-setup
  /etc/security/limits.d/realtime.conf
  /etc/sysconfig/realtime-setup
  /etc/udev/rules.d/99-rhel-rt.rules
  /usr/bin/realtime-setup
  /usr/bin/rt-setup-kdump
  /usr/bin/slub_cpu_partial_off
  /usr/lib/.build-id
  /usr/lib/.build-id/a4
  /usr/lib/.build-id/a4/da77908aa4c6f048939f3267f1c552c456d117
  /usr/lib/systemd/system/rt-entsk.service
  /usr/lib/systemd/system/rt-setup.service
  /usr/sbin/kernel-is-rt
  /usr/sbin/rt-entsk

• kernel-rt package, which is the RHEL for Real Time kernel package.
• RHEL for Real Time kernel test packages, which contains test programs for the real-time kernel.
• RHEL for Real Time debugging packages, which are for debugging and code tracing.

• To achieve optimal low-latency determinism, you perform RHEL for Real Time specific system tuning.
• You know about the module compatibility of the real-time kernel and the standard kernel.
• To enable kdump, you must configure RHEL for Real Time to provide crash dump information by enabling kexec/kdump.
• Verify that the Real Time kernel is the default kernel.

• EMC Powerpath
• NVidia graphics
• Advanced storage adapter configuration utilities from Qlogic

Procedure

• To view the default kernel:

  ~]# grubby --default-kernel
  /boot/vmlinuz-4.18.0-80.rt9.138.el8.x86_64

  The rt in the output of the command shows that the default kernel is a real time kernel.

Procedure

• To show which kernel the system is currently running.

  ~]# uname -a
  Linux rt-server.example.com 4.18.0-80.rt9.138.el8.x86_64 …

  Note

  When the system receives a minor update, for example, from 8.3 to 8.4, the default kernel might automatically change from the Real Time kernel back to the standard kernel.

Verification

• To verify kernel-rt is the default kernel, enter the following command:

  # grubby --default-kernel
  /boot/vmlinuz-5.14.0-284.11.1.rt14.296.el9_2.x86_64.x86_64+rt

Prerequisites

• An active RHEL subscription.
• A repository containing the kexec-tools package for your system CPU architecture.
• Fulfilled requirements for kdump configurations and targets. For details, see Supported kdump configurations and targets.

Procedure

• Execute the following command to generate a memory usage report:

  # makedumpfile --mem-usage /proc/kcore
  
  
  TYPE        PAGES    EXCLUDABLE    DESCRIPTION
  -------------------------------------------------------------
  ZERO          501635      yes        Pages filled with zero
  CACHE         51657       yes        Cache pages
  CACHE_PRIVATE 5442        yes        Cache pages + private
  USER          16301       yes        User process pages
  FREE          77738211    yes        Free pages
  KERN_DATA     1333192     no         Dumpable kernel data

Prerequisites

• You have root permissions on the system.
• You have fulfilled kdump requirements for configurations and targets. For details, see Supported kdump configurations and targets.
• You have installed the zipl utility if it is the IBM Z system.

Prerequisites

• You have root permissions on the system.
• Fulfilled requirements for kdump configurations and targets. For details, see Supported kdump configurations and targets.

Procedure

• To store the crash dump file in /var/crash/ directory of the local file system, edit the /etc/kdump.conf file and specify the path:

  path /var/crash

  The option path /var/crash represents the path to the file system in which kdump saves the crash dump file.

  Note

  • When you specify a dump target in the /etc/kdump.conf file, then the path is relative to the specified dump target.
  • When you do not specify a dump target in the /etc/kdump.conf file, then the path represents the absolute path from the root directory.

  Depending on what is mounted in the current system, the dump target and the adjusted dump path are taken automatically.

  Example 4.1. The kdump target configuration

  # grep -v ^# /etc/kdump.conf | grep -v ^$
  ext4 /dev/mapper/vg00-varcrashvol
  path /var/crash
  core_collector makedumpfile -c --message-level 1 -d 31

  Here, the dump target is specified (ext4 /dev/mapper/vg00-varcrashvol), and thus mounted at /var/crash. The path option is also set to /var/crash, so the kdump saves the vmcore file in the /var/crash/var/crash directory.

• To change the local directory in which the crash dump is to be saved, as root, edit the /etc/kdump.conf configuration file:

  1. Remove the hash sign (#) from the beginning of the #path /var/crash line.

  2. Replace the value with the intended directory path. For example:

     path /usr/local/cores

     Important

     In Red Hat Enterprise Linux 9, the directory defined as the kdump target using the path directive must exist when the kdump systemd service starts to avoid failures. This behavior is different from versions earlier than RHEL, where the directory is created automatically if it did not exist when the service starts.

• To write the file to a different partition, edit the /etc/kdump.conf configuration file:

  1. Remove the hash sign (#) from the beginning of the #ext4 line, depending on your choice.

     • device name (the #ext4 /dev/vg/lv_kdump line)
     • file system label (the #ext4 LABEL=/boot line)
     • UUID (the #ext4 UUID=03138356-5e61-4ab3-b58e-27507ac41937 line)

  2. Change the file system type and the device name, label or UUID, to the required values. The correct syntax for specifying UUID values is both UUID="correct-uuid" and UUID=correct-uuid. For example:

     ext4 UUID=03138356-5e61-4ab3-b58e-27507ac41937

     Important

     It is recommended to specify storage devices using a LABEL= or UUID=. Disk device names such as /dev/sda3 are not guaranteed to be consistent across reboot.

     When you use Direct Access Storage Device (DASD) on IBM Z hardware, ensure the dump devices are correctly specified in /etc/dasd.conf before you proceed with kdump.

• To write the crash dump directly to a device, edit the /etc/kdump.conf configuration file:

  1. Remove the hash sign (#) from the beginning of the #raw /dev/vg/lv_kdump line.

  2. Replace the value with the intended device name. For example:

     raw /dev/sdb1

• To store the crash dump to a remote machine using the NFS protocol:

  1. Remove the hash sign (#) from the beginning of the #nfs my.server.com:/export/tmp line.

  2. Replace the value with a valid hostname and directory path. For example:

     nfs penguin.example.com:/export/cores

  3. Restart the kdump service for the changes to take effect:

     sudo systemctl restart kdump.service

     Note

     When using the NFS directive to specify the NFS target, kdump.service automatically attempts to mount the NFS target to check the disk space. There is no need to mount the NFS target beforehand. To prevent kdump.service from mounting the target, use the dracut_args --mount directive in kdump.conf so that kdump.service calls the dracut utility with the --mount argument to specify the NFS target.

• To store the crash dump to a remote machine using the SSH protocol:

  1. Remove the hash sign (#) from the beginning of the #ssh user@my.server.com line.
  2. Replace the value with a valid username and hostname.

  3. Include your SSH key in the configuration.

     1. Remove the hash sign from the beginning of the #sshkey /root/.ssh/kdump_id_rsa line.

     2. Change the value to the location of a key valid on the server you are trying to dump to. For example:

        ssh john@penguin.example.com
        sshkey /root/.ssh/mykey

• Compressing the size of a crash dump file and copying only necessary pages using various dump levels.
• Excluding unnecessary crash dump pages.
• Filtering the page types to be included in the crash dump.

Options

• -c, -l or -p: specify compress dump file format by each page using either, zlib for -c option, lzo for -l option or snappy for -p option.
• -d (dump_level): excludes pages so that they are not copied to the dump file.
• --message-level : specify the message types. You can restrict outputs printed by specifying message_level with this option. For example, specifying 7 as message_level prints common messages and error messages. The maximum value of message_level is 31

Prerequisites

• You have root permissions on the system.
• Fulfilled requirements for kdump configurations and targets. For details, see Supported kdump configurations and targets.

• To compress a crash dump file using -c:

• To compress a crash dump file using -p:

Prerequisites

• Root permissions.
• Fulfilled requirements for kdump configurations and targets. For details, see Supported kdump configurations and targets.

Prerequisites

• You have root permissions on the system.
• You have saved all important data. The commands to test kdump cause the kernel to crash with loss of data.
• You have scheduled significant machine maintenance time depending on the system architecture.

Prerequisites

• You have root permissions on the system.
• You have completed kdump requirements for configurations and targets. See Supported kdump configurations and targets.
• All configurations for installing kdump are set up as required.

Procedure

• Enable the kdump service for multi-user.target:

  # systemctl enable kdump.service

• Start the service in the current session:

  # systemctl start kdump.service

• Stop the kdump service:

  # systemctl stop kdump.service

• Disable the kdump service:

  # systemctl disable kdump.service

• rd.driver.blacklist=<modules>
• modprobe.blacklist=<modules>

Prerequisites

• You have root permissions on the system.

Prerequisites

• You have administrator privileges.

Verification

• Check that the kdump service is running.

  # systemctl status kdump.service
  
  ○ kdump.service - Crash recovery kernel arming
       Loaded: loaded (/usr/lib/systemd/system/kdump.service; enabled; vendor preset: disabled)
       Active: active (live)

Prerequisites

• You have administrator privileges.

Verification

• Check that the kdump service is running.

  # systemctl status kdump.service
  
  ○ kdump.service - Crash recovery kernel arming
       Loaded: loaded (/usr/lib/systemd/system/kdump.service; enabled; vendor preset: disabled)
       Active: active (live)

Prerequisites

• Fulfilled requirements for kdump configurations and targets. For details, see Supported kdump configurations and targets.
• All configurations for installing kdump are set up according to your needs. For details, see Installing kdump.

Prerequisite:

• The latest version of RHEL for Real Time kernel is installed.

• You have a Red Hat Bugzilla account.