I can't get RHEL8 to use latest kernel

Latest response

I'm running RHEL 8.5 on an EC2 instance in AWS and I can't get it to run on the latest kernel.

[ec2-user@ip-10-10-2-220 ~]$ rpm -qa | grep ^kernel
kernel-modules-4.18.0-348.20.1.el8_5.x86_64
kernel-core-4.18.0-305.el8.x86_64
kernel-4.18.0-348.20.1.el8_5.x86_64
kernel-tools-libs-4.18.0-348.20.1.el8_5.x86_64
kernel-4.18.0-305.el8.x86_64
kernel-tools-4.18.0-348.20.1.el8_5.x86_64
kernel-modules-4.18.0-305.el8.x86_64
kernel-core-4.18.0-348.20.1.el8_5.x86_64

[ec2-user@ip-10-10-2-220 ~]$ uname -r
4.18.0-305.el8.x86_64

[ec2-user@ip-10-10-2-220 ~]$ df -h
Filesystem                         Size  Used Avail Use% Mounted on
devtmpfs                            32G     0   32G   0% /dev
tmpfs                               32G     0   32G   0% /dev/shm
tmpfs                               32G   17M   32G   1% /run
tmpfs                               32G     0   32G   0% /sys/fs/cgroup
/dev/nvme0n1p2                     128G  4.4G  124G   4% /
/dev/nvme1n1p1                     495M  413M   83M  84% /boot
/dev/mapper/newdirs-home           5.0G  128M  4.9G   3% /home
/dev/mapper/newdirs-tmp            2.0G   58M  2.0G   3% /tmp
/dev/mapper/newdirs-var            2.0G  756M  1.3G  38% /var
/dev/mapper/newdirs-var_tmp        2.0G   47M  2.0G   3% /var/tmp
/dev/mapper/newdirs-var_log        2.0G  1.9G  120M  95% /var/log
/dev/mapper/newdirs-var_log_audit  2.0G  2.0G   51M  98% /var/log/audit
tmpfs                              6.3G     0  6.3G   0% /run/user/1000

[ec2-user@ip-10-10-2-220 ~]$ ls /boot
config-4.18.0-305.el8.x86_64                             initramfs-4.18.0-305.el8.x86_64.img                   System.map-4.18.0-305.el8.x86_64
config-4.18.0-348.20.1.el8_5.x86_64                      initramfs-4.18.0-305.el8.x86_64.img.06-12-004514.bak  System.map-4.18.0-348.20.1.el8_5.x86_64
efi                                                      initramfs-4.18.0-305.el8.x86_64kdump.img              vmlinuz-0-rescue-bd6cf7372f924f33ab49bd9e121a3895
grub2                                                    initramfs-4.18.0-348.20.1.el8_5.x86_64.img            vmlinuz-0-rescue-cc248e8c4ee74660b67b51ef99a7837a
initramfs-0-rescue-bd6cf7372f924f33ab49bd9e121a3895.img  loader                                                vmlinuz-4.18.0-305.el8.x86_64
initramfs-0-rescue-cc248e8c4ee74660b67b51ef99a7837a.img  symvers-4.18.0-348.20.1.el8_5.x86_64.gz               vmlinuz-4.18.0-348.20.1.el8_5.x86_64

I have tried reinstalling and rebooting with no success. I followed the solution here here with no errors, but still no new kernel after a reboot.

JW
Log in to join the conversation

Responses

cm Community Member 80 points
22 April 2022 2:16 AM

edit...

Sadashiva Murthy M's picture Guru 1397 points
22 April 2022 6:08 AM

Hello John,

It looks to be me that it has successfully installed the latest kernel, but failed to update it as the current one. Do you see the list as given below when you run the command "grubby --info=ALL":

index=0
kernel="/boot/vmlinuz-4.18.0-348.20.1.el8_5.x86_64"
args="ro crashkernel=auto resume=/dev/mapper/rhel-swap rd.lvm.lv=rhel/root rd.lvm.lv=rhel/swap rhgb quiet $tuned_params"
root="/dev/mapper/rhel-root"
initrd="/boot/initramfs-4.18.0-348.20.1.el8_5.x86_64.img $tuned_initrd"
title="Red Hat Enterprise Linux (4.18.0-348.20.1.el8_5.x86_64) 8.5 (Ootpa)"
id="b67413d511a9427ba9748f17924cbab7-4.18.0-348.20.1.el8_5.x86_64"

Also, verify what does the file '/boot/grub2/grubenv' shows. If that is not properly set to the latest saved kernel entry then this can reset with the command "grub2-set-default 0". Good to verify if the link file in /etc/grub2.cfg exists and pointing to /boot/grub2/grub.cfg file. Another point to mention here is that the space under /boot & /var as per your "df" command output is showing less, make sure to get the space increased. You may remove older (current -2) kernel packages if not required. Better to verify if the defaults are not changed in '/etc/sysconfig/kernel' file. This file should have the defaults as "UPDATEDEFAULT=yes" & "DEFAULTKERNEL=kernel", verify this. It also helps the community if you could show all these file contents for further diagnosis.

Note: The default kernel entry can also be set or changed using the "grubby" command in RHEL8/9. Please refer this KB https://access.redhat.com/solutions/4326431

JW Community Member 26 points
22 April 2022 3:09 PM

Hi Sadashiva, thanks for your assistance! Everything looks right to me, but I am very far from being an expert. Here are the contents of all the files you mentioned:

[ec2-user@ip-10-10-2-220 ~]$ sudo grubby --info=ALL
index=0
kernel="/boot/vmlinuz-4.18.0-348.20.1.el8_5.x86_64"
args="ro console=ttyS0,115200n8 console=tty0 net.ifnames=0 rd.blacklist=nouveau nvme_core.io_timeout=4294967295 crashkernel=auto rd.lvm.lv=newdirs/home rd.lvm.lv=newdirs/tmp rd.lvm.lv=newdirs/var rd.lvm.lv=newdirs/var_log rd.lvm.lv=newdirs/var_log_audit rd.lvm.lv=newdirs/var_tmp fips=1 boot=UUID=7e8029d1-da36-4524-84d0-8091deedc38b audit_backlog_limit=8192 audit=1 pti=on vsyscall=none slub_debug=P page_poison=1"
root="UUID=c9aa25ee-e65c-4818-9b2f-fa411d89f585"
initrd="/boot/initramfs-4.18.0-348.20.1.el8_5.x86_64.img"
title="Red Hat Enterprise Linux (4.18.0-348.20.1.el8_5.x86_64) 8.5 (Ootpa)"
id="cc248e8c4ee74660b67b51ef99a7837a-4.18.0-348.20.1.el8_5.x86_64"
index=1
kernel="/boot/vmlinuz-4.18.0-348.2.1.el8_5.x86_64"
args="ro console=ttyS0,115200n8 console=tty0 net.ifnames=0 rd.blacklist=nouveau nvme_core.io_timeout=4294967295 crashkernel=auto rd.lvm.lv=newdirs/home rd.lvm.lv=newdirs/tmp rd.lvm.lv=newdirs/var rd.lvm.lv=newdirs/var_log rd.lvm.lv=newdirs/var_log_audit rd.lvm.lv=newdirs/var_tmp fips=1 boot=UUID=7e8029d1-da36-4524-84d0-8091deedc38b audit_backlog_limit=8192 audit=1 pti=on vsyscall=none slub_debug=P page_poison=1"
root="UUID=c9aa25ee-e65c-4818-9b2f-fa411d89f585"
initrd="/boot/initramfs-4.18.0-348.2.1.el8_5.x86_64.img"
title="Red Hat Enterprise Linux (4.18.0-348.2.1.el8_5.x86_64) 8.5 (Ootpa)"
id="cc248e8c4ee74660b67b51ef99a7837a-4.18.0-348.2.1.el8_5.x86_64"
index=2
kernel="/boot/vmlinuz-0-rescue-cc248e8c4ee74660b67b51ef99a7837a"
args="ro console=ttyS0,115200n8 console=tty0 net.ifnames=0 rd.blacklist=nouveau nvme_core.io_timeout=4294967295 crashkernel=auto rd.lvm.lv=newdirs/home rd.lvm.lv=newdirs/tmp rd.lvm.lv=newdirs/var rd.lvm.lv=newdirs/var_log rd.lvm.lv=newdirs/var_log_audit rd.lvm.lv=newdirs/var_tmp fips=1 boot=UUID=7e8029d1-da36-4524-84d0-8091deedc38b audit_backlog_limit=8192 audit=1 pti=on vsyscall=none slub_debug=P page_poison=1"
root="UUID=c9aa25ee-e65c-4818-9b2f-fa411d89f585"
initrd="/boot/initramfs-0-rescue-cc248e8c4ee74660b67b51ef99a7837a.img"
title="Red Hat Enterprise Linux (0-rescue-cc248e8c4ee74660b67b51ef99a7837a) 8.4 (Ootpa)"
id="cc248e8c4ee74660b67b51ef99a7837a-0-rescue"
index=3
kernel="/boot/boot/vmlinuz-4.18.0-305.el8.x86_64"
args="ro console=ttyS0,115200n8 console=tty0 net.ifnames=0 rd.blacklist=nouveau nvme_core.io_timeout=4294967295 crashkernel=auto rd.lvm.lv=newdirs/home rd.lvm.lv=newdirs/tmp rd.lvm.lv=newdirs/var rd.lvm.lv=newdirs/var_log rd.lvm.lv=newdirs/var_log_audit rd.lvm.lv=newdirs/var_tmp fips=1 boot=UUID=7e8029d1-da36-4524-84d0-8091deedc38b audit_backlog_limit=8192 audit=1 pti=on vsyscall=none slub_debug=P page_poison=1 $tuned_params"
root="UUID=c9aa25ee-e65c-4818-9b2f-fa411d89f585"
initrd="/boot/boot/initramfs-4.18.0-305.el8.x86_64.img $tuned_initrd"
title="Red Hat Enterprise Linux (4.18.0-305.el8.x86_64) 8.4 (Ootpa)"
id="bd6cf7372f924f33ab49bd9e121a3895-4.18.0-305.el8.x86_64"
index=4
kernel="/boot/boot/vmlinuz-0-rescue-bd6cf7372f924f33ab49bd9e121a3895"
args="ro console=ttyS0,115200n8 console=tty0 net.ifnames=0 rd.blacklist=nouveau nvme_core.io_timeout=4294967295 crashkernel=auto rd.lvm.lv=newdirs/home rd.lvm.lv=newdirs/tmp rd.lvm.lv=newdirs/var rd.lvm.lv=newdirs/var_log rd.lvm.lv=newdirs/var_log_audit rd.lvm.lv=newdirs/var_tmp fips=1 boot=UUID=7e8029d1-da36-4524-84d0-8091deedc38b audit_backlog_limit=8192 audit=1 pti=on vsyscall=none slub_debug=P page_poison=1"
root="UUID=c9aa25ee-e65c-4818-9b2f-fa411d89f585"
initrd="/boot/boot/initramfs-0-rescue-bd6cf7372f924f33ab49bd9e121a3895.img"
title="Red Hat Enterprise Linux (0-rescue-bd6cf7372f924f33ab49bd9e121a3895) 8.4 (Ootpa)"
id="bd6cf7372f924f33ab49bd9e121a3895-0-rescue"

[ec2-user@ip-10-10-2-220 ~]$ sudo cat /boot/grub2/grubenv
# GRUB Environment Block
saved_entry=cc248e8c4ee74660b67b51ef99a7837a-4.18.0-348.20.1.el8_5.x86_64
kernelopts=root=UUID=c9aa25ee-e65c-4818-9b2f-fa411d89f585 ro console=ttyS0,115200n8 console=tty0 net.ifnames=0 rd.blacklist=nouveau nvme_core.io_timeout=4294967295 crashkernel=auto rd.lvm.lv=newdirs/home rd.lvm.lv=newdirs/tmp rd.lvm.lv=newdirs/var rd.lvm.lv=newdirs/var_log rd.lvm.lv=newdirs/var_log_audit rd.lvm.lv=newdirs/var_tmp fips=1 boot=UUID=7e8029d1-da36-4524-84d0-8091deedc38b  audit_backlog_limit=8192 audit=1 pti=on vsyscall=none slub_debug=P page_poison=1
boot_success=1
boot_indeterminate=1
#####################################################################################################################################################################################################################################################################################################################################################################################################################################

[ec2-user@ip-10-10-2-220 ~]$ sudo cat /etc/grub2.cfg
#
# DO NOT EDIT THIS FILE
#
# It is automatically generated by grub2-mkconfig using templates
# from /etc/grub.d and settings from /etc/default/grub
#

### BEGIN /etc/grub.d/00_header ###
set pager=1

if [ -f ${config_directory}/grubenv ]; then
  load_env -f ${config_directory}/grubenv
elif [ -s $prefix/grubenv ]; then
  load_env
fi
if [ "${next_entry}" ] ; then
   set default="${next_entry}"
   set next_entry=
   save_env next_entry
   set boot_once=true
else
   set default="${saved_entry}"
fi

if [ x"${feature_menuentry_id}" = xy ]; then
  menuentry_id_option="--id"
else
  menuentry_id_option=""
fi

export menuentry_id_option

if [ "${prev_saved_entry}" ]; then
  set saved_entry="${prev_saved_entry}"
  save_env saved_entry
  set prev_saved_entry=
  save_env prev_saved_entry
  set boot_once=true
fi

function savedefault {
  if [ -z "${boot_once}" ]; then
    saved_entry="${chosen}"
    save_env saved_entry
  fi
}

function load_video {
  if [ x$feature_all_video_module = xy ]; then
    insmod all_video
  else
    insmod efi_gop
    insmod efi_uga
    insmod ieee1275_fb
    insmod vbe
    insmod vga
    insmod video_bochs
    insmod video_cirrus
  fi
}

terminal_output console
if [ x$feature_timeout_style = xy ] ; then
  set timeout_style=menu
  set timeout=1
# Fallback normal timeout code in case the timeout_style feature is
# unavailable.
else
  set timeout=1
fi
### END /etc/grub.d/00_header ###

### BEGIN /etc/grub.d/01_users ###
if [ -f ${prefix}/user.cfg ]; then
  source ${prefix}/user.cfg
  if [ -n "${GRUB2_PASSWORD}" ]; then
    set superusers="root"
    export superusers
    password_pbkdf2 root ${GRUB2_PASSWORD}
  fi
fi
### END /etc/grub.d/01_users ###

### BEGIN /etc/grub.d/08_fallback_counting ###
insmod increment
# Check if boot_counter exists and boot_success=0 to activate this behaviour.
if [ -n "${boot_counter}" -a "${boot_success}" = "0" ]; then
  # if countdown has ended, choose to boot rollback deployment,
  # i.e. default=1 on OSTree-based systems.
  if  [ "${boot_counter}" = "0" -o "${boot_counter}" = "-1" ]; then
    set default=1
    set boot_counter=-1
  # otherwise decrement boot_counter
  else
    decrement boot_counter
  fi
  save_env boot_counter
fi
### END /etc/grub.d/08_fallback_counting ###

### BEGIN /etc/grub.d/10_linux ###
insmod part_msdos
insmod xfs
if [ x$feature_platform_search_hint = xy ]; then
  search --no-floppy --fs-uuid --set=root  7e8029d1-da36-4524-84d0-8091deedc38b
else
  search --no-floppy --fs-uuid --set=root 7e8029d1-da36-4524-84d0-8091deedc38b
fi
insmod part_msdos
insmod xfs
if [ x$feature_platform_search_hint = xy ]; then
  search --no-floppy --fs-uuid --set=boot  7e8029d1-da36-4524-84d0-8091deedc38b
else
  search --no-floppy --fs-uuid --set=boot 7e8029d1-da36-4524-84d0-8091deedc38b
fi

# This section was generated by a script. Do not modify the generated file - all changes
# will be lost the next time file is regenerated. Instead edit the BootLoaderSpec files.
#
# The blscfg command parses the BootLoaderSpec files stored in /boot/loader/entries and
# populates the boot menu. Please refer to the Boot Loader Specification documentation
# for the files format: https://www.freedesktop.org/wiki/Specifications/BootLoaderSpec/.

# The kernelopts variable should be defined in the grubenv file. But to ensure that menu
# entries populated from BootLoaderSpec files that use this variable work correctly even
# without a grubenv file, define a fallback kernelopts variable if this has not been set.
#
# The kernelopts variable in the grubenv file can be modified using the grubby tool or by
# executing the grub2-mkconfig tool. For the latter, the values of the GRUB_CMDLINE_LINUX
# and GRUB_CMDLINE_LINUX_DEFAULT options from /etc/default/grub file are used to set both
# the kernelopts variable in the grubenv file and the fallback kernelopts variable.
if [ -z "${kernelopts}" ]; then
  set kernelopts="root=UUID=c9aa25ee-e65c-4818-9b2f-fa411d89f585 ro console=ttyS0,115200n8 console=tty0 net.ifnames=0 rd.blacklist=nouveau nvme_core.io_timeout=4294967295 crashkernel=auto rd.lvm.lv=newdirs/home rd.lvm.lv=newdirs/tmp rd.lvm.lv=newdirs/var rd.lvm.lv=newdirs/var_log rd.lvm.lv=newdirs/var_log_audit rd.lvm.lv=newdirs/var_tmp fips=1 boot=UUID=7e8029d1-da36-4524-84d0-8091deedc38b "
fi

insmod blscfg
blscfg
### END /etc/grub.d/10_linux ###

### BEGIN /etc/grub.d/10_reset_boot_success ###
# Hiding the menu is ok if last boot was ok or if this is a first boot attempt to boot the entry
if [ "${boot_success}" = "1" -o "${boot_indeterminate}" = "1" ]; then
  set menu_hide_ok=1
else
  set menu_hide_ok=0 
fi
# Reset boot_indeterminate after a successful boot
if [ "${boot_success}" = "1" ] ; then
  set boot_indeterminate=0
# Avoid boot_indeterminate causing the menu to be hidden more then once
elif [ "${boot_indeterminate}" = "1" ]; then
  set boot_indeterminate=2
fi
# Reset boot_success for current boot 
set boot_success=0
save_env boot_success boot_indeterminate
### END /etc/grub.d/10_reset_boot_success ###

### BEGIN /etc/grub.d/12_menu_auto_hide ###
if [ x$feature_timeout_style = xy ] ; then
  if [ "${menu_show_once}" ]; then
    unset menu_show_once
    save_env menu_show_once
    set timeout_style=menu
    set timeout=60
  elif [ "${menu_auto_hide}" -a "${menu_hide_ok}" = "1" ]; then
    set orig_timeout_style=${timeout_style}
    set orig_timeout=${timeout}
    if [ "${fastboot}" = "1" ]; then
      # timeout_style=menu + timeout=0 avoids the countdown code keypress check
      set timeout_style=menu
      set timeout=0
    else
      set timeout_style=hidden
      set timeout=1
    fi
  fi
fi
### END /etc/grub.d/12_menu_auto_hide ###

### BEGIN /etc/grub.d/20_linux_xen ###
### END /etc/grub.d/20_linux_xen ###

### BEGIN /etc/grub.d/20_ppc_terminfo ###
### END /etc/grub.d/20_ppc_terminfo ###

### BEGIN /etc/grub.d/30_os-prober ###
### END /etc/grub.d/30_os-prober ###

### BEGIN /etc/grub.d/30_uefi-firmware ###
### END /etc/grub.d/30_uefi-firmware ###

### BEGIN /etc/grub.d/40_custom ###
# This file provides an easy way to add custom menu entries.  Simply type the
# menu entries you want to add after this comment.  Be careful not to change
# the 'exec tail' line above.
### END /etc/grub.d/40_custom ###

### BEGIN /etc/grub.d/41_custom ###
if [ -f  ${config_directory}/custom.cfg ]; then
  source ${config_directory}/custom.cfg
elif [ -z "${config_directory}" -a -f  $prefix/custom.cfg ]; then
  source $prefix/custom.cfg;
fi
### END /etc/grub.d/41_custom ###

[ec2-user@ip-10-10-2-220 ~]$ sudo cat /etc/sysconfig/kernel 
# UPDATEDEFAULT specifies if new-kernel-pkg should make
# new kernels the default
UPDATEDEFAULT=yes

# DEFAULTKERNEL specifies the default kernel package type
DEFAULTKERNEL=kernel
Sadashiva Murthy M's picture Guru 1397 points
22 April 2022 3:38 PM

Yes, that looks perfect as well to me. Try setting the default kernel using this command "grubby --set-default-index=0", and later reboot and test if that helps. Just noticed that some people have reported stating that grub2-set-default didn't work on RHEL8.x versions and only "grubby" did work well.

JW Community Member 26 points
22 April 2022 10:07 PM

I ran the command and rebooted, but it doesn't look like anything has changed:

[ec2-user@ip-10-10-2-220 ~]$ uname -r
4.18.0-305.el8.x86_64
Sadashiva Murthy M's picture Guru 1397 points
23 April 2022 7:11 AM

If you attempt to choose the latest kernel stanza/line while the system is booting up i.e when grub prompts up on the screen, would that loads the latest kernel successfully? If you have console access then you could check this. Also, send the output of these commands:

# du -sh  /boot/*
# cat /etc/default/grub
# grubby --default-kernel; grubby --default-index; grubby --default-title
MY Guru 1695 points
23 April 2022 7:29 PM

It could be the order of the files in /boot/loader/entries/. You could try forcing it onto the latest kernel at next reboot via

grub2-reboot 'cc248e8c4ee74660b67b51ef99a7837a-4.18.0-348.20.1.el8_5.x86_64'

where the setting is the id of the kernel you want as listed in your grubby output.

JW Community Member 26 points
6 May 2022 8:57 PM

Hi, sorry I got pulled to a different project and forgot about this. I should be able to test the order of files on Monday and I'll update here. Thanks!

DB Newbie 16 points
21 June 2023 1:52 PM

Hi

Did you managed to fix the issue?

AU Newbie 10 points
21 November 2023 1:59 AM

I have the same issue with Red Hat 8 running on AWS ec2 instance. Everything looks good, separate boot partition is mounted, latest kernel installed and is default index, default kernel except uname is pointing still to the old kernel. Not sure what else to try.