kernel panic while booting after updating to kernel-3.10.0-693.1.1.el7.x86_64

Latest response

Everything was OK with kernel-3.10.0-514.26.2.el7.x86_64 and old versions, but after i update it to kernel-3.10.0-693.el7.x86_64 and kernel-3.10.0-693.1.1.el7.x86_64 OS failed to boot with CapsLock keep blinking.
the crash starts with
atombios stuck in loop for more than 5secs aborting

and end with
kernel panic not syncing : fatal exception
Kernel Offset : disabled

kernel-3.10.0-693.el7.x86_64 Image :
https://www.dropbox.com/s/f6gok5s23v4ib52/kernel-3.10.0-693.el7.x86_64.jpg?dl=0

kernel-3.10.0-693.1.1.el7.x86_64 Images :
https://www.dropbox.com/s/eyh7sfv7hftfs83/kernel-3.10.0-693.1.1.el7.x86_64_1.jpg?dl=0
https://www.dropbox.com/s/9fiyogxfj1w9ur7/kernel-3.10.0-693.1.1.el7.x86_64_2.jpg?dl=0

forget to mention that every time i boot with my working kernel a notification with kernel issues and cannot be reported because it is a hardware problem :
$abrt-cli list --since 1504030233
reason: mce: [Hardware Error]: Machine check events logged
time: Tue 29 Aug 2017 08:14:02 PM EET
cmdline: BOOT_IMAGE=/boot/vmlinuz-3.10.0-514.26.2.el7.x86_64 root=/dev/mapper/rhel-root ro crashkernel=auto rd.lvm.lv=rhel/root rhgb quiet LANG=en_US.UTF-8
uid: 0
Directory: /var/spool/abrt/oops-2017-08-29-20:14:02-774-0

my laptop is :
Dell inspiron 5567 i7-7500U
Intel® Core™ i7-7500U CPU @ 2.70GHz × 4
Graphic AMD R7 M445 4G

any help please

Responses

I found a few more references to this around the internet, it seems to be an amdgpu regression which is resolved in 4.10 and later, eg: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1656649

I logged Red Hat Private Bug 1486100 - amdgpu Radeon R7 panic in smu7_populate_single_firmware_entry.isra.3 to address this in RHEL.

For now, you could use the previous working kernel as a workaround. You might also have luck using the latest ELRepo kernel-ml package if you want to try that, though it is not supported by us.

Hi Jamie,

Alternatively Hassan can disable the AMD GPU by adding radeon.modeset=0 to the GRUB_CMDLINE_LINUX line in the /etc/default/grub file until this issue will be fixed, and then he can use the latest available Red Hat kernel. :)

Regards,
Christian

I'm thinking of excluding any kernel* updates in yum.conf because if kernel 3.10.0-514.26.2 been replaced i'll not be able to boot again until i found a solution

Hi Hassan,

Good idea ... please check what I wrote in my comment concerning AMD / open source GPU drivers. :)

Regards,
Christian

Hi Hassan,

In case that you want to choose the alternative solution I have posted in my reply to Jamie's suggestion above, please don't forget to update the GRUB configuration afterwards by executing sudo grub2-mkconfig -o /boot/efi/EFI/redhat/grub.cfg. Good luck !

Cheers :)
Christian

I've tried what you have suggested
my /etc/default/grub

GRUB_TIMEOUT=10
GRUB_DISTRIBUTOR="$(sed 's, release .*$,,g' /etc/system-release)"
GRUB_DEFAULT=saved
GRUB_DISABLE_SUBMENU=true
GRUB_TERMINAL_OUTPUT="console"
GRUB_CMDLINE_LINUX="crashkernel=auto rd.lvm.lv=rhel/root rhgb quiet radeon.modeset=0"
GRUB_DISABLE_RECOVERY="true"

and update the GRUB configuration noting change

and when i add radeon.modeset=0 amdgpu.runpm=0 acpi_backlight=intel_backlight to GRUB_CMDLINE_LINUX the only change that happen is the line iwlwifi capa flags index 3 larger than supported by driver appears

Images : https://www.dropbox.com/s/8hoxtr1op1yeefk/kernel_1.jpg?dl=0 https://www.dropbox.com/s/jb1q5x4q5kfml5h/kernel_2.jpg?dl=0

Hi Hassan,

I did some further investigation and have a few questions. Did you install the AMD drivers that are provided on the AMD website ? In case yes, I can't find your AMD R7 M445 4G GPU on the list of supported products -> AMD Linux Download Center Also, I can't find Linux drivers which already support RHEL 7.4, the latest supported version as of today is RHEL 7.3, so my advice would be to remove these drivers and check if the open source radeon drivers work.

Please check if you have these drivers installed : xorg-x11-drv-ati, and in case not, install them and report back if the system boots with these drivers (with and without the parameter radeon.modeset=0). Reboot the system after you uninstalled the AMD drivers and in case you reinstalled the open source drivers, reboot the system afterwards again.

Best practice is to do everything with GUI disabled from a virtual console :
sudo systemctl set-default multi-user.target and sudo reboot
Once you have finished the drivers installation - switch back to GUI mode : sudo systemctl set-default graphical.target and sudo reboot

Regards,
Christian

I've the xorg* already installed but i didn't try to install any AMD amdgpu* package, could this be the problem !! should i try to install the EL7.3 amdgpu version and see what will happens

Why not Hassan ? It cannot get worse - but I recommend to create a system backup before, otherwise you might have to reinstall the whole system ... drivers installation is not a trivial thing. I recommend to do it with Clonezilla, I am using this tool for years and it never let me down. I made presentations that show how to do it : Clonezilla - Create partition backup | Clonezilla - Restore partition backup

Regards,
Christian

Sorry i didn't mention that, my old labtop didn't have external GPU and this is my first linux installation with GPU card

No problem Hassan, drivers management for hybrid GPU solutions is something special. Things that work on one machine may not necessarily work on other machines. AMD graphics generally are a bit more complicated than NVIDIA graphics, so you have to try things out, if the AMD drivers don't work properly, we will have to wait until the issue is fixed in one of the next kernel releases - normally the radeon drivers work fine. By the way, I've checked the notebook specifications before responding. :)

Regards,
Christian

We've had another customer report the same issue overnight. Setting radeon.modeset=0 didn't work for them either.

I've raised the priority of the bug and we're tracking this on the knowledgebase at:

Hi Jamie,
Thanks :)

This will be much appreciated by many users ... AMD graphics drivers on Linux systems really are a pain these days.

Regards,
Christian

No worries, solving problems is what we're here for :)

I am also an AMD graphics fan at home. I haven't had any experience with amdgpu yet, but the old radeon driver has been very reliable for me for a long run of kernels. I gave up on the proprietary fglrx driver ages ago, last I checked it didn't even work with current Xorg.

Hi Jamie,

I can confirm the same, I have installed RHEL 7.4 on my old DELL Studio machine with a dedicated AMD graphics chip and that thing is running perfectly fine with the open source radeon drivers. :)

Regards,
Christian

We've seen the same issue with two RHEL 7.4 VMWare guests that were recently rebooted with the 3.10.0-693.1.1.el7 kernel. We discovered independently (before finding this issue) that booting back into an older kernel works around the issue.

Hey Tim, did you find a working solution? We have the same issue with an ESXi 6.5 Client (RHEL 7.2 and 7.4). But the only working "Kernel" is ther rescue boot-option.

We haven't found any reasonable solution. I initially indicated in my response that Red Hat had not even acknowledged this problem with a Knowledgebase article, but that was incorrect. Jamie links to the article, above. I guess we go with one of the two options listed there, until there's a final recommendation.

We've experienced the problem on both physical hardware (headless servers) and VM guests -- systems without any AMD GPU. Right now, we're very hesitant about rebooting any of our systems, as there's a strong chance the system will experience this problem.

I've now seen two separate reports that adding nomodeset - not radeon.modeset=0 - fixes this.

We have unfortunately found same here - about 50 out of 141 running RHEL7 VMs exhibited same issue after kernel-3.10.0-693.11.1.el7.x86_64 got installed. yum -y reinstall kernel seems to fix the issue most of the time.

BTW - found same issue after the installation of the previous kernel-3.10.0-514.10.2.el7.x86_64 - random at various VMs.

I ended up writing a script to compare the # of installed kernels against the number of created "/boot/initramfs-3.10..." files and do an automatice re-install of the kernel if the number is a mis-match.

So far so good - except for our developers are not happy at all as an occasional restart of a VM may not work.

If you're still looking into this, is it possible to try amdgpu.runpm=0 radeon.runpm=0 on the GRUB kernel line? This was confirmed to resolve the issue in at least one other case.

Hello, I got the same issue when I try to install the latest rhel alongside my ubuntu and windows 10.

kernel panic not syncing : fatal exception Kernel Offset : disabled

Is there new solutions about this issue. I cant install rhel because of this kernel issue. Tried everything above and still doesnt work.

Now that RHEL 7.5 is out, is this behaving differently for anyone?

Close

Welcome! Check out the Getting Started with Red Hat page for quick tours and guides for common tasks.