Why does the kernel panic with error "dracut: FATAL: No or empty root= argument"

Solution Verified - Updated -

Environment

  • Red Hat Enterprise Linux 6
  • Red Hat Enterprise Linux 7

Issue

Issue A:

  • Booting the system with the new kernel fails. However, the same works for the older versions of the kernel.
  • The new kernel panics with the following error messages on console:
dracut: FATAL: No or empty root= argument
dracut: Refusing to continue

Kernel panic - not syncing: Attempting to kill init!
Pid; 1, comm: init Not tainted 2.6.32.279.14.1.el6.x86_64 #1
Call Trace:
 [<ffffffff814fd98a>] ? panic+0xa0/0x168
 [<ffffffff81070c42>] ? do_exit+0x862/0x870
 [<ffffffff8117d0a5>] ? fput+0x25/0x30
 [<ffffffff81070ca8>] ? do_group_exit+0x58/0xd0
 [<ffffffff81070d37>] ? sys_exit_group+0x17/0x20
 [<ffffffff8100b0f2>] ? system_call_fastpath+0x16/0x1b
  • The below error is seen on the console before the kernel panics:
dracut: One or more specified logical volumes(s) not found.

Issue B:

  • System does not boot with the following message after update the kernel, although root= is given correctly.
mknod: '/dev/null': File exists
mknod: '/dev/console': File exists
/init: 102: cannot open /proc/cmdline: No such file
mkdir: cannot create directory '/dev/pts': File exists
dracut: FATAL: No or empty root=argument
dracut: Refusing to continue

Kernel panic - not syncing: Attempted to kill init!

Resolution

There are two resolutions depends on the root cause.

Resolution A:

Add a missing parameter to the root= option on the kernel command line in /boot/grub/grub.conf file, e.g. root=/dev/mapper/vg00-lvol1.

Example:

title Red Hat Enterprise Linux Server (2.6.32-279.14.1.el6.x86_64)
    root (hd0,0)
    kernel /vmlinuz-2.6.32-279.14.1.el6.x86_64 ro root=/dev/mapper/vg00-lvol1 rd_NO_LUKS LANG=en_US.UTF-8 rd_LVM_LV=vg00/lvol2 rd_NO_MD SYSFONT=latarcyrheb-sun16 rd_LVM_LV=vg00/lvol1  KEYBOARDTYPE=pc KEYTABLE=us crashkernel=auto rhgb quiet rd_NO_DM rhgb quiet
    initrd /initramfs-2.6.32-279.14.1.el6.x86_64.img
  • Verify root and swap logical volume names in /boot/grub/grub.conf.

Resolution B:

Re-create the initramfs with the following steps.

  1. Remove "0" user in the /etc/passwd file by running userdel command:

    # grep "^0" /etc/passwd  
0:x:1234:1234::/home/0:/sbin/nologin
# userdel 0

    Note, that files in the user's home directory and the user's mail spool will remain after running userdel. These files and files located in other file systems will have to be searched for and deleted manually, if needed.

  2. Re-create the initramfs used for this boot. See How to rebuild the initial ram disk, including in linux rescue mode

Root Cause

Root Cause A:

  • The value of the root= option on the kernel command line was not correct. This option is required by dracut, which handles the boot.

Example:

kernel /vmlinuz-2.6.32-279.14.1.el6.x86_64 ro root= rd_NO_LUKS LANG=en_US.UTF-8 rd_LVM_LV=vg00/lvol2 rd_NO_MD ..
                                                ^
                                                +
                                            [ Value of the root= option is Missing ]

Root Cause B:

  • If there is a user account with user name "0", owner for all files under the initramfs will be the "0" user, and mount in the initramfs cannot mount the root file system.

  • The root cause for this behaviour is one of the commands that dracut uses during initramfs build process:

    cpio -R 0:0 -H newc -o --quiet
  • Note the "-R 0:0" part which tries to change the user and group ownership of all files in initiramfs to root:root. Since there is an account with username "0", it's set as the owner of all files - and not root, and in turn makes /bin/mount binary unusable - which leads to a non-bootable system.
  • There is also a private bugzilla bug to resolve this problem in cpio / dracut.

Diagnostic Steps

  • Capture a screen-shot of the complete panic message.
  • Collect sosreport of the system.
  • Verify the value of the root= option in /boot/grub/grub.conf for all kernel entries.

Example:

# cat /boot/grub/grub.conf 
# grub.conf generated by anaconda
#
# Note that you do not have to rerun grub after making changes to this file
# NOTICE:  You have a /boot partition.  This means that
#          all kernel and initrd paths are relative to /boot/, eg.
#          root (hd0,0)
#          kernel /vmlinuz-version ro root=/dev/mapper/vg00-lvol1
#          initrd /initrd-[generic-]version.img
#boot=/dev/sdd
default=0
timeout=5
splashimage=(hd0,0)/grub/splash.xpm.gz
hiddenmenu
title Red Hat Enterprise Linux Server (2.6.32-279.14.1.el6.x86_64)
    root (hd0,0)
    kernel /vmlinuz-2.6.32-279.14.1.el6.x86_64 ro root= rd_NO_LUKS LANG=en_US.UTF-8 rd_LVM_LV=vg00/lvol2 rd_NO_MD SYSFONT=latarcyrheb-sun16 rd_LVM_LV=vg00/lvol1  KEYBOARDTYPE=pc KEYTABLE=us crashkernel=auto rhgb quiet rd_NO_DM rhgb quiet
    initrd /initramfs-2.6.32-279.14.1.el6.x86_64.img
title Red Hat Enterprise Linux (2.6.32-279.el6.x86_64)
    root (hd0,0)
    kernel /vmlinuz-2.6.32-279.el6.x86_64 ro root=/dev/mapper/vg00-lvol1 rd_NO_LUKS LANG=en_US.UTF-8 rd_LVM_LV=vg00/lvol2 rd_NO_MD SYSFONT=latarcyrheb-sun16 rd_LVM_LV=vg00/lvol1  KEYBOARDTYPE=pc KEYTABLE=us crashkernel=auto rhgb quiet rd_NO_DM rhgb quiet
    initrd /initramfs-2.6.32-279.el6.x86_64.img
  • Expand the initramfs used for this boot with root user.
  • Check the owner of /bin/mount file under the expanded initramfs.

This solution is part of Red Hat’s fast-track publication program, providing a huge library of solutions that Red Hat engineers have created while supporting our customers. To give you the knowledge you need the instant it becomes available, these articles may be presented in a raw and unedited form.

Comments