RHEL 6.6 problem after kickstart install

Latest response

I am deploying RHEL 6.6 to a HP blade and storage is located on a SAN connecting via FCOE (boot from SAN enabled).

Using the interactive installation to install the base operating system (using basic server option) the installation works and boots fine.

After using the anaconda-ks.cfg file generated during the interactive installation, I am seeing errors while booting.

The errors are being printing to the console, eg:

Enabling /etc/fstab swaps:                                 [  OK  ]
Entering non-interactive startup
Calling the system activity data collector (sadc)... /etc/rc3.d/S01sysstat: line 31: /usr/lib64/sa/sa1: No such file or directory
.
.
.
Starting auditd: /etc/rc3.d/S11auditd: line 60: warning: setlocale: LC_TIME: cannot change locale (en_US): No such file or directory
/etc/rc3.d/S11auditd: line 61: warning: setlocale: LC_ALL: cannot change locale (en_US): No such file or directory
/etc/rc3.d/S11auditd: line 66: warning: setlocale: LC_ALL: cannot change locale (en_US): No such file or directory
/etc/init.d/functions: line 151: /usr/bin/dirname: No such file or directory
/bin/bash: warning: setlocale: LC_ALL: cannot change locale (en_US): No such file or directory
                                                           [FAILED]
Starting portreserve: /etc/rc.d/init.d/functions: line 151: /usr/bin/dirname: No such file or directory
                                                           [  OK  ]
Starting system logger: /etc/init.d/functions: line 151: /usr/bin/dirname: No such file or directory
                                                           [  OK  ]
/etc/rc3.d/S13irqbalance: line 20: id: command not found
/etc/rc3.d/S13irqbalance: line 20: [: =: unary operator expected
/etc/rc3.d/S13rpcbind: line 34: id: command not found
.
.
.

There are other similar errors printed later during booting as well.

After booting completes, I can log in and see that the files do exist (eg. /usr/bin/dirname).

I've also noticed that /var/log/boot.log does not exist (so I've had to type out the above).

Any idea what the difference might be when deploying via kickstart?

Thanks

Responses

I use the same method to create my kickstart file (interactive). Are you still using the DVD as the install source, or did you change it to use a network-based installation? Also - did you happen to create a separate /usr? I have had issues like this arise when I installed to a SAN volume (by accident) where I had installed initrd, /boot, etc.. to a local disk, but the rest of the OS was on the SAN. Unfortunately I'm still at a bit of a loss what is going on in your particular situation though. It seems as though either a filesystem is not available or a global ENV variable was modified in an undesired way. I'm actually leaning towards the filesystem not being available and I suspect it is due to the initrd.img not being valid (for your current hardware). It sounds like you are testing and this is a new build. I would try to rebuild the boot images using dracut and see if that helps.

http://people.redhat.com/harald/dracut-rhel6.html

mv /boot/initramfs-$(uname -r).img /boot/initramfs-$(uname -r).img.backup
dracut -v /boot/initramfs-$(uname -r).img $(uname -r)

Or

dracut --force /boot/initramfs-`uname -r`.img 

Sorry if my response is all over the place. I believe the issue I had was how the devices were ordered on my system during build time and it assigned a SAN device "sda" - which I had to resolve using ONBIOSDIKS (if I recall correctly). Anyhow - I would validate that the partitions and volumes wound up on the devices you had intended and they try rebuilding the boot files (and hopefully if I am not right, someone else chimes in). Good luck!

Thanks, I've done some more investigation. I think it has to do with how some of the file system is mounted, not the kickstart process as I originally thought.
The LUN that I am booting from has a /boot partition on it, with the remainder containing a Logical Volume Group.
Within the Logical Volume group, logical volumes are used to mount the following:
/
/home
/var/log
/usr
/tmp
/opt
Using this scheme I am getting the errors, as seen in the original post.

I tried altering the above scheme to just contain one logical volume mounted to / and the boot errors I was seeing do not appear.

It seems that the /var/log, /usr etc are not properly available during the boot process.

I did try running the dracut commands mentioned but it didnt make any difference.

I have a similar boot from SAN setup working on some different hardware with a logical volume group setup as above and it works fine (it's not using FCOE though).

PREFACE: I'm not advising you against what you are trying to do. However, I recall coming across some documents online which talked about separating /usr (and that most people felt it was not a good idea).

Here's one example: https://access.redhat.com/solutions/53005
which references:
https://access.redhat.com/documentation/en-US/Red_Hat_Enterprise_Linux/6/html/Installation_Guide/s2-diskpartrecommend-x86.html
"/usr 250 MB, but avoid placing this on a separate partition "

I have never used a separate mount for /usr (even when I was a UNIX admin) but I recall things like Veritas would put some binaries in /etc and I thought it was strange, and then someone explained that they put them there so that the minimum functionality was available before /usr was mounted - which seems logical.

I imagine that since /usr is not available, not much is going to work (i.e. mounting any other FS'es). However I thought the dracut process should figure out what is needed and add it to the boot image. I could only find the /usr module referenced at a Gentoo site
http://wiki.gentoo.org/wiki/Dracut
Look under List of Modules for "usrmount arranges for /usr to be mounted always "

Unfortunately I won't be able to provide any guidance as I have never personally had to work through this before. I do wonder, though, if you made /usr a partition and not a volume, if that would work?

I just did some poking...

root@blah # grep ^#add /etc/dracut.conf
#add_dracutmodules+=""
#add_drivers+=""

Change the following (I think?)
add_dracutmodules+="usrmount"

Check out all the modules in /usr/lib/dracut/modules.d

I tried putting /usr on a separate partition but still have the error. I don't mind if I have /usr as part of the root volume, which gets rid of some problems, but I also have /var/log as a logical volume. During boot up things aren't getting written there either (eg. boot.log). I would prefer /var/log to be separate to the root volume at least.
I had a look thought dmesg and the following is what dracut is doing:

dracut: Scanning devices dm-6  for LVM logical volumes RootVG/rootVol
dracut: inactive 'dev/RootVG/rootVol' [19.53 GiB] inherit
dracut: inactive 'dev/RootVG/varlogVol' [9.51 GiB] inherit
dracut: inactive 'dev/RootVG/homeVol' [49.11 GiB] inherit
EXT4-fs {dm-11): mounted filesystem with ordered data mode. Opts:
dracut: Remounting /dev/mapper/RootVG-rootVol with -o _rnetdev,ro
EXT4-fs {dm-11): mounted filesystem with ordered data mode. Opts:
dracut: Mounted root filesystem /dev/mapper/RootVG-rootVol

So it looks like only the root vol is getting mounted for use, which explains why the other volumes can't be used.
I will do some reading to see what how to get dracut to mount the other logical volumes.

Just opening myself regarding this FCoE bug. Above is no mention what network adapter is involved but it sounds like Broadcom.
We had the exact same issue with Broadcom 57810-k on Dell M630 server. It effects both Rhel6.6 and 7.1.
The basic difference between working Qlogic QMD8262-K (not anymore available) and Emulex OCm14102-U4-D is that Broadcom doesn't advertise itself as hard disk c: to server system bios. So it's up to linux drivers to handle devices and it fails. Don't know why. Maybe you do.
Another reason with Broadcom is that it seems this card hasn't mend to boot from san as everyone our physical servers does. Special cases excluded. We tried to get Broadcon working over three weeks with no luck. Then we tried Emulex and got it working in a half of a hour. RedHat 6.6 an 7.1 both are working perfectly. Now we are more than happy.

I am experiencing the same issue, using Broadcom-based chip, in HP blade. Did you find any solution why using kickstart installation the mounting of logical volumes fails during boot?

Hmm, FYI i tried v6.8 and everything worked fine.

Close

Welcome! Check out the Getting Started with Red Hat page for quick tours and guides for common tasks.