Red Hat Training

A Red Hat training course is available for Red Hat Enterprise Linux

Chapter 36. Basic System Recovery

When things go wrong, there are ways to fix problems. However, these methods require that you understand the system well. This chapter describes how to boot into rescue mode, single-user mode, and emergency mode, where you can use your own knowledge to repair the system.

36.1. Rescue Mode

36.1.1. Common Problems

You might need to boot into one of these recovery modes for any of the following reasons:
  • You are unable to boot normally into Red Hat Enterprise Linux (runlevel 3 or 5).
  • You are having hardware or software problems, and you want to get a few important files off of your system's hard drive.
  • You forgot the root password.

36.1.1.1. Unable to Boot into Red Hat Enterprise Linux

This problem is often caused by the installation of another operating system after you have installed Red Hat Enterprise Linux. Some other operating systems assume that you have no other operating system(s) on your computer. They overwrite the Master Boot Record (MBR) that originally contained the GRUB boot loader. If the boot loader is overwritten in this manner, you cannot boot Red Hat Enterprise Linux unless you can get into rescue mode and reconfigure the boot loader.
Another common problem occurs when using a partitioning tool to resize a partition or create a new partition from free space after installation, and it changes the order of your partitions. If the partition number of your / partition changes, the boot loader might not be able to find it to mount the partition. To fix this problem, boot in rescue mode and modify the /boot/grub/grub.conf file.
For instructions on how to reinstall the GRUB boot loader from a rescue environment, refer to Section 36.1.2.1, “Reinstalling the Boot Loader”.

36.1.1.2. Hardware/Software Problems

This category includes a wide variety of different situations. Two examples include failing hard drives and specifying an invalid root device or kernel in the boot loader configuration file. If either of these occur, you might not be able to reboot into Red Hat Enterprise Linux. However, if you boot into one of the system recovery modes, you might be able to resolve the problem or at least get copies of your most important files.

36.1.1.3. Root Password

What can you do if you forget your root password? To reset it to a different password, boot into rescue mode or single-user mode, and use the passwd command to reset the root password.

36.1.2. Booting into Rescue Mode

Rescue mode provides the ability to boot a small Red Hat Enterprise Linux environment entirely from CD-ROM, or some other boot method, instead of the system's hard drive.
As the name implies, rescue mode is provided to rescue you from something. During normal operation, your Red Hat Enterprise Linux system uses files located on your system's hard drive to do everything — run programs, store your files, and more.
However, there may be times when you are unable to get Red Hat Enterprise Linux running completely enough to access files on your system's hard drive. Using rescue mode, you can access the files stored on your system's hard drive, even if you cannot actually run Red Hat Enterprise Linux from that hard drive.
To boot into rescue mode, you must be able to boot the system using one of the following methods[14]:
  • By booting the system from a boot CD-ROM or DVD.
  • By booting the system from other installation boot media, such as USB flash devices.
  • By booting the system from the Red Hat Enterprise Linux installation DVD.
Once you have booted using one of the described methods, add the keyword rescue as a kernel parameter. For example, for an x86 system, type the following command at the installation boot prompt:
linux rescue
If your system requires a third-party driver provided on a driver disc to boot, load the driver with the additional option dd:
linux rescue dd
For more information on using a driver disc at boot time, refer to Section 6.3.3, “Use a Boot Option to Specify a Driver Update Disk” for x86 systems or Section 13.3.3, “Use a Boot Option to Specify a Driver Update Disk” for Power Systems servers.
If a driver that is part of the Red Hat Enterprise Linux 6.9 distribution prevents the system from booting, blacklist that driver with the rdblacklist option. For example, to boot into rescue mode without the foobar driver, run:
linux rescue rdblacklist=foobar
You are prompted to answer a few basic questions, including which language to use. It also prompts you to select where a valid rescue image is located. Select from Local CD-ROM, Hard Drive, NFS image, FTP, or HTTP. The location selected must contain a valid installation tree, and the installation tree must be for the same version of Red Hat Enterprise Linux as the Red Hat Enterprise Linux disk from which you booted. If you used a boot CD-ROM or other media to start rescue mode, the installation tree must be from the same tree from which the media was created. For more information about how to setup an installation tree on a hard drive, NFS server, FTP server, or HTTP server, refer to the earlier section of this guide.
If you select a rescue image that does not require a network connection, you are asked whether or not you want to establish a network connection. A network connection is useful if you need to backup files to a different computer or install some RPM packages from a shared network location, for example.
The following message is displayed:
The rescue environment will now attempt to find your Linux installation and mount it under the directory /mnt/sysimage. You can then make any changes required to your system. If you want to proceed with this step choose 'Continue'. You can also choose to mount your file systems read-only instead of read-write by choosing 'Read-only'. If for some reason this process fails you can choose 'Skip' and this step will be skipped and you will go directly to a command shell.
If you select Continue, it attempts to mount your file system under the directory /mnt/sysimage/. If it fails to mount a partition, it notifies you. If you select Read-Only, it attempts to mount your file system under the directory /mnt/sysimage/, but in read-only mode. If you select Skip, your file system is not mounted. Choose Skip if you think your file system is corrupted.
Once you have your system in rescue mode, a prompt appears on VC (virtual console) 1 and VC 2 (use the Ctrl-Alt-F1 key combination to access VC 1 and Ctrl-Alt-F2 to access VC 2):
sh-3.00b#
If you selected Continue to mount your partitions automatically and they were mounted successfully, you are in single-user mode.
Even if your file system is mounted, the default root partition while in rescue mode is a temporary root partition, not the root partition of the file system used during normal user mode (runlevel 3 or 5). If you selected to mount your file system and it mounted successfully, you can change the root partition of the rescue mode environment to the root partition of your file system by executing the following command:
chroot /mnt/sysimage
This is useful if you need to run commands such as rpm that require your root partition to be mounted as /. To exit the chroot environment, type exit to return to the prompt.
If you selected Skip, you can still try to mount a partition or LVM2 logical volume manually inside rescue mode by creating a directory such as /foo, and typing the following command:
mount -t ext4 /dev/mapper/VolGroup00-LogVol02 /foo
In the above command, /foo is a directory that you have created and /dev/mapper/VolGroup00-LogVol02 is the LVM2 logical volume you want to mount. If the partition is of type ext2 or ext3 replace ext4 with ext2 or ext3 respectively.
If you do not know the names of all physical partitions, use the following command to list them:
fdisk -l
If you do not know the names of all LVM2 physical volumes, volume groups, or logical volumes, use the pvdisplay, vgdisplay or lvdisplay commands, respectively.
From the prompt, you can run many useful commands, such as:
  • ssh, scp, and ping if the network is started
  • dump and restore for users with tape drives
  • parted and fdisk for managing partitions
  • rpm for installing or upgrading software
  • vi for editing text files

36.1.2.1. Reinstalling the Boot Loader

In many cases, the GRUB boot loader can mistakenly be deleted, corrupted, or replaced by other operating systems.
The following steps detail the process on how GRUB is reinstalled on the master boot record:
  • Boot the system from an installation boot medium.
  • Type linux rescue at the installation boot prompt to enter the rescue environment.
  • Type chroot /mnt/sysimage to mount the root partition.
  • Type /sbin/grub-install bootpart to reinstall the GRUB boot loader, where bootpart is the boot partition (typically, /dev/sda).
  • Review the /boot/grub/grub.conf file, as additional entries may be needed for GRUB to control additional operating systems.
  • Reboot the system.

36.1.3. Booting into Single-User Mode

One of the advantages of single-user mode is that you do not need a boot CD-ROM; however, it does not give you the option to mount the file systems as read-only or not mount them at all.
If your system boots, but does not allow you to log in when it has completed booting, try single-user mode.
In single-user mode, your computer boots to runlevel 1. Your local file systems are mounted, but your network is not activated. You have a usable system maintenance shell. Unlike rescue mode, single-user mode automatically tries to mount your file system. Do not use single-user mode if your file system cannot be mounted successfully. You cannot use single-user mode if the runlevel 1 configuration on your system is corrupted.
On an x86 system using GRUB, use the following steps to boot into single-user mode:
  1. At the GRUB splash screen at boot time, press any key to enter the GRUB interactive menu.
  2. Select Red Hat Enterprise Linux with the version of the kernel that you wish to boot and type a to append the line.
  3. Go to the end of the line and type single as a separate word (press the Spacebar and then type single). Press Enter to exit edit mode.

36.1.4. Booting into Emergency Mode

In emergency mode, you are booted into the most minimal environment possible. The root file system is mounted read-only and almost nothing is set up. The main advantage of emergency mode over single-user mode is that the init files are not loaded. If init is corrupted or not working, you can still mount file systems to recover data that could be lost during a re-installation.
To boot into emergency mode, use the same method as described for single-user mode in Section 36.1.3, “Booting into Single-User Mode” with one exception, replace the keyword single with the keyword emergency.


[14] Refer to the earlier sections of this guide for more details.