Root file system in readonly mode

Latest response

I have a system that has hardware raid and three LVM's. We recently had the motherboard replaced and since then the system will not book. All the disks report being in read-only mode.
If we boot with a CD into rescue mode then the disks are just fine and we can read and write. We have run fsck but it does not fix the problem. fsck reports that all is well. fstab does not have the file systems in readonly mode.... It simply says "defaults"

We thing somewhere there must be a flag indicating the LVM or disk is in readonly mode but we don't know where to look.

Any ideas?

Responses

hi Velocity Global,

Just curious,

Which version of Red Hat are you using? Red Hat Enterprise linux 6 or 7? Workstation or Server? I ask because this solution only affects RHEL 6 https://access.redhat.com/solutions/372543 and if you are using RHEL 7, that solution may not work. Also, the file system fix differers between RHEL 6 and RHEL 7. RHEL 7 has an xfs repair command and RHEL 6 does not.

When you boot normally into the system, can you do the following command and make any of the file systems read-write? This procedure comes from this Red Hat link - and this is not a permanent fix, but just a test.

Also, examine this solution https://access.redhat.com/solutions/9494 but I really don't know if it will apply given the limited details.

NOTE: Open two terminals, as root

  • terminal window 1 (if needed, use psudo-terminals, ctrl-alt-f2, ctrl-alt-f3)
  • Look for recently written log files after mounting and do a "tail -f /var/log/nameoflogfile" that is being currently written to (you can do a tail -f of more than one, /var/log/messages /var/log/anotherfilename)

Terminal 1

[root@yoursystem ~]# cd /var/log
[root@yoursystem log]# tail -f messages

Terminal 2

# the next command is just for you to view your mointpoints
[root@yoursystem] # df -PhT 
# find a mountpoint and try this command: (and let's say "/tmp" is affected, I don't know in your case)
[root@yoursystem] # mount -o remount,rw /tmp
[root@yoursystem] # cd /tmp
[root@yoursystem] # date > /tmp/dateandtime.txt
[root@yoursystem] # cat /tmp/dateandtime.txt

Are you running multipath on this system? Another question, is this system VMware? https://access.redhat.com/solutions/532163

Examine for yourself the output of commands such as pvdisplay vgdisplay and lvdisplay

Reply back and let us know, someone ought to help you,

Regards

RJ

Hi Velocity Global,

Your initial message is a bit confusing, as you said

We recently had the motherboard replaced and since then the system will not book. All the disks report being in read-only mode.

Does this means system has failed to boot up after motherboard replacement? Or does it boot into read-only mode (root being mounted as read-only)? Please explain.

Check for '/etc/sysconfig/readonly-root' file and observe if 'READONLY=' is set to yes. This is just a quick guess, however, I suggest that you follow what RJ has mentioned about and make sure to write back...

Many thanks for your responses. They have vastly improved our knowledge. However we did finally get to the bottom of it: The mother board replacement resulted in a change of firmware.
This in turn required an upgrade of RedHat from 6.3 to 6.7. This all went smoothly but the system would not boot correctly. We had failed to notice that the GRUB loader had a second option for the new Kernel, so we were try to boot RHEL6.7 with the old 6.3 Kernel. Selecting the correct Kernel solved the problem, so we set that as the default in /etc/grub.cfg.

Once again, many thanks for your useful posts.

Thanks Velocity Global for the update for your situation. Please highly consider upgrading to RHEL 6.10 which is supported until some time late (November I think) of 2020 See Maintenance support 2 in this link.

Regards

RJ

Or even better : take the actual problem as an opportunity to install the latest stable edition RHEL 7.6 which
brings along a lot of advantages. Backup your data and restore it back to the new system. Yes, it is a little bit
more work that upgrading to RHEL 6.10, but the profit you gain should be worth the effort ... correct, RJ ? :)

Regards,
Christian

Couldn't agree more Christian

Velocity Global, please highly consider what Christian said. Whatever function your server is performing, highly consider a strategy where you can take it to a supported operating system, especially considering you have time between now and the end of life of RHEL 6 to take action and plan it out.