Post SAN Failure How to Restore VMs?
We had a power outage that took down all of our external storage. As a result, everything that was attached went down too. Our ESXi hosts were somewhat tolerant and easy to get going. However, our RHEV machines are not. It may be because I'm not very familiar with RHEV. Of utmost importance is that I was told a few months ago that our RHEV-M database is lost, so we have no connection to a GUI, and everything is command line.
So, rather than having one hypervisor and one management machine, we have 2 hypervisors, both that have domains (virtual machines?). I have a screen shot of running a "virsh -r list" command a few months ago, and both had 8 running domains. Now we have zero. I have rebooted in hopes that it would pick back up the storage that is online, but to no avail.
I restarted the ovirt-engine but that didn't produce fruit. Our domains are not using LVM for storage, but I think they were on external block storage (the suspect SAN). FDISK tells me that it sees what appears to be the raw disks and the /dev/mapper disks where the domains are stored. And on one of the hypervisors I can see disk files in /rhev/data-center/mnt/blockSD but none on the other hypervisor. and there is nothing in /var/run/vdsm/storage.
We're totally down. Any advice is greatly appreciated!
Responses
Hi F.D.,
Due to the nature of your issues and as you mentioned for some reason needing to use CLI instead of the UI I would highly recommend if you have access to the customer portal and opening new support cases to do so and depending on your entitlements/issues present a Support Engineer can review/provide assistance. The issues you are describing above will require considerable digging to ensure all is correct. In addition without knowing what account/environment version you are presently using getting to 4.1 if not already on that version will need to be a high consideration during this process.
Welcome! Check out the Getting Started with Red Hat page for quick tours and guides for common tasks.
