Backing up RHEV VMs on NetApp SAN
Hi there,
We are about to deploy a RHEV environment with three hypervisors on our NetApp SAN via Fibre Channel.
One thing I have not been able to find any information about in the official docs or otherwise, is how to use LUN snapshots for backing up and restoring the virtual machines.
As I see it, you would need to put similar VMs in a 1-LUN storage domain, make sure they are in a consistent state at once, and then create a snapshot of the underlying LUN?
How would you recover just one of these VMs later?
Is there a more clever way of doing this?
/Martin
Responses
You're not clearly saying what your virtualization solution is (VMware, HyperV, KVM, Xen, etc.) or what your available backup technologies are (NetBackup, Veeam, Legato, SnapManager, etc.). If you're using NetBackup for your backup system, it supports several hypervisors and methods for providing good, consistent-state save-sets and granular restore capabilities. If you're using VMware for your hypervisor, you've got the whole vSphere API available to you to create consistent-state save-sets.
In general, unless the your backup system is designed to support your particular virtualization environment, most of your backups are going to be "crash-consistent". This tends to be sub-ideal, but can be adequate.
Unfortunately, with applications that have dynamic data (e.g., a database), your backup solutions need to consist of more than just snapshotting the filesystem. You need to have a backup agent (such as SnapManager for Oracle) or a scripted process that quiesces/pauses the application, takes a snapshot, resumes the application and then backs up the snapshot. Otherwise, all you have is a snapshot of active data. This typically results in a "crash consistent" copy of the data. In the case of a database, when you go to recover from such a backup, you'll have to do a data restore and then recover to the database's last consistency point before you'll be able to resume operations.
I mentioned SnapManager since you made no mention of other backup technologies available to you. SnapManager's generally the way NetApp tries to get their storage customers to back things up. Dunno whether/how well it works in a KVM context, however (only ever used it on physical servers that were presented storage directly from the NetApp).
Hi Martin,
NetApp, in cooperation with Red Hat, has actually released a whitepaper titled "Designing a Shared Storage Infrastructure with Red Hat Enterprise Virtualization and NetApp Storage" that outlines a few suggested and supported snapshot use-cases for backups. It can be found here: http://www.netapp.com/us/library/technical-reports/tr-3914.html
The information described in the whitepaper was also presented by NetApp at Red Hat's 2011 Summit, and a copy of the slide deck used in that presentation can be found here: http://www.redhat.com/summit/2011/presentations/summit/whats_next/friday/benedict_f_945_enabling_rhev.pdf
As you'll find, there's no supported way to backup a single VM at the storage layer in RHEV. In the future you should be able to assign specific storage to a VM similar to how you can with KVM/libvirt now.
Hi Martin,
I *think* your plan sounds sane, but I'm not a RHEV expert, so I'm trying to get someone from that group to give it a read over and make sure there are no issues there.
From a storage perspective what you are trying to do is fine. Also, in step #4 you can use 'vgimportclone'. This will take a volume group that has a duplicate name to an active vg, and change the metadata so it does not conflict. This was only added in later updates of RHEL 5 (starting with update 5, if I remember correctly).
Hopefully we should have an answer for you soon on whether RHEV would have any issues with what you're doing.
Regards,
John Ruemker, RHCA
Red Hat Technical Account Manager
Online User Groups Moderator
Thanks for sharing Martin, information on use cases and best practices is always appreciated.
I'd like to hear more about the restore process, things might definitely get hairy there, because RHEV 2.x relies on raw data, and after a restore the extent offsets might move about.
The current supported solution is to use Export Domains, but this is an offline process. There's also an option of just backing up the VMs using guest based backup agents.
We still haven't got an answer on how to do a complete restore of a Virtual Machine, including configuration that fx. defines the path's to virtual disk devices etc.
We are at current point, in a posistion where we really need to do a full VM restore.
I can have a snapshot from NetApp presented from the StorageDomain, which should give access to files and configurations - but can we just replace the files on the Production StorageDomain with the ones from the Snapshot?
/Yngve
Welcome! Check out the Getting Started with Red Hat page for quick tours and guides for common tasks.
