RHEV 3.2 - Thin Provisioned Virtual Disks "expanding" when Live Migrated
I don't know if the condition I am witnessing is misreporting by RHEV, or a valid size.
I create a VM on one array that had a 50GB Thin Provisioned VD.
I then migrated to another array.
When I now review the VD, it indicates the Virtual Size is 50GB and the Actual Size is 53GB
Has anyone else seen this, or know what is going on here?
Thanks!
Responses
Red Hat
Guru
4649 points
You mean you migrated it from storage domain to another storage domain? If yes, was that a live storage migration?
If there are snapshots on the disks, then an overhead like this is expected. We can give more details if you open a case with a log collector after reviewing details of the disk.
Guru
6863 points
Thanks Sadique, (I also opened a case as I believe I might have found an issue).
A: Yes, I migrated from one domain to another using a live migration.
The odd thing is I actually built out this entire RHEV environment JUST for testing.
1 RHEV Manager
1 RHEL Hypervisor (6.5)
2 x Ent Arrays (and 2 storage domains).
I literally did the following:
* I built up the RHEV envrionment
* I built the VM using one storage domain, which should have consumed about 1.7GB
* I then executed a live migration to the other domain
I now understand the snapshot aspect a bit better (having done this about 40 times now ;-) which is also somewhat disappointing as you have to shut the VM down to consolidate the snapshots (if I recall correctly) which puts me in a bad spot also as I now have to schedule/coordinate downtime.
I thought there was a way to attach an image to a thread, but I am not seeing it.
EDIT: Fortunately this behavior was not observed in my production RHEV Cluster. So, I will continue to work with Support to see what is causing this particular issue.
I have something like this in my system:
Thin-provision disks:
A) Virtual Size: 100GB, Actual Size: 106GB
B) Virtual Size: 60GB, Actual Size: 84GB
C) Virtual Size: 25GB, Actual Size: 28GB
D) Virtual Size: 70GB, Actual Size: 73GB
E) Virtual Size: 30GB, Actual Size: 35GB
So I found at least 5 cases in my system.
3 Hypervisors based on FC Array.
Red Hat
Guru
4649 points
If you have done snapshot operations on thin privisioned disks, like creating snapshots, deleting snapshots, committing snapshots and etc, then an overhead like this is expected. Eg, When you merge a snapshot (delete), it involves merging data in two volumes to a single volumes. To rule out possibility of the single volume running out of space during merge, RHEV may over extend the disk to approximate 10% of the virtual size in some scenario which it cannot reclaim.
If you are using snapshot on pre-allocated disks, then big overhead is expected depending upon how much % of data is written. Eg, if you create a 100GB pre-allocated disk and write 10GB of data, create snapshot and then write 90 GB of data, you will see Virtual size 100G and actual size 190GB. When you delete the snapshot, you would end up seeing 100GB virtual size and around 110GB actual size.
I found out that my Virtal Machines had live migrated disks which create snapshot for the moment of migration. Try seeing VM -> Disks & snapshots. I found my machines had snapshots named something with "Live migration".
You can delete it when VM is shutdown.
Guru
6863 points
Thanks for the feedback Sadique and Krzysztof. That makes sense that the migration would consume the space.
Krzysztof - Support provided me with a bugid to the RFE for "hot snapshot consolidation" (as I would call it) - 647386. It's internal so we can't view it.
I believe my specific issue is that once on the new storage (as 53GB) RHEV attempts to collapse everything that it brought over, and to do so, it needs space for the entire VM to consolidate, then collapse (another 53GB, then it would drop back down to 3GB) but I only have 42GB available. Which.. at this point, I cannot remove the snapshot even with the VM down. I'm going to repeat this test using a VM that only has 20GB VD, migrating from a 1.2TB domain to one that is 100GB and see what happens.
Welcome! Check out the Getting Started with Red Hat page for quick tours and guides for common tasks.