Ideal LVM2 Extent Size for VMFS on SAN?

Latest response

What is the ideal LVM2 Extent size, if not the default of 4MiB, when running EL5 as a guest under VMware ESX 4 or 5, using VMFS and SAN storage?

Are there any considerations based on VMware ESX VMFS extents?  SAN allocation?  Etc...?  Anything that may affect this, or factor in?

I assume storage alignment on the LVM2 Extent (e.g., 4MiB) is a given.  But are there any others?  Or do people fall back to 1MiB?  Increase to 32MiB?  128MiB?  Other?

My client seems to be using 128MiB by default for bare metal, local disk (2.5" SAS).  For VMware VMFS on SAN, I'm suggesting we fall back to the 4MiB default.

Wanted to see if there are any recommendations others have, or any factors that would cause one to use anything but the default 4MiB.

Responses

You're looking at a stack of:

* Guest FS on top of Guest LV (assuming your app doesn't use what it thinks are raw devices).

* Guest LV on top of VMDK

* VMDK on top of VMFS

* VMFS on top of SAN LUN.

 

You'd probably want to verify the I/O widths of each of the underlying devices in the path, as well as the desired I/O characteristics of the application using the guest's FS before you can decide on what to do at an intermediate layer like the LV running in your ESX guest.

 

Also, probably goes without saying, but you'll want to have made sure that whoever set up your VMFSes aligned the datastore's boundaries with the underlying LUNS. Otherwise, you're probably wasting your time trying to optimize widths and boundaries of higher-up data-structures.