[RHEV] Unable to start the VM in the vm pool with "Can not run VM. There are no available running hosts with sufficient memory in VM's cluster" error message.

Solution Unverified - Updated -

Environment

  • Red Hat Enterprise Virtualization 3.0

Issue

  • While trying to start the VM in the vm pool, getting the following error message.
Can not run VM. There are no available running hosts with sufficient memory in VM's cluster.

Resolution

  • According to the new vm memory check logic if the Current Memory is lessthen or equals to Memory Limit then vm can start else vm can't start.

  • Where is the difference between Defined Memory and Physical Memory Guaranteed?

"Physical Memory Guaranteed" is the minimum threshold which will always be available to the VM - it's a value that RHEV takes care, but is not setted anywhere to the VM. VM will use as much memory as needed, till the limit of "Defined Memory". This flexible memory allocation is called "memory ballooning".

Using "free" comand, the "Total Memory" is always the "Defined Memory". You can also check the actual active memory on VM:

# grep "Active:" /proc/meminfo
  • System monitor have never been over 50% in use of memory for each host. Why is this happening? If the hosts are using all memory, why does not it appear in the system monitor? Why is the use of memory for each VM in 0%?
1. System monitor have never been over 50% in use of memory for each host. Why is this happening?
This value is coming from RHEV Host memory statistcs on "/proc/meminfo". It depends on how much memory is being used by the entire system, including VMs and all processes.

2. If the hosts are using all memory, why does not it appear in the system monitor?
Even when VMs are not using all Host memory, RHEV must assure that will have enough memory when VMs become high loaded. To do so, RHEV considers the worst case: The total "Defined Memory" (which I called mem_commited on comment#5) of all running VMs + some values, all mentioned on comment#5.

3. Why is the use of memory for each VM in 0%?
Do you have already installed rhev-agent on your VMs? You have to do so in order to see the individual VM memory statistics.
  • If I edit cluster Memory Optimization and I change it to Optimized for Server Load (it was in None), I understand that Page Sharing is enabled, and when I have identical VMs (it is our case) the number of VMs should increase significantly. I have done a test only running VMs of a pool (with 20GB Defined Memory and 2GB Phisical Memory Guaranteed), first with Windows XP VMs and later with RHEL6. I can only run 5 VMs in each server, it has not increase significantly. Then how is overcommit working? Is it sharing memory pages?

  • Memory overcommit (RHEV Memory Optimization) is not directly related with memory page de-duplication (RHEV Shared Memory).
    When you set your cluster to "Memory optimization" you are telling to RHEV: You are free to start VMs with 50% (server load) more memory than you effectively has (regarding my last comment#5 calculations).

  • Shared Memory is the result of KSM[1] memory page de-duplication. RHEV 3.0 currently has a bug colecting this statistic, which cause the almost all time 0% of Shared Memory, even with KSM daemon working hard. This bug is already fixed to next versions. As workarround, you can check the memory de-duplication statistcs through the command (on RHEV Hosts):

# echo $(( ($(</sys/kernel/mm/ksm/pages_sharing) * $(getconf PAGE_SIZE)) * (1024 * 1024) ))
  • The result is the amount of memory being saved by KSM de-duplication, in Mbytes. RHEV does not considers this value to any operation/calculation, overcommit included. But KSM will help your RHEV Hosts to avoid using swap when you overcommit them and they become highly loaded.

http://www.linux-kvm.org/page/KSM

Root Cause

  • No available memory on any host, so vm failed to start.

Diagnostic Steps

This solution is part of Red Hat’s fast-track publication program, providing a huge library of solutions that Red Hat engineers have created while supporting our customers. To give you the knowledge you need the instant it becomes available, these articles may be presented in a raw and unedited form.

Comments