Appendix E. Tuning the Nova reserved memory and CPU allocation manually

Tuning the Nova environment for the planned workload can be a trial and error process. Red Hat recommends starting will a calculated base set of defaults and tune from there.

By tuning the reserved_host_memory_mb and cpu_allocation_ratio parameters, you can maximize the number of possible guests for the workload. Also, by fine tuning these values you can find the desired trade off between determinism and guest-hosting capacity for the workload.

Tuning Nova reserved memory

Nova’s reserved_host_memory_mb parameter is the amount of memory, in megabytes (MB), to reserve for the node. Keep in mind, that on a hyper-converged Compute/OSD nodes, the memory must be shared between the two services, as not to starve either service of their required resources.

The following is an example of how to determine the reserved_host_memory_mb value for a hyper-converged node. Given a node with 256GB of RAM and 10 OSDs, assuming that each OSD consumes 3GB of RAM, that is 30GB of RAM for Ceph, and leaving 226GB of RAM for Nova Compute. If the average guest each uses 2GB of RAM, then the overall system could host 113 guest machines. However, there is the additional overhead for each guest machine running on the hypervisor that you must account for. Assuming this overhead is 500MB, the maximum number of 2GB guest machines that could be ran would be approximately 90.

Here is the mathematical formulas:

Approximate Number of Guest Machines = ( Memory Available for Nova in GB / ( Memory per Guest Machine in GB + Hypervisor Memory Overhead in GB ) )

Example

90.4 = ( 226 / ( 2 + .5 ) )

Given the approximate number of guest machines and the number of OSDs, the amount of memory to reserve for Nova can be calculated.

Nova Reserved Memory in MB = 1000 * ( ( OSD Memory Size in GB * Number of OSDs ) + ( Approximate Number of Guest Machines * Hypervisor Memory Overhead in GB ) )

Example

75000 = 1000 * ( ( 3 * 10 ) + ( 90 * .5 ) )

Thus, reserved_host_memory_mb would equal 75000. The parameter value must be in megabytes (MB).

Tuning CPU allocation ratio

Nova’s cpu_allocation_ratio parameter is used by the Nova scheduler when choosing which compute nodes to run the guest machines. If the ratio of guest machines to compute nodes is 16:1, and the number of cores (vCPUs) on a node is 56, then the Nova scheduler may schedule enough guests to consume 896 cores, before it considers the node is unable to handle any more guest machines. The reason is because, the Nova scheduler does not take into account the CPU needs of the Ceph OSD services running on the same node as the Nova scheduler. Modifying the cpu_allocation_ratio parameter allows Ceph to have the CPU resources it needs to operate effectively without those CPU resources being given to Nova Compute.

The following is an example of how to determine the cpu_allocation_ratio value for a hyper-converged node. Given a node has 56 cores and 10 OSDs, and assuming that one core is used by each OSD, that leaves 46 cores for Nova. If each guest machine utilizes 100% of its core, then the number of available cores for guest machines is divided by the total number of cores on the node. In this scenario, the cpu_allocation_ratio value is 0.821429.

However, because guest machines do not typically utilize 100% of their cores, the ratio must take into account an anticipated utilization percentage when determining the number of cores per guest machine. In a scenario, where you only anticipate on average, 10% core utilization per guest machine, the cpu_allocation_ratio value must be 8.214286.

Here is the mathematical formulas:

  1. Number of Non Ceph Cores = Total Number of Cores on the Node - ( Number of Cores per OSD * Number of OSDs)
  2. Number of Guest Machine vCPUs = Number of Non Ceph Cores / Average Guest Machine CPU Utilization
  3. CPU Allocation Ratio = Number of Guest Machine vCPUs / Total Number of Cores on the Node

Example

  1. 46 = 56 - ( 1 * 10 )
  2. 460 = 46 / .1
  3. 8.214286 = 460 / 56

Nova memory and CPU calculator

Red Hat provides a calculator script to do all these calculations for you. The script name is nova_mem_cpu_calc.py, and takes 5 input parameters: 

nova_mem_cpu_calc.py TOTAL_NODE_RAM_GB TOTAL_NODE_CORES NUM_OSDs_PER_NODE AVG_GUEST_MEM_SIZE_GB AVG_GUEST_CPU_UTIL
Replace…​
  • TOTAL_NODE_RAM_GB with the total size of RAM in GB on the node.
  • TOTAL_NODE_CORES with the total number of cores on the node.
  • NUM_OSDs_PER_NODE with the number of Ceph OSDs per node.
  • AVG_GUEST_MEM_SIZE_GB with the average memory size in GB for the guest machine.
  • AVG_GUEST_CPU_UTIL with the average CPU utilization, expressed as a decimal, for the guest machine.

Example

[stack@director ~]$ ./nova_mem_cpu_calc.py 256 56 10 2 0.1

Additional Resources

  • See the appendix for the full source code of the nova_mem_cpu_calc.py script.