Chapter 7. Configuring Compute nodes for performance

Procedure

1. Reserve CPU cores for the VMs by setting the NovaVcpuPinSet configuration in the Compute environment file:

   NovaVcpuPinSet: 2,3,6,7

2. Set the NovaReservedHostMemory option in the same file to the amount of RAM to reserve for host processes. For example, if you want to reserve 512 MB, use:

   NovaReservedHostMemory: 512

3. To ensure that host processes do not run on the CPU cores reserved for VMs, set the parameter IsolCpusList in the Compute environment file to the CPU cores you have reserved for VMs. Specify the value of the IsolCpusList parameter using a list of CPU indices, or ranges separated by a whitespace. For example:

   IsolCpusList: 2 3 6 7

   Note

   The IsolCpusList parameter ensures that the underlying compute node is not able to use the corresponding pCPUs for itself. The pCPUs are dedicated to the VMs.

4. To apply this configuration, deploy the overcloud:

   (undercloud) $ openstack overcloud deploy --templates \
     -e /home/stack/templates/<compute_environment_file>.yaml

Procedure

1. Open your Compute environment file.

2. Add the following values to the NovaSchedulerDefaultFilters parameter, if they are not already present:

   • NUMATopologyFilter
   • AggregateInstanceExtraSpecsFilter
3. Save the configuration file.
4. Deploy the overcloud.

Procedure

1. Load the admin credentials:

   source ~/keystonerc_admin

2. Create an aggregate for the hosts that will receive pinning requests:

   nova aggregate-create <aggregate-name-pinned>

3. Enable the pinning by editing the metadata for the aggregate:

   nova aggregate-set-metadata <aggregate-pinned-UUID> pinned=true

4. Create an aggregate for other hosts:

   nova aggregate-create <aggregate-name-unpinned>

5. Edit the metadata for this aggregate accordingly:

   nova aggregate-set-metadata <aggregate-unpinned-UUID> pinned=false

6. Change your existing flavors' specifications to this one:

   for i in $(nova flavor-list | cut -f 2 -d ' ' | grep -o '[0-9]*'); do nova flavor-key $i set "aggregate_instance_extra_specs:pinned"="false"; done

7. Create a flavor for the hosts that will receive pinning requests:

   nova flavor-create <flavor-name-pinned> <flavor-ID> <RAM> <disk-size> <vCPUs>

   Where:

   • <flavor-ID> - Set to auto if you want nova to generate a UUID.
   • <RAM> - Specify the required RAM in MB.
   • <disk-size> - Specify the required disk size in GB.
   • <vCPUs> - The number of virtual CPUs that you want to reserve.

8. Set the hw:cpu_policy specification of this flavor to dedicated so as to require dedicated resources, which enables CPU pinning, and also the hw:cpu_thread_policy specification to require, which places each vCPU on thread siblings:

   nova flavor-key <flavor-name-pinned> set hw:cpu_policy=dedicated
   nova flavor-key <flavor-name-pinned> set hw:cpu_thread_policy=require

   Note

   If the host does not have an SMT architecture or enough CPU cores with free thread siblings, scheduling will fail. If such behavior is undesired, or if your hosts simply do not have an SMT architecture, do not use the hw:cpu_thread_policy specification, or set it to prefer instead of require. The (default) prefer policy ensures that thread siblings are used when available.

9. Set the aggregate_instance_extra_specs:pinned specification to "true" to ensure that instances based on this flavor have this specification in their aggregate metadata:

   nova flavor-key <flavor-name-pinned> set aggregate_instance_extra_specs:pinned=true

10. Add some hosts to the new aggregates:

    nova aggregate-add-host <aggregate-pinned-UUID> <host_name>
    nova aggregate-add-host <aggregate-unpinned-UUID> <host_name>

11. Boot an instance using the new flavor:

    nova boot --image <image-name> --flavor <flavor-name-pinned> <server-name>

12. To verify that the new server has been placed correctly, run the following command and check for OS-EXT-SRV-ATTR:hypervisor_hostname in the output:

    nova show <server-name>

Procedure

1. Create a flavor for instances that require huge pages:

   $ openstack flavor create --ram <size-mb> --disk <size-gb> --vcpus <no_reserved_vcpus> huge_pages

2. Set the flavor for huge pages:

   $ openstack flavor set huge_pages --property hw:mem_page_size=1GB

   Valid values for hw:mem_page_size:

   • large - Selects the largest page size supported on the host, which may be 2 MB or 1 GB on x86_64 systems.
   • small - (Default) Selects the smallest page size supported on the host. On x86_64 systems this is 4 kB (normal pages).
   • any - Selects the largest available huge page size, as determined by the libvirt driver.
   • <pagesize>: (string) Set an explicit page size if the workload has specific requirements. Use an integer value for the page size in KB, or any standard suffix. For example: 4KB, 2MB, 2048, 1GB.

3. Create an instance using the new flavor:

   $ openstack server create --flavor huge_pages --image <image> huge_pages_instance