第7章 Configuring instance scheduling and placement

The Compute scheduler service determines on which Compute node or host aggregate to place an instance. When the Compute (nova) service receives a request to launch or move an instance, it uses the specifications provided in the request, the flavor, and the image to find a suitable host. For example, a flavor can specify the traits an instance requires a host to have, such as the type of storage disk, or the Intel CPU instruction set extension.

The Compute scheduler service uses the configuration of the following components, in the following order, to determine on which Compute node to launch or move an instance:

  1. Placement service prefilters: The Compute scheduler service uses the Placement service to filter the set of candidate Compute nodes based on specific attributes. For example, the Placement service automatically excludes disabled Compute nodes.
  2. Filters: Used by the Compute scheduler service to determine the initial set of Compute nodes on which to launch an instance.
  3. Weights: The Compute scheduler service prioritizes the filtered Compute nodes using a weighting system. The highest weight has the highest priority.

In the following diagram, host 1 and 3 are eligible after filtering. Host 1 has the highest weight and therefore has the highest priority for scheduling.

Scheduling Hosts

7.1. Prefiltering using the Placement service

The Compute (nova) service interacts with the Placement service when it creates and manages instances. The Placement service tracks the inventory and usage of resource providers, such as a Compute node, a shared storage pool, or an IP allocation pool, and their available quantitative resources, such as the available vCPUs. Any service that needs to manage the selection and consumption of resources can use the Placement service.

The Placement service also tracks the mapping of available qualitative resources to resource providers, such as the type of storage disk trait a resource provider has.

The Placement service applies prefilters to the set of candidate Compute nodes based on Placement service resource provider inventories and traits. You can create prefilters based on the following criteria:

  • Supported image types
  • Traits
  • Projects or tenants
  • Availability zone

7.1.1. Filtering by requested image type support

You can exclude Compute nodes that do not support the disk format of the image used to launch an instance. This is useful when your environment uses Red Hat Ceph Storage as an ephemeral backend, which does not support QCOW2 images. Enabling this feature ensures that the scheduler does not send requests to launch instances using a QCOW2 image to Compute nodes backed by Red Hat Ceph Storage.

Procedure

  1. Open your Compute environment file.
  2. To exclude Compute nodes that do not support the disk format of the image used to launch an instance, set the NovaSchedulerQueryImageType parameter to True in the Compute environment file.
  3. Save the updates to your Compute environment file.
  4. Add your Compute environment file to the stack with your other environment files and deploy the overcloud:

    (undercloud)$ openstack overcloud deploy --templates \
     -e [your environment files] \
     -e /home/stack/templates/<compute_environment_file>.yaml

7.1.2. Filtering by resource provider traits

Each resource provider has a set of traits. Traits are the qualitative aspects of a resource provider, for example, the type of storage disk, or the Intel CPU instruction set extension.

The Compute node reports its capabilities to the Placement service as traits. An instance can specify which of these traits it requires, or which traits the resource provider must not have. The Compute scheduler can use these traits to identify a suitable Compute node or host aggregate to host an instance.

To enable your cloud users to create instances on hosts that have particular traits, you can define a flavor that requires or forbids a particular trait, and you can create an image that requires or forbids a particular trait.

For a list of the available traits, see the os-traits library. You can also create custom traits, as required.

7.1.2.1. Creating an image that requires or forbids a resource provider trait

You can create an instance image that your cloud users can use to launch instances on hosts that have particular traits.

Procedure

  1. Create a new image:

    (overcloud)$ openstack image create ... trait-image
  2. Identify the trait you require a host or host aggregate to have. You can select an existing trait, or create a new trait:

    • To use an existing trait, list the existing traits to retrieve the trait name:

      (overcloud)$ openstack --os-placement-api-version 1.6 trait list
    • To create a new trait, enter the following command:

      (overcloud)$ openstack --os-placement-api-version 1.6 trait \
       create CUSTOM_TRAIT_NAME

      Custom traits must begin with the prefix CUSTOM_ and contain only the letters A through Z, the numbers 0 through 9 and the underscore “_” character.

  3. To schedule instances on a host or host aggregate that has a required trait, add the trait to the image extra specs. For example, to schedule instances on a host or host aggregate that supports AVX-512, add the following trait to the image extra specs:

    (overcloud)$ openstack image set \
     --property trait:HW_CPU_X86_AVX512BW=required \
     trait-image
  4. To filter out hosts or host aggregates that have a forbidden trait, add the trait to the image extra specs. For example, to prevent instances from being scheduled on a host or host aggregate that supports multi-attach volumes, add the following trait to the image extra specs:

    (overcloud)$ openstack image set \
     --property trait:COMPUTE_VOLUME_MULTI_ATTACH=forbidden \
     trait-image

7.1.2.2. Creating a flavor that requires or forbids a resource provider trait

You can create flavors that your cloud users can use to launch instances on hosts that have particular traits.

Procedure

  1. Create a flavor:

    (overcloud)$ openstack flavor create --vcpus 1 --ram 512 \
     --disk 2 trait-flavor
  2. Identify the trait you require a host or host aggregate to have. You can select an existing trait, or create a new trait:

    • To use an existing trait, list the existing traits to retrieve the trait name:

      (overcloud)$ openstack --os-placement-api-version 1.6 trait list
    • To create a new trait, enter the following command:

      (overcloud)$ openstack --os-placement-api-version 1.6 trait \
       create CUSTOM_TRAIT_NAME

      Custom traits must begin with the prefix CUSTOM_ and contain only the letters A through Z, the numbers 0 through 9 and the underscore “_” character.

  3. To schedule instances on a host or host aggregate that has a required trait, add the trait to the flavor extra specs. For example, to schedule instances on a host or host aggregate that supports AVX-512, add the following trait to the flavor extra specs:

    (overcloud)$ openstack flavor set \
     --property trait:HW_CPU_X86_AVX512BW=required \
     trait-flavor
  4. To filter out hosts or host aggregates that have a forbidden trait, add the trait to the flavor extra specs. For example, to prevent instances from being scheduled on a host or host aggregate that supports multi-attach volumes, add the following trait to the flavor extra specs:

    (overcloud)$ openstack flavor set \
     --property trait:COMPUTE_VOLUME_MULTI_ATTACH=forbidden \
     trait-flavor

7.1.3. Filtering by isolating host aggregates

You can restrict scheduling on a host aggregate to only those instances whose flavor and image traits match the metadata of the host aggregate. The combination of flavor and image metadata must require all the host aggregate traits to be eligible for scheduling on Compute nodes in that host aggregate.

Procedure

  1. Open your Compute environment file.
  2. To isolate host aggregates to host only instances whose flavor and image traits match the aggregate metadata, set the NovaSchedulerEnableIsolatedAggregateFiltering parameter to True in the Compute environment file.
  3. Save the updates to your Compute environment file.
  4. Add your Compute environment file to the stack with your other environment files and deploy the overcloud:

    (undercloud)$ openstack overcloud deploy --templates \
     -e [your environment files] \
     -e /home/stack/templates/<compute_environment_file>.yaml
  5. Identify the traits you want to isolate the host aggregate for. You can select an existing trait, or create a new trait:

    • To use an existing trait, list the existing traits to retrieve the trait name:

      (overcloud)$ openstack --os-placement-api-version 1.6 trait list
    • To create a new trait, enter the following command:

      (overcloud)$ openstack --os-placement-api-version 1.6 trait \
       create CUSTOM_TRAIT_NAME

      Custom traits must begin with the prefix CUSTOM_ and contain only the letters A through Z, the numbers 0 through 9 and the underscore “_” character.

  6. Collect the existing traits of each Compute node:

    (overcloud)$ traits=$(openstack --os-placement-api-version 1.6 resource provider trait list -f value <host_uuid> | sed 's/^/--trait /')
  7. Add the traits to the resource providers for each Compute node in the host aggregate:

    (overcloud)$ openstack --os-placement-api-version 1.6 \
     resource provider trait set --trait $traits \
     --trait CUSTOM_TRAIT_NAME \
     <host_uuid>
  8. Repeat steps 6 and 7 for each Compute node in the host aggregate.
  9. Add the metadata property for the trait to the host aggregate:

    (overcloud)$ openstack --os-compute-api-version 2.53 aggregate set \
     --property trait:TRAIT_NAME=required <aggregate_name>
  10. Add the trait to a flavor or an image:

    (overcloud)$ openstack flavor set \
     --property trait:<TRAIT_NAME>=required <flavor>
    (overcloud)$ openstack image set \
     --property trait:<TRAIT_NAME>=required <image>

7.1.4. Filtering by availability zone using the Placement service

You can use the Placement service to honor availability zone requests. To use the Placement service to filter by availability zone, placement aggregates must exist that match the membership and UUID of the availability zone host aggregates.

When using the Placement service to filter by availability zone, you can remove the AvailabilityZoneFilter filter from NovaSchedulerDefaultFilters.

Procedure

  1. Open your Compute environment file.
  2. To use the Placement service to filter by availability zone, set the NovaSchedulerQueryPlacementForAvailabilityZone parameter to True in the Compute environment file.
  3. Save the updates to your Compute environment file.
  4. Add your Compute environment file to the stack with your other environment files and deploy the overcloud:

    (undercloud)$ openstack overcloud deploy --templates \
     -e [your environment files] \
     -e /home/stack/templates/<compute_environment_file>.yaml
  5. Create the host aggregate to use as an availability zone.
  6. Add the availability zone host aggregate as a resource provider to the placement aggregate:

    $ openstack --os-placement-api-version=1.2 resource provider \
     aggregate set --aggregate <az_agg_uuid> <resource_provider_uuid>

Additional resources