Procedure

1. Show the sites on which a copy of the image exists:

   $ glance image-show ID | grep "stores"
   
   | stores |  default_backend,dcn1,dcn2

   In the example, the image is present on the central site, the default_backend, and on the two edge sites dcn1 and dcn2.

2. Alternatively, you can run the glance image-list command with the --include-stores option to see the sites where the images exist:

   $ glance image-list --include-stores
   
   | ID                                   | Name    | Stores
   
   | 2bd882e7-1da0-4078-97fe-f1bb81f61b00 | cirros | default_backend,dcn1,dcn2

3. List the image locations properties to show the details of each location:

   $ openstack image show ID -c properties
   
   | properties |
   
   (--- cut ---)
   locations='[{'url': 'rbd://79b70c32-df46-4741-93c0-8118ae2ae284/images/2bd882e7-1da0-4078-97fe-f1bb81f61b00/snap', 'metadata': {'store': 'default_backend'}}, {'url': 'rbd://63df2767-8ddb-4e06-8186-8c155334f487/images/2bd882e7-1da0-4078-97fe-f1bb81f61b00/snap', 'metadata': {'store': 'dcn1'}}, {'url': 'rbd://1b324138-2ef9-4ef9-bd9e-aa7e6d6ead78/images/2bd882e7-1da0-4078-97fe-f1bb81f61b00/snap', 'metadata': {'store': 'dcn2'}}]',
   (--- cut --)

   The image properties show the different Ceph RBD URIs for the location of each image.

   In the example, the central image location URI is:

   rbd://79b70c32-df46-4741-93c0-8118ae2ae284/images/2bd882e7-1da0-4078-97fe-f1bb81f61b00/snap', 'metadata': {'store': 'default_backend'}}

   The URI is composed of the following data:

   • 79b70c32-df46-4741-93c0-8118ae2ae284 corresponds to the central Ceph FSID. Each Ceph cluster has a unique FSID.
   • The default value for all sites is images, which corresponds to the Ceph pool on which the images are stored.
   • 2bd882e7-1da0-4078-97fe-f1bb81f61b00 corresponds to the image UUID. The UUID is the same for a given image regardless of its location.
   • The metadata shows the glance store to which this location maps. In this example, it maps to the default_backend, which is the central hub site.

Procedure

1. Copy the /usr/share/openstack-tripleo-heat-templates/environments/enable-swap.yaml file to your environment file directory:

   $ cp /usr/share/openstack-tripleo-heat-templates/environments/enable-swap.yaml /home/stack/templates/enable-swap.yaml

2. Configure the swap size by adding the following parameters to your enable-swap.yaml file:

   parameter_defaults:
     swap_size_megabytes: <swap size in MB>
     swap_path: <full path to location of swap, default: /swap>

3. To apply this configuration, add the enable_swap.yaml environment file to the stack with your other environment files and deploy the overcloud:

   (undercloud) $ openstack overcloud deploy --templates \
     -e [your environment files] \
     -e /home/stack/templates/enable-swap.yaml \

1. Extracting parameter information from the default first cell in the basic overcloud. This cell becomes the global Controller after you redeploy the overcloud.
2. Configuring a custom role and flavor for the cell.
3. Creating an environment file with cell-specific parameters.
4. Redeploying the overcloud with the new cell stack.

Extract parameter information from the overcloud

1. Create a new directory for the new cell and export the contents to the new directory. For example:

   $ source ~/stackrc
   (undercloud) $ mkdir cell1
   (undercloud) $ export DIR=cell1

2. Export the EndpointMap, HostsEntry, AllNodesConfig, GlobalConfig parameters, and the password information from the overcloud to a new environment file for the cell. For example:

   (undercloud) $ openstack overcloud cell export cell1 -o cell1/cell1-ctrl-input.yaml

   Note

   If the environment file already exists, run the command with the --force-overwrite or -f option.

Configure a custom role for a cell

1. Add the CellController role to your roles data file and regenerate the file. For example:

   (undercloud) $ openstack overcloud roles generate --roles-path \
                  /usr/share/openstack-tripleo-heat-templates/roles \
                  -o $DIR/cell_roles_data.yaml Compute CellController

   The CellController custom role includes the services from the default Compute role and additional configuration for the following services:

   • Galera database
   • RabbitMQ
   • nova-conductor
   • nova novnc proxy
   • nova metadata (only in case you set the NovaLocalMetadataPerCell parameter)

2. In case you want to divide your network between the global Controller and the cells, configure network access in the roles file that you created. For example:

   name: Compute
    description: |
      Basic Compute Node role
    CountDefault: 1
    # Create external Neutron bridge (unset if using ML2/OVS without DVR)
    tags:
      - external_bridge
    networks:
      InternalApi:
        subnet: internal_api_cell1
      Tenant:
        subnet: tenant_subnet
      Storage:
        subnet: storage_cell1
   ...
   - name: CellController
      description: |
        CellController role for the nova cell_v2 controller services
      CountDefault: 1
      tags:
        - primary
        - controller
      networks:
        External:
          subnet: external_cell1
        InternalApi:
          subnet: internal_api_cell1
        Storage:
          subnet: storage_cell1
        StorageMgmt:
          subnet: storage_mgmt_cell1
        Tenant:
          subnet: tenant_subnet

Configure a flavor and tag nodes to a cell

1. Create the cellcontroller flavor to tag nodes that you want to allocate to the cell. For example:

   (undercloud) $ openstack flavor create --id auto --ram 4096 --disk 40 --vcpus 1 cellcontroller
   (undercloud) $ openstack flavor set --property "cpu_arch"="x86_64" \
                   --property "capabilities:boot_option"="local" \
                   --property "capabilities:profile"="cellcontroller" \
                   --property "resources:CUSTOM_BAREMETAL=1" \
                   --property "resources:DISK_GB=0" \
                   --property "resources:MEMORY_MB=0" \
                   --property "resources:VCPU=0" \
                  cellcontroller

2. Tag each node that you want to assign to the cell with the cellcontroller profile.

   (undercloud) $ openstack baremetal node set --property \
                  capabilities='profile:cellcontroller,boot_option:local' <NODE_UUID>

   Replace <NODE_UUID> with the actual ID of the Compute node that you want to assign to the cell.

Create environment files with cell parameters

1. Create a new environment file in the directory for the cell, such as /cell1/cell1.yaml, and add the following parameters:

   resource_registry:
     # since the same networks are used in this example, the
     # creation of the different networks is omitted
     OS::TripleO::Network::External: OS::Heat::None
     OS::TripleO::Network::InternalApi: OS::Heat::None
     OS::TripleO::Network::Storage: OS::Heat::None
     OS::TripleO::Network::StorageMgmt: OS::Heat::None
     OS::TripleO::Network::Tenant: OS::Heat::None
     OS::TripleO::Network::Management: OS::Heat::None
     OS::TripleO::Network::Ports::OVNDBsVipPort: /usr/share/openstack-tripleo-heat-templates/network/ports/noop.yaml
     OS::TripleO::Network::Ports::RedisVipPort: /usr/share/openstack-tripleo-heat-templates/network/ports/noop.yaml
   
   parameter_defaults:
     # CELL Parameter to reflect that this is an additional CELL
     NovaAdditionalCell: True
   
     # mapping of the CellController flavor to the CellController role
     CellControllerFlavor: cellcontroller
   
     # The DNS names for the VIPs for the cell
     CloudName: cell1.ooo.test
     CloudNameInternal: cell1.internalapi.ooo.test
     CloudNameStorage: cell1.storage.ooo.test
     CloudNameStorageManagement: cell1.storagemgmt.ooo.test
     CloudNameCtlplane: cell1.ctlplane.ooo.test
   
     # Flavors used for the cell controller and computes
     OvercloudCellControllerFlavor: cellcontroller
     OvercloudComputeFlavor: compute
   
     # Number of controllers/computes in the cell
     CellControllerCount: 3
     ComputeCount: 1
   
     # Compute node name (must be unique)
     ComputeHostnameFormat: 'cell1-compute-%index%'
   
     # default gateway
     ControlPlaneStaticRoutes:
       - ip_netmask: 0.0.0.0/0
         next_hop: 192.168.24.1
         default: true
     DnsServers:
       - x.x.x.x

   Change the parameter values in this example according to your deployment needs.

2. Depending on your network configuration, you might need to allocate a network resource to the cell. Add the following parameter if you need to register cells to the network:

   resource_registry:
     OS::TripleO::CellController::Net::SoftwareConfig: single-nic-vlans/controller.yaml
     OS::TripleO::Compute::Net::SoftwareConfig: single-nic-vlans/compute.yaml

3. If you divide your network between the global Controller and the cells and want to run the Compute metadata API in each cell instead of in the global Controller, add the following parameter:

   parameter_defaults:
      NovaLocalMetadataPerCell: True

   Note

   • The parameters in this file restrict the overcloud to use a single network for all cells.
   • The Compute host names must be unique across all cells.

4. Copy the network_data.yaml file and name it according to the cell name. For example:

   (undercloud) $ cp /usr/share/openstack-tripleo-heat-templates/network_data.yaml cell1/network_data-ctrl.yaml

5. Add the UUIDs for the network components you want to reuse for the cells to the new network data file.

   external_resource_network_id: [EXISTING_NETWORK_UUID]
   external_resource_subnet_id: [EXISTING_SUBNET_UUID]
   external_resource_segment_id: [EXISTING_SEGMENT_UUID]
   external_resource_vip_id: [EXISTING_VIP_UUID]

1. Get the IP addresses of the control plane and cell controller.

   $ CTRL_IP=$(openstack server list -f value -c Networks --name overcloud-controller-0 | sed 's/ctlplane=//')
   $ CELL_CTRL_IP=$(openstack server list -f value -c Networks --name cellcontroller-0 | sed 's/ctlplane=//')

2. Add the cell information to all Controller nodes. This information is used to connect to the cell endpoint from the undercloud.

   (undercloud) [stack@undercloud ~]$ CELL_INTERNALAPI_INFO=$(ssh heat-admin@${CELL_CTRL_IP} egrep \
                                       cellcontrol.*\.internalapi /etc/hosts)
   (undercloud) [stack@undercloud ~]$ ansible -i /usr/bin/tripleo-ansible-inventory Controller -b \
                                       -m lineinfile -a "dest=/etc/hosts line=\"$CELL_INTERNALAPI_INFO\""

3. Get the transport_url and database.connection endpoint information from the controller cell.

   (undercloud) [stack@undercloud ~]$ CELL_TRANSPORT_URL=$(ssh heat-admin@${CELL_CTRL_IP} sudo \
                                       crudini --get /var/lib/config-data/nova/etc/nova/nova.conf DEFAULT transport_url)
   (undercloud) [stack@undercloud ~]$ CELL_MYSQL_VIP=$(ssh heat-admin@${CELL_CTRL_IP} sudo \
                                       crudini --get /var/lib/config-data/nova/etc/nova/nova.conf database connection \
                                       | perl -nle'/(\d+\.\d+\.\d+\.\d+)/ && print $1')

4. Log in to one of the global Controller nodes to create the cell based on the information that you retrieved in the previous steps. For example:

   $ export CONTAINERCLI='podman'
   
   $ ssh heat-admin@${CTRL_IP} sudo ${CONTAINERCLI} exec -i -u root nova_api \
     nova-manage cell_v2 create_cell --name computecell1 \
     --database_connection "{scheme}://{username}:{password}@$CELL_MYSQL_VIP/nova?{query}" \
     --transport-url "$CELL_TRANSPORT_URL"

5. Check that the cell is created and appears in the cell list.

   $ ssh heat-admin@${CTRL_IP} sudo ${CONTAINERCLI} exec -i -u root nova_api \
   nova-manage cell_v2 list_cells --verbose

6. Restart the Compute services on the Controller nodes.

   $ ansible -i /usr/bin/tripleo-ansible-inventory Controller -b -a \
   "systemctl restart tripleo_nova_api tripleo_nova_conductor tripleo_nova_scheduler"

7. Check that the cell controller services are provisioned.

   (overcloud) [stack@undercloud ~]$ nova service-list

1. Log into one of the Controller nodes.

2. Get the IP address of the control plane for the cell and run the host discovery command to expose and assign Compute hosts to the cell.

   $ CTRL=overcloud-controller-0
   $ CTRL_IP=$(openstack server list -f value -c Networks --name $CTRL | sed 's/ctlplane=//')
   
   $ export CONTAINERCLI='podman'
   
   $ ssh heat-admin@${CTRL_IP} sudo ${CONTAINERCLI} exec -i -u root nova_api \
     nova-manage cell_v2 discover_hosts --by-service --verbose

3. Verify that the Compute hosts were assigned to the cell.

   $ ssh heat-admin@${CTRL_IP} sudo ${CONTAINERCLI} exec -i -u root nova_api \
     nova-manage cell_v2 list_hosts

1. Delete all instances from the Compute nodes in the cell.

   Note

   Migrating instances between cells is not supported. You must delete the instances and re-create them in another cell.

2. On one of the global Controllers, delete all Compute nodes from the cell.

   $ CTRL=overcloud-controller-0
   $ CTRL_IP=$(openstack server list -f value -c Networks --name $CTRL | sed 's/ctlplane=//')
   
   $ export CONTAINERCLI='podman'
   
   $ ssh heat-admin@${CTRL_IP} sudo ${CONTAINERCLI} exec -i -u root nova_api \
     nova-manage cell_v2 list_hosts
   
   $ ssh heat-admin@${CTRL_IP} sudo ${CONTAINERCLI} exec -i -u root nova_api \
     nova-manage cell_v2 delete_host --cell_uuid <uuid> --host <compute>

3. Delete the resource providers for the cell from the Placement service, to ensure that the host name is available in case you want to add Compute nodes with the same host name to another cell later. For example:

   (undercloud) $ source ~/overcloudrc
   
   (overcloud) $ openstack resource provider list
               +--------------------------------------+---------------------------------------+------------+
               | uuid                                 | name                                  | generation |
               +--------------------------------------+---------------------------------------+------------+
               | 9cd04a8b-5e6c-428e-a643-397c9bebcc16 | computecell1-novacompute-0.site1.test |         11 |
               +--------------------------------------+---------------------------------------+------------+
   
   (overcloud) $ openstack resource provider delete 9cd04a8b-5e6c-428e-a643-397c9bebcc16

1. On one of the global Controllers, delete the cell.

   $ CTRL=overcloud-controller-0
   $ CTRL_IP=$(openstack server list -f value -c Networks --name $CTRL | sed 's/ctlplane=//')
   
   $ export CONTAINERCLI='podman'
   
   $ ssh heat-admin@${CTRL_IP} sudo ${CONTAINERCLI} exec -i -u root nova_api \
     nova-manage cell_v2 list_cells
   
   $ ssh heat-admin@${CTRL_IP} sudo ${CONTAINERCLI} exec -i -u root nova_api \
     nova-manage cell_v2 delete_cell --cell_uuid <uuid>

2. Delete the cell stack from the overcloud.

   $ openstack stack delete <stack name> --wait --yes && openstack overcloud plan delete <STACK_NAME>

   Note

   If you deployed separate cell stacks for a Controller and Compute cell, delete the Compute cell stack first and then the Controller cell stack.

Procedure

1. Open your Compute environment file.

2. Add the following values to the NovaSchedulerDefaultFilters parameter, if they are not already present:

   • AggregateInstanceExtraSpecsFilter: Add this value to filter Compute nodes by host aggregate metadata that match flavor extra specs.

     Note

     For this filter to perform as expected, you must scope the flavor extra specs by prefixing the extra_specs key with the aggregate_instance_extra_specs: namespace.

   • AggregateImagePropertiesIsolation: Add this value to filter Compute nodes by host aggregate metadata that match image metadata properties.

     Note

     To filter host aggregate metadata using image metadata properties, the host aggregate metadata key must match a valid image metadata property. For details on valid image metadata properties, see Appendix A. Image Configuration Parameters.

   • AvailabilityZoneFilter: Add this value to filter by availability zone when launching an instance.

     Note

     Instead of using the AvailabilityZoneFilter Compute scheduler service filter, you can use the Placement service to process availability zone requests.

3. Save the updates to your Compute environment file.

4. Deploy the overcloud, adding your Compute environment file to the stack along with your other environment files:

   (undercloud)$ openstack overcloud deploy --templates \
     -e [your environment files] \
     -e /home/stack/templates/<compute_environment_file>.yaml

Procedure

1. To create a host aggregate, enter the following command:

   (overcloud)# openstack aggregate create <aggregate_name>

   Replace <aggregate_name> with the name you want to assign to the host aggregate.

2. Add metadata to the host aggregate:

   (overcloud)# openstack aggregate set --property <key=value> \
     --property <key=value> \
     <aggregate_name>

   • Replace <key=value> with the metadata key-value pair. If you are using the AggregateInstanceExtraSpecsFilter filter, the key can be any arbitrary string, for example, ssd=true. If you are using the AggregateImagePropertiesIsolation filter, the key must match a valid image metadata property. For more information on valid image metadata properties, see Appendix A. Image Configuration Parameters.
   • Replace <aggregate_name> with the name of the host aggregate.

3. Add the Compute nodes to the host aggregate:

   (overcloud)# openstack aggregate add host <aggregate_name> \
     <host_name>

   • Replace <aggregate_name> with the name of the host aggregate to add the Compute node to.
   • Replace <host_name> with the name of the Compute node to add to the host aggregate.

4. Create a flavor or image for the host aggregate:

   • Create a flavor:

     (overcloud)$ openstack flavor create \
       --ram <size-mb> \
       --disk <size-gb> \
       --vcpus <no_reserved_vcpus> \
       host-agg-flavor

   • Create an image:

     (overcloud)$ openstack image create host-agg-image

     For information on how to create an image, see Creating an image.

5. Set one or more key-value pairs on the flavor or image that match the key-value pairs on the host aggregate.

   • To set the key-value pairs on a flavor, use the scope aggregate_instance_extra_specs:

     (overcloud)# openstack flavor set \
       --property aggregate_instance_extra_specs:ssd=true \
       host-agg-flavor

   • To set the key-value pairs on an image, use valid image metadata properties as the key:

     (overcloud)# openstack image set --property os_type=linux \
       host-agg-image

Procedure

1. To create an availability zone, you can create a new availability zone host aggregate, or make an existing host aggregate an availability zone:

   1. To create a new availability zone host aggregate, enter the following command:

      (overcloud)# openstack aggregate create --zone <availability_zone> \
        <aggregate_name>

      • Replace <availability_zone> with the name you want to assign to the availability zone.
      • Replace <aggregate_name> with the name you want to assign to the host aggregate.

   2. To make an existing host aggregate an availability zone, enter the following command:

      (overcloud)# openstack aggregate set --zone <availability_zone> \
        <aggregate_name>

      • Replace <availability_zone> with the name you want to assign to the availability zone.
      • Replace <aggregate_name> with the name of the host aggregate.

2. Optional: Add metadata to the availability zone:

   (overcloud)# openstack aggregate set --property <key=value> \
     <aggregate_name>

   • Replace <key=value> with your metadata key-value pair. You can add as many key-value properties as required.
   • Replace <aggregate_name> with the name of the availability zone host aggregate.

3. Add Compute nodes to the availability zone host aggregate:

   (overcloud)# openstack aggregate add host <aggregate_name> \
     <host_name>

   • Replace <aggregate_name> with the name of the availability zone host aggregate to add the Compute node to.
   • Replace <host_name> with the name of the Compute node to add to the availability zone.

Procedure

1. To view a list of all the Compute nodes assigned to the host aggregate, enter the following command:

   (overcloud)# openstack aggregate show <aggregate_name>

2. To remove all assigned Compute nodes from the host aggregate, run the following command for each Compute node:

   (overcloud)# openstack aggregate remove host <aggregate_name> \
     <host_name>

   • Replace <aggregate_name> with the name of the host aggregate to remove the Compute node from.
   • Replace <host_name> with the name of the Compute node to remove from the host aggregate.

3. After you remove all the Compute nodes from the host aggregate, enter the following command to delete the host aggregate:

   (overcloud)# openstack aggregate delete <aggregate_name>

Procedure

1. Open your Compute environment file.
2. To schedule the tenant instances on the tenant-isolated host aggregate, set the NovaSchedulerLimitTenantsToPlacementAggregate parameter to True in the Compute environment file.

3. Optional: To ensure that only the tenants that you assign to a host aggregate can create instances on your cloud, set the NovaSchedulerPlacementAggregateRequiredForTenants parameter to True.

   Note

   NovaSchedulerPlacementAggregateRequiredForTenants is False by default. When this parameter is False, tenants that are not assigned to a host aggregate can create instances on any host aggregate.

4. Save the updates to your Compute environment file.

5. Deploy the overcloud, adding your Compute environment file to the stack along with your other environment files:

   (undercloud)$ openstack overcloud deploy --templates \
     -e [your environment files] \
     -e /home/stack/templates/<compute_environment_file>.yaml \

6. Create the host aggregate. For more information, see Creating a host aggregate.

7. Retrieve the list of tenant IDs:

   (overcloud)# openstack project list

8. Use the filter_tenant_id<suffix> metadata key to assign tenants to the host aggregate:

   (overcloud)# openstack aggregate set \
     --property filter_tenant_id<ID0>=<tenant_id0> \
     --property filter_tenant_id<ID1>=<tenant_id1> \
     ...
     --property filter_tenant_id<IDn>=<tenant_idn> \
     <aggregate_name>

   • Replace <ID0>, <ID1>, and all IDs up to <IDn> with unique values for each tenant filter that you want to create.
   • Replace <tenant_id0>, <tenant_id1>, and all tenant IDs up to <tenant_idn> with the ID of each tenant that you want to assign to the host aggregate.

   • Replace <aggregate_name> with the name of the tenant-isolated host aggregate.

     For example, use the following syntax to assign tenants 78f1, 9d3t, and aa29 to the host aggregate tenant-isolated-aggregate:

     (overcloud) # openstack aggregate set \
       --property filter_tenant_id0=78f1 \
       --property filter_tenant_id1=9d3t \
       --property filter_tenant_id2=aa29 \
       tenant-isolated-aggregate

     Tip

     You can create a host aggregate that is available only to a single specific tenant by omitting the suffix from the filter_tenant_id metadata key:

     (overcloud) # openstack aggregate set \
       --property filter_tenant_id=78f1 \
       single-tenant-isolated-aggregate

1. Cache the Image service’s backing image as the libvirt base.
2. Convert the base image to the raw format (if configured).
3. Resize the base image to match the VM’s flavor specifications.
4. Use the base image to create the libvirt instance disk.

Procedure

1. Reserve physical CPU cores for the dedicated instances by setting the NovaComputeCpuDedicatedSet configuration in the Compute environment file for each Compute node:

   NovaComputeCpuDedicatedSet: 1,3,5,7

2. Reserve physical CPU cores for the shared instances by setting the NovaComputeCpuSharedSet configuration in the Compute environment file for each Compute node:

   NovaComputeCpuSharedSet: 2,6

3. Set the NovaReservedHostMemory option in the same files to the amount of RAM to reserve for host processes. For example, if you want to reserve 512 MB, use:

   NovaReservedHostMemory: 512

4. To ensure that host processes do not run on the CPU cores reserved for instances, set the parameter IsolCpusList in each Compute environment file to the CPU cores you have reserved for instances. Specify the value of the IsolCpusList parameter using a list, or ranges, of CPU indices separated by a whitespace.

   IsolCpusList: 1 2 3 5 6 7

5. To filter out hosts based on its NUMA topology, add NUMATopologyFilter to the NovaSchedulerDefaultFilters parameter in each Compute environment file.

6. To apply this configuration, add the environment file(s) to your deployment command and deploy the overcloud:

   (undercloud) $ openstack overcloud deploy --templates \
     -e [your environment files]
     -e /home/stack/templates/<compute_environment_file>.yaml

Procedure

1. Configure the amount of huge page memory to reserve on each NUMA node for processes that are not instances:

   parameter_defaults:
     NovaReservedHugePages: ["node:0,size:2048,count:64","node:1,size:1GB,count:1"]

   Where:

   Attribute	Description
   size	The size of the allocated huge page. Valid values: * 2048 (for 2MB) * 1GB
   count	The number of huge pages used by OVS per NUMA node. For example, for 4096 of socket memory used by Open vSwitch, set this to 2.

2. (Optional) To allow instances to allocate 1GB huge pages, configure the CPU feature flags, cpu_model_extra_flags, to include "pdpe1gb":

   parameter_defaults:
      ComputeExtraConfig:
        nova::compute::libvirt::libvirt_cpu_mode: 'custom'
        nova::compute::libvirt::libvirt_cpu_model: 'Haswell-noTSX'
        nova::compute::libvirt::libvirt_cpu_model_extra_flags: 'vmx, pdpe1gb'

   Note

   • CPU feature flags do not need to be configured to allow instances to only request 2 MB huge pages.
   • You can only allocate 1G huge pages to an instance if the host supports 1G huge page allocation.
   • You only need to set cpu_model_extra_flags to pdpe1gb when cpu_mode is set to host-model or custom.
   • If the host supports pdpe1gb, and host-passthrough is used as the cpu_mode, then you do not need to set pdpe1gb as a cpu_model_extra_flags. The pdpe1gb flag is only included in Opteron_G4 and Opteron_G5 CPU models, it is not included in any of the Intel CPU models supported by QEMU.
   • To mitigate for CPU hardware issues, such as Microarchitectural Data Sampling (MDS), you might need to configure other CPU flags. For more information, see RHOS Mitigation for MDS ("Microarchitectural Data Sampling") Security Flaws.

3. To avoid loss of performance after applying Meltdown protection, configure the CPU feature flags, cpu_model_extra_flags, to include "+pcid":

   parameter_defaults:
      ComputeExtraConfig:
        nova::compute::libvirt::libvirt_cpu_mode: 'custom'
        nova::compute::libvirt::libvirt_cpu_model: 'Haswell-noTSX'
        nova::compute::libvirt::libvirt_cpu_model_extra_flags: 'vmx, pdpe1gb, +pcid'

   Tip

   For more information, see Reducing the performance impact of Meltdown CVE fixes for OpenStack guests with "PCID" CPU feature flag.

4. Add NUMATopologyFilter to the NovaSchedulerDefaultFilters parameter in each Compute environment file, if not already present.

5. Apply this huge page configuration by adding the environment file(s) to your deployment command and deploying the overcloud:

   (undercloud) $ openstack overcloud deploy --templates \
     -e [your environment files]
     -e /home/stack/templates/<compute_environment_file>.yaml

Procedure

1. Open your Compute environment file.

2. Configure the amount of host disk space, in MiB, to make available for instance RAM, by adding the following parameter to your Compute environment file:

   parameter_defaults:
     NovaLibvirtFileBackedMemory: 102400

3. Optional: To configure the directory to store the memory backing files, set the QemuMemoryBackingDir parameter in your Compute environment file. If not set, the memory backing directory defaults to /var/lib/libvirt/qemu/ram/.

   Note

   You must locate your backing store in a directory at or above the default directory location, /var/lib/libvirt/qemu/ram/.

   You can also change the host disk for the backing store. For more information, see Section 8.3.1, “Changing the memory backing directory host disk”.

4. Save the updates to your Compute environment file.

5. To apply this configuration, add your Compute environment file to the stack with your other environment files and deploy the overcloud:

   (undercloud) $ openstack overcloud deploy --templates \
     -e [your environment files] \
     -e /home/stack/templates/<compute_environment_file>.yaml

Procedure

1. Open your Compute environment file.

2. Optional: Add the following configuration to your Compute environment file to specify the maximum number of memory-encrypted instances the SEV-capable Compute node can host concurrently:

   parameter_defaults:
       ComputeExtraConfig:
           nova::config::nova_config:
               libvirt/num_memory_encrypted_guests:
                   value: 15

   Note

   If not set, libvirt/num_memory_encrypted_guests defaults to none, which means the SEV-capable Compute node does not impose a limit on the number of memory-encrypted instances that can be hosted concurrently. Instead, the hardware determines the maximum number of memory-encrypted instances the SEV-capable Compute node can host concurrently, which might cause some memory-encrypted instances to fail to launch.

3. Optional: To specify that all x86_64 images use the q35 machine type by default, add the NovaHWMachineType parameter to the Compute environment file, and set it to x86_64=q35.

   This configuration removes the need to set the hw_machine_type property to q35 on every SEV-enabled instance image.

4. To prevent memory overcommit, set the NovaRAMAllocationRatio parameter to 1.0 in the Compute environment file.
5. To ensure that the SEV-capable Compute nodes reserve enough memory for host-level services to function, add 16MB for each potential SEV instance (the maximum number of concurrent SEV instances), to your value for NovaReservedHostMemory in the Compute environment file.

6. Add the following configuration to your Compute environment file to schedule memory-encrypted instances on a SEV-capable Compute node aggregate:

   parameter_defaults:
       ControllerExtraConfig:
           nova::config::nova_config:
               scheduler/enable_isolated_aggregate_filtering:
                   value: 'True'

7. Save the updates to your Compute environment file.

8. To apply this configuration, add your Compute environment file to the stack with your other environment files and deploy the overcloud:

   (undercloud) $ openstack overcloud deploy --templates \
     -e [your environment files]
     -e /home/stack/templates/<compute_environment_file>.yaml

9. Create a host aggregate for SEV Compute nodes to ensure that instances that do not request memory encryption are not created on SEV-capable hosts:

   (undercloud) $ source ~/overcloudrc
   (overcloud) $ openstack aggregate create sev_agg
   (overcloud) $ openstack aggregate add host sev_agg hostA
   (overcloud) $ openstack aggregate add host sev_agg hostB
   (overcloud) $ openstack --os-compute-api-version 2.53 aggregate set --property trait:HW_CPU_X86_AMD_SEV=required sev_agg

Procedure

1. Identify the source Compute node host name and the destination Compute node host name:

   (undercloud) $ source ~/overcloudrc
   (overcloud) $ openstack compute service list

2. List the instances on the source Compute node and locate the ID of the instance or instances that you want to migrate:

   (overcloud) $ openstack server list --host <source> --all-projects

   Replace <source> with the name or ID of the source Compute node.

3. Optional: If you are migrating instances from a source Compute node to perform maintenance on the node, you must disable the node to prevent the scheduler from assigning new instances to the node during maintenance:

   (overcloud) $ source ~/stackrc
   (undercloud) $ openstack compute service set <source> nova-compute --disable

   Replace <source> with the name or ID of the source Compute node.

Procedure

1. To cold migrate an instance, enter the following command to power off and move the instance:

   (overcloud) $ openstack server migrate <vm> --wait

   • Replace <vm> with the name or ID of the instance to migrate.
   • Specify the --block-migration flag if migrating a locally stored volume.
2. Wait for migration to complete. While you wait for the instance migration to complete, you can check the migration status. For more information, see Checking migration status.

3. Check the status of the instance:

   (overcloud) $ openstack server list --all-projects

   A status of "VERIFY_RESIZE" indicates you need to confirm or revert the migration:

   • If the migration worked as expected, confirm it:

     (overcloud) $ openstack server resize --confirm <vm>`

     Replace <vm> with the name or ID of the instance to migrate. A status of "ACTIVE" indicates that the instance is ready to use.

   • If the migration did not work as expected, revert it:

     (overcloud) $ openstack server resize --revert <vm>`

     Replace <vm> with the name or ID of the instance.

4. Restart the instance:

   (overcloud) $ openstack server start <vm>

   Replace <vm> with the name or ID of the instance.

5. Optional: If you disabled the source Compute node for maintenance, you must re-enable the node so that new instances can be assigned to it:

   (overcloud) $ source ~/stackrc
   (undercloud) $ openstack compute service set <source> nova-compute --enable

   Replace <source> with the host name of the source Compute node.

Procedure

1. To live migrate an instance, specify the instance and the destination Compute node:

   (overcloud) $ openstack server migrate <vm> --live-migration [--host <dest>] --wait

   • Replace <vm> with the name or ID of the instance.

   • Replace <dest> with the name or ID of the destination Compute node.

     Note

     The openstack server migrate command covers migrating instances with shared storage, which is the default. Specify the --block-migration flag to migrate a locally stored volume:

     (overcloud) $ openstack server migrate <vm> --live-migration [--host <dest>] --wait --block-migration

2. Confirm that the instance is migrating:

   (overloud) $ openstack server show <vm>
   
   +----------------------+--------------------------------------+
   | Field                | Value                                |
   +----------------------+--------------------------------------+
   | ...                  | ...                                  |
   | status               | MIGRATING                            |
   | ...                  | ...                                  |
   +----------------------+--------------------------------------+

3. Wait for migration to complete. While you wait for the instance migration to complete, you can check the migration status. For more information, see Checking migration status.

4. Check the status of the instance to confirm if the migration was successful:

   (overcloud) $ openstack server list --host <dest> --all-projects

   Replace <dest> with the name or ID of the destination Compute node.

5. Optional: If you disabled the source Compute node for maintenance, you must re-enable the node so that new instances can be assigned to it:

   (overcloud) $ source ~/stackrc
   (undercloud) $ openstack compute service set <source> nova-compute --enable

   Replace <source> with the host name of the source Compute node.

1. Queued: The Compute service has accepted the request to migrate an instance, and migration is pending.
2. Preparing: The Compute service is preparing to migrate the instance.
3. Running: The Compute service is migrating the instance.
4. Post-migrating: The Compute service has built the instance on the destination Compute node and is releasing resources on the source Compute node.
5. Completed: The Compute service has completed migrating the instance and finished releasing resources on the source Compute node.

Procedure

1. Retrieve the list of migration IDs for the instance:

   $ nova server-migration-list <vm>
   
   +----+-------------+-----------  (...)
   | Id | Source Node | Dest Node | (...)
   +----+-------------+-----------+ (...)
   | 2  | -           | -         | (...)
   +----+-------------+-----------+ (...)

   Replace <vm> with the name or ID of the instance.

2. Show the status of the migration:

   $  <vm> <migration-id>

   • Replace <vm> with the name or ID of the instance.

   • Replace <migration-id> with the ID of the migration.

     Running the nova server-migration-show command returns the following example output:

     +------------------------+--------------------------------------+
     | Property               | Value                                |
     +------------------------+--------------------------------------+
     | created_at             | 2017-03-08T02:53:06.000000           |
     | dest_compute           | controller                           |
     | dest_host              | -                                    |
     | dest_node              | -                                    |
     | disk_processed_bytes   | 0                                    |
     | disk_remaining_bytes   | 0                                    |
     | disk_total_bytes       | 0                                    |
     | id                     | 2                                    |
     | memory_processed_bytes | 65502513                             |
     | memory_remaining_bytes | 786427904                            |
     | memory_total_bytes     | 1091379200                           |
     | server_uuid            | d1df1b5a-70c4-4fed-98b7-423362f2c47c |
     | source_compute         | compute2                             |
     | source_node            | -                                    |
     | status                 | running                              |
     | updated_at             | 2017-03-08T02:53:47.000000           |
     +------------------------+--------------------------------------+

     Tip

     The OpenStack Compute service measures progress of the migration by the number of remaining memory bytes to copy. If this number does not decrease over time, the migration might be unable to complete, and the Compute service might abort it.

Procedure

1. Open your Compute environment file.

2. Add the heat parameter that controls the cron job that you want to add or modify. For example, to purge the shadow tables immediately after they are archived, set the following parameter to "True":

   parameter_defaults:
     …
     NovaCronArchiveDeleteRowsPurge: True

   For a complete list of the heat parameters to manage database cron jobs, see Configuration options for Openstack Compute (nova) automated database management.

3. Save the updates to your Compute environment file.

4. Add your Compute environment file to the stack with your other environment files and deploy the overcloud:

   (undercloud)$ openstack overcloud deploy --templates \
     -e [your environment files] \
     -e /home/stack/templates/<compute_environment_file>.yaml

1. Build a custom GPU-enabled overcloud image.
2. Prepare the GPU role, profile, and flavor for designating Compute nodes for vGPU.
3. Configure the Compute node for vGPU.
4. Deploy the overcloud.

Procedure

1. To determine the vendor ID and product ID for each passthrough device type, run the following command on the physical server that has the PCI cards:

   # lspci -nn | grep -i <gpu_name>

   For example, to determine the vendor and product ID for an NVIDIA GPU, run the following command:

   # lspci -nn | grep -i nvidia
   3b:00.0 3D controller [0302]: NVIDIA Corporation TU104GL [Tesla T4] [10de:1eb8] (rev a1)
   d8:00.0 3D controller [0302]: NVIDIA Corporation TU104GL [Tesla T4] [10de:1db4] (rev a1)

2. To configure the Controller node on the overcloud for PCI passthrough, create an environment file, for example, pci_passthru_controller.yaml.

3. Add PciPassthroughFilter to the NovaSchedulerDefaultFilters parameter in pci_passthru_controller.yaml:

   parameter_defaults:
     NovaSchedulerDefaultFilters: ['RetryFilter','AvailabilityZoneFilter','ComputeFilter','ComputeCapabilitiesFilter','ImagePropertiesFilter','ServerGroupAntiAffinityFilter','ServerGroupAffinityFilter','PciPassthroughFilter','NUMATopologyFilter']

4. To specify the PCI alias for the devices on the Controller node, add the following to pci_passthru_controller.yaml:

   ControllerExtraConfig:
       nova::pci::aliases:
         -  name: "t4"
            product_id: "1eb8"
            vendor_id: "10de"
         -  name: "v100"
            product_id: "1db4"
            vendor_id: "10de"

   Note

   If the nova-api service is running in a role other than the Controller, then replace ControllerExtraConfig with the user role, in the format <Role>ExtraConfig.

5. To configure the Compute node on the overcloud for PCI passthrough, create an environment file, for example, pci_passthru_compute.yaml.

6. To specify the available PCIs for the devices on the Compute node, add the following to pci_passthru_compute.yaml:

   parameter_defaults:
     NovaPCIPassthrough:
       - vendor_id: "10de"
         product_id: "1eb8"

7. To enable IOMMU in the server BIOS of the Compute nodes to support PCI passthrough, add the KernelArgs parameter to pci_passthru_compute.yaml:

      parameter_defaults:
         ...
         ComputeParameters:
           KernelArgs: "intel_iommu=on iommu=pt"

8. Deploy the overcloud, adding your custom environment files to the stack along with your other environment files:

   (undercloud) $ openstack overcloud deploy --templates \
     -e [your environment files]
     -e /home/stack/templates/pci_passthru_controller.yaml
     -e /home/stack/templates/pci_passthru_compute.yaml

9. Configure a flavor to request the PCI devices. The following example requests two devices, each with a vendor ID of 10de and a product ID of 13f2:

   # openstack flavor set m1.large --property "pci_passthrough:alias"="t4:2"

10. Create an instance with a PCI passthrough device:

    # openstack server create --flavor m1.large --image rhelgpu --wait test-pci

11. Log in to the instance as a cloud-user. For more information, see Log in to an Instance.

12. Install the GPU driver on the instance. For example, run the following script to install an NVIDIA driver:

    $ sh NVIDIA-Linux-x86_64-430.24-grid.run

Verification

1. To verify that the GPU is accessible from the instance, run the following command from the instance:

   $ lspci -nn | grep <gpu_name>

2. To check the NVIDIA System Management Interface status, run the following command from the instance:

   $ nvidia-smi

   Example output:

   -----------------------------------------------------------------------------
   | NVIDIA-SMI 440.33.01    Driver Version: 440.33.01    CUDA Version: 10.2     |
   |---------------------------------------------------------------------------+
   | GPU  Name        Persistence-M| Bus-Id        Disp.A | Volatile Uncorr. ECC |
   | Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |
   |===========================================================================|
   |   0  Tesla T4            Off  | 00000000:01:00.0 Off |                    0 |
   | N/A   43C    P0    20W /  70W |      0MiB / 15109MiB |      0%      Default |
   ---------------------------------------------------------------------------
   
   -----------------------------------------------------------------------------
   | Processes:                                                       GPU Memory |
   |  GPU       PID   Type   Process name                             Usage      |
   |=============================================================================|
   |  No running processes found                                                 |
   -----------------------------------------------------------------------------

1. Install the libguestfs-tools package on the undercloud to get the virt-customize tool:

   (undercloud) [stack@undercloud-0 ~]$ sudo dnf install libguestfs-tools

   Important

   If you install the libguestfs-tools package on the undercloud, disable iscsid.socket to avoid port conflicts with the tripleo_iscsid service on the undercloud:

   $ sudo systemctl disable --now iscsid.socket

2. Extract the images:

   (undercloud) [stack@undercloud-0 ~]$ tar -xf /usr/share/rhosp-director-images/overcloud-full.tar
   (undercloud) [stack@undercloud-0 ~]$ tar -xf /usr/share/rhosp-director-images/ironic-python-agent.tar

3. Copy the default image:

   (undercloud) [stack@undercloud-0 ~]$ cp overcloud-full.qcow2 overcloud-realtime-compute.qcow2

4. Register the image and configure the required subscriptions:

   (undercloud) [stack@undercloud-0 ~]$  virt-customize -a overcloud-realtime-compute.qcow2 --run-command 'subscription-manager register --username=[username] --password=[password]'
   [  0.0] Examining the guest ...
   [ 10.0] Setting a random seed
   [ 10.0] Running: subscription-manager register --username=[username] --password=[password]
   [ 24.0] Finishing off

   Replace the username and password values with your Red Hat customer account details. For general information about building a Real-time overcloud image, see the Modifying the Red Hat Enterprise Linux OpenStack Platform Overcloud Image with virt-customize knowledgebase article.

5. Find the SKU of the Red Hat OpenStack Platform for Real Time subscription. The SKU might be located on a system that is already registered to the Red Hat Subscription Manager with the same account and credentials. For example:

   $ sudo subscription-manager list

6. Attach the Red Hat OpenStack Platform for Real Time subscription to the image:

   (undercloud) [stack@undercloud-0 ~]$  virt-customize -a overcloud-realtime-compute.qcow2 --run-command 'subscription-manager attach --pool [subscription-pool]'

7. Create a script to configure rt on the image:

   (undercloud) [stack@undercloud-0 ~]$ cat rt.sh
     #!/bin/bash
   
     set -eux
   
     subscription-manager repos --enable=[REPO_ID]
     dnf -v -y --setopt=protected_packages= erase kernel.$(uname -m)
     dnf -v -y install kernel-rt kernel-rt-kvm tuned-profiles-nfv-host
   
     # END OF SCRIPT

8. Run the script to configure the real-time image:

   (undercloud) [stack@undercloud-0 ~]$ virt-customize -a overcloud-realtime-compute.qcow2 -v --run rt.sh 2>&1 | tee virt-customize.log

9. Re-label SELinux:

   (undercloud) [stack@undercloud-0 ~]$ virt-customize -a overcloud-realtime-compute.qcow2 --selinux-relabel

10. Extract vmlinuz and initrd. For example:

    (undercloud) [stack@undercloud-0 ~]$ mkdir image
    (undercloud) [stack@undercloud-0 ~]$ guestmount -a overcloud-realtime-compute.qcow2 -i --ro image
    (undercloud) [stack@undercloud-0 ~]$ cp image/boot/vmlinuz-4.18.0-80.7.1.rt9.153.el8_0.x86_64 ./overcloud-realtime-compute.vmlinuz
    (undercloud) [stack@undercloud-0 ~]$ cp image/boot/initramfs-4.18.0-80.7.1.rt9.153.el8_0.x86_64.img ./overcloud-realtime-compute.initrd
    (undercloud) [stack@undercloud-0 ~]$ guestunmount image

    Note

    The software version in the vmlinuz and initramfs filenames vary with the kernel version.

11. Upload the image:

    (undercloud) [stack@undercloud-0 ~]$ openstack overcloud image upload --update-existing --os-image-name overcloud-realtime-compute.qcow2

Procedure

1. Based on the /usr/share/openstack-tripleo-heat-templates/environments/compute-real-time-example.yaml file, create a compute-real-time.yaml environment file that sets the parameters for the ComputeRealTime role.

   cp /usr/share/openstack-tripleo-heat-templates/environments/compute-real-time-example.yaml /home/stack/templates/compute-real-time.yaml

   The file must include values for the following parameters:

   • IsolCpusList and NovaComputeCpuDedicatedSet: List of isolated CPU cores and virtual CPU pins to reserve for real-time workloads. This value depends on the CPU hardware of your real-time Compute nodes.
   • NovaComputeCpuSharedSet: List of host CPUs to reserve for emulator threads.
   • KernelArgs: Arguments to pass to the kernel of the Real-time Compute nodes. For example, you can use default_hugepagesz=1G hugepagesz=1G hugepages=<number_of_1G_pages_to_reserve> hugepagesz=2M hugepages=<number_of_2M_pages> to define the memory requirements of guests that have huge pages with multiple sizes. In this example, the default size is 1GB but you can also reserve 2M huge pages.

2. Add the ComputeRealTime role to your roles data file and regenerate the file. For example:

   $ openstack overcloud roles generate -o /home/stack/templates/rt_roles_data.yaml Controller Compute ComputeRealTime

   This command generates a ComputeRealTime role with contents similar to the following example, and also sets the ImageDefault option to overcloud-realtime-compute.

   - name: ComputeRealTime
     description: |
       Compute role that is optimized for real-time behaviour. When using this role
       it is mandatory that an overcloud-realtime-compute image is available and
       the role specific parameters IsolCpusList, NovaComputeCpuDedicatedSet and
       NovaComputeCpuSharedSet are set accordingly to the hardware of the real-time compute nodes.
     CountDefault: 1
     networks:
       InternalApi:
         subnet: internal_api_subnet
       Tenant:
         subnet: tenant_subnet
       Storage:
         subnet: storage_subnet
     HostnameFormatDefault: '%stackname%-computerealtime-%index%'
     ImageDefault: overcloud-realtime-compute
     RoleParametersDefault:
       TunedProfileName: "realtime-virtual-host"
       KernelArgs: ""      # these must be set in an environment file
       IsolCpusList: ""    # or similar according to the hardware
       NovaComputeCpuDedicatedSet: ""  # of real-time nodes
       NovaComputeCpuSharedSet: ""     #
       NovaLibvirtMemStatsPeriodSeconds: 0
     ServicesDefault:
       - OS::TripleO::Services::Aide
       - OS::TripleO::Services::AuditD
       - OS::TripleO::Services::BootParams
       - OS::TripleO::Services::CACerts
       - OS::TripleO::Services::CephClient
       - OS::TripleO::Services::CephExternal
       - OS::TripleO::Services::CertmongerUser
       - OS::TripleO::Services::Collectd
       - OS::TripleO::Services::ComputeCeilometerAgent
       - OS::TripleO::Services::ComputeNeutronCorePlugin
       - OS::TripleO::Services::ComputeNeutronL3Agent
       - OS::TripleO::Services::ComputeNeutronMetadataAgent
       - OS::TripleO::Services::ComputeNeutronOvsAgent
       - OS::TripleO::Services::Docker
       - OS::TripleO::Services::Fluentd
       - OS::TripleO::Services::IpaClient
       - OS::TripleO::Services::Ipsec
       - OS::TripleO::Services::Iscsid
       - OS::TripleO::Services::Kernel
       - OS::TripleO::Services::LoginDefs
       - OS::TripleO::Services::MetricsQdr
       - OS::TripleO::Services::MySQLClient
       - OS::TripleO::Services::NeutronBgpVpnBagpipe
       - OS::TripleO::Services::NeutronLinuxbridgeAgent
       - OS::TripleO::Services::NeutronVppAgent
       - OS::TripleO::Services::NovaCompute
       - OS::TripleO::Services::NovaLibvirt
       - OS::TripleO::Services::NovaLibvirtGuests
       - OS::TripleO::Services::NovaMigrationTarget
       - OS::TripleO::Services::ContainersLogrotateCrond
       - OS::TripleO::Services::OpenDaylightOvs
       - OS::TripleO::Services::Podman
       - OS::TripleO::Services::Rhsm
       - OS::TripleO::Services::RsyslogSidecar
       - OS::TripleO::Services::Securetty
       - OS::TripleO::Services::SensuClient
       - OS::TripleO::Services::SkydiveAgent
       - OS::TripleO::Services::Snmp
       - OS::TripleO::Services::Sshd
       - OS::TripleO::Services::Timesync
       - OS::TripleO::Services::Timezone
       - OS::TripleO::Services::TripleoFirewall
       - OS::TripleO::Services::TripleoPackages
       - OS::TripleO::Services::Vpp
       - OS::TripleO::Services::OVNController
       - OS::TripleO::Services::OVNMetadataAgent

   For general information about custom roles and about the roles-data.yaml, see the Roles section.

3. Create the compute-realtime flavor to tag nodes that you want to designate for real-time workloads. For example:

   $ source ~/stackrc
   $ openstack flavor create --id auto --ram 6144 --disk 40 --vcpus 4 compute-realtime
   $ openstack flavor set --property "cpu_arch"="x86_64" --property "capabilities:boot_option"="local" --property "capabilities:profile"="compute-realtime" compute-realtime

4. Tag each node that you want to designate for real-time workloads with the compute-realtime profile.

   $ openstack baremetal node set --property capabilities='profile:compute-realtime,boot_option:local' <NODE UUID>

5. Map the ComputeRealTime role to the compute-realtime flavor by creating an environment file with the following content:

   parameter_defaults:
     OvercloudComputeRealTimeFlavor: compute-realtime

6. Run the openstack overcloud deploy command with the -e option and specify all the environment files that you created, as well as the new roles file. For example:

   $ openstack overcloud deploy -r /home/stack/templates/rt~/my_roles_data.yaml  -e home/stack/templates/compute-real-time.yaml <FLAVOR_ENV_FILE>

1. Create the compute-real-time.yaml file with the following parameters:

   parameter_defaults:
     ComputeRealTimeParameters:
       IsolCpusList: "1"
       NovaComputeCpuDedicatedSet: "1"
       NovaComputeCpuSharedSet: "0"
       KernelArgs: "default_hugepagesz=1G hugepagesz=1G hugepages=16"

2. Create a new rt_roles_data.yaml file with the ComputeRealTime role:

   $ openstack overcloud roles generate -o ~/rt_roles_data.yaml Controller ComputeRealTime

3. Deploy the overcloud, adding both your new real-time roles data file and your real-time environment file to the stack along with your other environment files:

   (undercloud) $ openstack overcloud deploy --templates \
     -r /home/stack/rt_roles_data.yaml
     -e [your environment files]
     -e /home/stack/templates/compute-real-time.yaml

   This command deploys one Controller node and one Real-time Compute node.

4. Log into the Real-time Compute node and check the following parameters. Replace <...> with the values of the relevant parameters from the compute-real-time.yaml.

   [root@overcloud-computerealtime-0 ~]# uname -a
   Linux overcloud-computerealtime-0 4.18.0-80.7.1.rt9.153.el8_0.x86_64 #1 SMP PREEMPT RT Wed Dec 13 13:37:53 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux
   [root@overcloud-computerealtime-0 ~]# cat /proc/cmdline
   BOOT_IMAGE=/boot/vmlinuz-4.18.0-80.7.1.rt9.153.el8_0.x86_64 root=UUID=45ae42d0-58e7-44fe-b5b1-993fe97b760f ro console=tty0 crashkernel=auto console=ttyS0,115200 default_hugepagesz=1G hugepagesz=1G hugepages=16
   [root@overcloud-computerealtime-0 ~]# tuned-adm active
   Current active profile: realtime-virtual-host
   [root@overcloud-computerealtime-0 ~]# grep ^isolated_cores /etc/tuned/realtime-virtual-host-variables.conf
   isolated_cores=<IsolCpusList>
   [root@overcloud-computerealtime-0 ~]# cat /usr/lib/tuned/realtime-virtual-host/lapic_timer_adv_ns
   X (X != 0)
   [root@overcloud-computerealtime-0 ~]# cat /sys/module/kvm/parameters/lapic_timer_advance_ns
   X (X != 0)
   [root@overcloud-computerealtime-0 ~]# cat /sys/devices/system/node/node0/hugepages/hugepages-1048576kB/nr_hugepages
   X (X != 0)
   [root@overcloud-computerealtime-0 ~]# crudini --get /var/lib/config-data/puppet-generated/nova_libvirt/etc/nova/nova.conf compute cpu_dedicated_set
   <NovaComputeCpuDedicatedSet>
   [root@overcloud-computerealtime-0 ~]# crudini --get /var/lib/config-data/puppet-generated/nova_libvirt/etc/nova/nova.conf compute cpu_shared_set
   <NovaComputeCpuSharedSet>

Launching a real-time instance

1. Make sure that the compute-realtime flavor exists on the overcloud, as described in the Deploying the Real-time Compute Role section.

2. Launch the real-time instance.

   # openstack server create  --image <rhel> --flavor r1.small --nic net-id=<dpdk-net> test-rt

3. Optionally, verify that the instance uses the assigned emulator threads.

   # virsh dumpxml <instance-id> | grep vcpu -A1
   <vcpu placement='static'>4</vcpu>
   <cputune>
     <vcpupin vcpu='0' cpuset='1'/>
     <vcpupin vcpu='1' cpuset='3'/>
     <vcpupin vcpu='2' cpuset='5'/>
     <vcpupin vcpu='3' cpuset='7'/>
     <emulatorpin cpuset='0-1'/>
     <vcpusched vcpus='2-3' scheduler='fifo'
     priority='1'/>
   </cputune>