Chapter 11. Tuning a Red Hat OpenStack Platform environment

Procedure

1. Deploy an overcloud with NovaComputeCpuSharedSet defined for a given role. The value of NovaComputeCpuSharedSet applies to the cpu_shared_set parameter in the nova.conf file for hosts within that role.

   parameter_defaults:
       ComputeOvsDpdkParameters:
           NovaComputeCpuSharedSet: "0-1,16-17"
           NovaComputeCpuDedicatedSet: "2-15,18-31"

2. Create a flavor to build instances with emulator threads separated into a shared pool.

   openstack flavor create --ram <size_mb> --disk <size_gb> --vcpus <vcpus> <flavor>

3. Add the hw:emulator_threads_policy extra specification, and set the value to share. Instances created with this flavor will use the instance CPUs defined in the cpu_share_set parameter in the nova.conf file.

   openstack flavor set <flavor> --property hw:emulator_threads_policy=share

Verification

1. Identify the host and name for a given instance.

   openstack server show <instance_id>

2. Use SSH to log on to the identified host as tripleo-admin.

   ssh tripleo-admin@compute-1
   [compute-1]$ sudo virsh dumpxml instance-00001 | grep `'emulatorpin cpuset'`

1. Add the NeutronPhysicalDevMappings parameter in the parameter_defaults section to link between the logical network name and the physical interface.

   parameter_defaults:
     NeutronPhysicalDevMappings:
       - sriov2:p5p2

2. Add the new property, trusted, to the SR-IOV parameters.

   parameter_defaults:
     NeutronPhysicalDevMappings:
       - sriov2:p5p2
     NovaPCIPassthrough:
       - vendor_id: "8086"
         product_id: "1572"
         physical_network: "sriov2"
         trusted: "true"

   Note

   You must include double quotation marks around the value "true".

1. Create a network of type vlan.

   openstack network create trusted_vf_network  --provider-network-type vlan \
    --provider-segment 111 --provider-physical-network sriov2 \
    --external --disable-port-security

2. Create a subnet.

   openstack subnet create --network trusted_vf_network \
     --ip-version 4 --subnet-range 192.168.111.0/24 --no-dhcp \
    subnet-trusted_vf_network

3. Create a port. Set the vnic-type option to direct, and the binding-profile option to true.

   openstack port create --network sriov111 \
   --vnic-type direct --binding-profile trusted=true \
   sriov111_port_trusted

4. Create an instance, and bind it to the previously-created trusted port.

   openstack server create --image rhel --flavor dpdk  --network internal --port trusted_vf_network_port_trusted --config-drive True --wait rhel-dpdk-sriov_trusted

1. On the compute node that you created the instance, enter the following command:

   # ip link
   7: p5p2: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 9000 qdisc mq state UP mode DEFAULT group default qlen 1000
       link/ether b4:96:91:1c:40:fa brd ff:ff:ff:ff:ff:ff
       vf 6 MAC fa:16:3e:b8:91:c2, vlan 111, spoof checking off, link-state auto, trust on, query_rss off
       vf 7 MAC fa:16:3e:84:cf:c8, vlan 111, spoof checking off, link-state auto, trust off, query_rss off

2. Verify that the trust status of the VF is trust on. The example output contains details of an environment that contains two ports. Note that vf 6 contains the text trust on.
3. You can disable spoof checking if you set port_security_enabled: false in the Networking service (neutron) network, or if you include the argument --disable-port-security when you run the openstack port create command.

Procedure

1. Log in to the undercloud host as the stack user.

2. Source the stackrc undercloud credentials file:

   $ source ~/stackrc

3. Create a custom environment YAML file and under parameter_defaults add the following definitions to increase the RX and TX queue size:

   parameter_defaults:
     NovaLibvirtRxQueueSize: 1024
     NovaLibvirtTxQueueSize: 1024

4. Run the deployment command and include the core heat templates, other environment files, the environment file that contains your RX and TX queue size changes:

   Example

   $ openstack overcloud deploy --templates \
   -e <other_environment_files> \
   -e /home/stack/my_tx-rx_queue_sizes.yaml

Verification

1. Observe the values for RX queue size and TX queue size in the nova.conf file.

   $ egrep "^[rt]x_queue_size" /var/lib/config-data/puppet-generated/\
   nova_libvirt/etc/nova/nova.conf

   You should see the following:

   rx_queue_size=1024
   tx_queue_size=1024

2. Check the values for RX queue size and TX queue size in the VM instance XML file generated by libvirt on the Compute host:

   1. Create a new instance.

   2. Obtain the Compute host and and instance name:

      $ openstack server show testvm-queue-sizes -c OS-EXT-SRV-ATTR:\
      hypervisor_hostname -c OS-EXT-SRV-ATTR:instance_name

      Sample output

      You should see output similar to the following:

      +-------------------------------------+------------------------------------+
      | Field                               | Value                              |
      +-------------------------------------+------------------------------------+
      | OS-EXT-SRV-ATTR:hypervisor_hostname | overcloud-novacompute-1.sales      |
      | OS-EXT-SRV-ATTR:instance_name       | instance-00000059                  |
      +-------------------------------------+------------------------------------+

   3. Log into the Compute host and dump the instance definition.

      Example

      $ podman exec nova_libvirt virsh dumpxml instance-00000059

      Sample output

      You should see output similar to the following:

      ...
         <interface type='vhostuser'>
           <mac address='56:48:4f:4d:5e:6f'/>
           <source type='unix' path='/tmp/vhost-user1' mode='server'/>
           <model type='virtio'/>
           <driver name='vhost' rx_queue_size='1024'   tx_queue_size='1024' />
           <address type='pci' domain='0x0000' bus='0x00' slot='0x10' function='0x0'/>
         </interface>
      ...

Procedure

1. Set a new NeutronPhysnetNUMANodesMapping parameter to map the physical network to the NUMA node that you associate with the physical network.

2. If you use tunnels, such as VxLAN or GRE, you must also set the NeutronTunnelNUMANodes parameter.

   parameter_defaults:
     NeutronPhysnetNUMANodesMapping: {<physnet_name>: [<NUMA_NODE>]}
     NeutronTunnelNUMANodes: <NUMA_NODE>,<NUMA_NODE>

   Example

   Here is an example with two physical networks tunneled to NUMA node 0:

   • one project network associated with NUMA node 0

   • one management network without any affinity

     parameter_defaults:
       NeutronBridgeMappings:
         - tenant:br-link0
       NeutronPhysnetNUMANodesMapping: {tenant: [1], mgmt: [0,1]}
       NeutronTunnelNUMANodes: 0

   • In this example, assign the physnet of the device named eno2 to NUMA number 0.

     # ethtool -i eno2
     bus-info: 0000:18:00.1
     
     # cat /sys/devices/pci0000:16/0000:16:02.0/0000:18:00.1/numa_node
     0

     Observe the physnet settings in the example heat template:

     NeutronBridgeMappings: 'physnet1:br-physnet1'
     NeutronPhysnetNUMANodesMapping: {physnet1: [0] }
     
     - type: ovs_user_bridge
                     name: br-physnet1
                     mtu: 9000
                     members:
                       - type: ovs_dpdk_port
                         name: dpdk2
                         members:
                           - type: interface
                             name: eno2

1. Observe the configuration in the file /var/lib/config-data/puppet-generated/nova_libvirt/etc/nova/nova.conf:

   [neutron_physnet_tenant]
   numa_nodes=1
   [neutron_tunnel]
   numa_nodes=1

2. Confirm the new configuration with the lscpu command:

   $ lscpu

3. Launch a VM with the NIC attached to the appropriate network.