Copy the tripleo-heat-templates/plan-samples/plan-environment-derived-params.yaml file to a local directory and set the input parameters to suit your environment.

  workflow_parameters:
    tripleo.derive_params.v1.derive_parameters:
      # DPDK Parameters #
      # Specifices the minimum number of CPU physical cores to be allocated for DPDK
      # PMD threads. The actual allocation will be based on network config, if
      # the a DPDK port is associated with a numa node, then this configuration
      # will be used, else 1.
      num_phy_cores_per_numa_node_for_pmd: 1
      # Amount of memory to be configured as huge pages in percentage. Ouf the
      # total available memory (excluding the NovaReservedHostMemory), the
      # specified percentage of the remaining is configured as huge pages.
      huge_page_allocation_percentage: 50

Deploy the overcloud with the update-plan-only parameter to calculate the derived parameters.

  $ openstack overcloud deploy --templates --update-plan-only -r /home/stack/ospd-13-sriov-dpdk-heterogeneous-cluster/roles_data.yaml -e /usr/share/openstack-tripleo-heat-templates/environments/network-isolation.yaml -e /home/stack/ospd-13-sriov-dpdk-heterogeneous-cluster/docker-images.yaml -e /usr/share/openstack-tripleo-heat-templates/environments/host-config-and-reboot.yaml -e /home/stack/ospd-13-sriov-dpdk-heterogeneous-cluster/network-environment.yaml -e /usr/share/openstack-tripleo-heat-templates/environments/neutron-sriov.yaml
  -e /usr/share/openstack-tripleo-heat-templates/environments/neutron-ovs-dpdk.yaml
  -p /home/stack/plan-environment-derived-params.yaml

Copy the derived parameters from the plan-environment.yaml to the network-environment.yaml file.

  # DPDK compute node.
  ComputeOvsDpdkParameters:
    KernelArgs: default_hugepagesz=1GB hugepagesz=1G hugepages=32 iommu=pt intel_iommu=on
    TunedProfileName: "cpu-partitioning"
    IsolCpusList: "1,2,3,4,5,6,7,9,10,17,18,19,20,21,22,23,11,12,13,14,15,25,26,27,28,29,30,31"
    NovaVcpuPinSet: ['2,3,4,5,6,7,18,19,20,21,22,23,10,11,12,13,14,15,26,27,28,29,30,31']
    NovaReservedHostMemory: 4096
    OvsDpdkSocketMemory: "1024,1024"
    OvsDpdkMemoryChannels: "4"
    OvsDpdkCoreList: "0,16,8,24"
    OvsPmdCoreList: "1,17,9,25"

Set the NeutronGlobalPhysnetMtu parameter in the network-environment.yaml file.

parameter_defaults:
  # MTU global configuration
  NeutronGlobalPhysnetMtu: 9000

Set the MTU value on the bridge to the Compute node in the controller.yaml file.

  -
    type: ovs_bridge
    name: br-link0
    use_dhcp: false
    members:
      -
        type: interface
        name: nic3
        mtu: 9000

Set the MTU values for an OVS-DPDK bond in the compute.yaml file:

- type: ovs_user_bridge
  name: br-link0
  use_dhcp: false
  members:
    - type: ovs_dpdk_bond
      name: dpdkbond0
      mtu: 9000
      rx_queue: 2
      members:
        - type: ovs_dpdk_port
          name: dpdk0
          mtu: 9000
          members:
            - type: interface
              name: nic4
        - type: ovs_dpdk_port
          name: dpdk1
          mtu: 9000
          members:
            - type: interface
              name: nic5

Create an aggregate group and add a host to it for OVS-DPDK.

 # openstack aggregate create --zone=dpdk dpdk
 # openstack aggregate add host dpdk compute-ovs-dpdk-0.localdomain

Create a flavor.

# openstack flavor create m1.medium_huge_4cpu --ram 4096 --disk 150 --vcpus 4

Here, m1.medium_huge_4cpu is the flavor name, 4096 is the memory size in MB, 150 is the disk size in GB (default 0G), and 4 is the number of vCPUs.

Set additional flavor properties.

# openstack flavor set --property hw:cpu_policy=dedicated --property hw:mem_page_size=1GB m1.medium_huge_4cpu --property hw:emulator_threads_policy=isolate

Here, m1.medium_huge_4cpu is the flavor name and the remaining parameters set the other properties for the flavor.

Create the network.

# openstack network create net1 --provider-physical-network tenant --provider-network-type vlan --provider-segment <VLAN-ID>

Deploy an instance.

# openstack server create --flavor m1.medium_huge_4cpu --availability-zone dpdk --image rhel_7.3 --nic net-id=net1

Where:

Set the image properties.

# openstack image set --property hw_vif_multiqueue_enabled=true <image-id>

Here, hw_vif_multiqueue_enabled=true is a property on this image to enable multiqueue, <image-id> is the name or ID of the image to modify.

Set additional flavor properties.

# openstack flavor set m1.vm_mq set hw:vif_multiqueue_enabled=true

Here, m1.vm_mq is the flavor ID or name, and the remaining options enable multiqueue for the flavor.

Review the bridge configuration and confirm that the bridge was created with the datapath_type=netdev.

# ovs-vsctl list bridge br0
_uuid               : bdce0825-e263-4d15-b256-f01222df96f3
auto_attach         : []
controller          : []
datapath_id         : "00002608cebd154d"
datapath_type       : netdev
datapath_version    : "<built-in>"
external_ids        : {}
fail_mode           : []
flood_vlans         : []
flow_tables         : {}
ipfix               : []
mcast_snooping_enable: false
mirrors             : []
name                : "br0"
netflow             : []
other_config        : {}
ports               : [52725b91-de7f-41e7-bb49-3b7e50354138]
protocols           : []
rstp_enable         : false
rstp_status         : {}
sflow               : []
status              : {}
stp_enable          : false

Review the OVS service by confirming that the neutron-ovs-agent is configured to start automatically.

# systemctl status neutron-openvswitch-agent.service
neutron-openvswitch-agent.service - OpenStack Neutron Open vSwitch Agent
Loaded: loaded (/usr/lib/systemd/system/neutron-openvswitch-agent.service; enabled; vendor preset: disabled)
Active: active (running) since Mon 2015-11-23 14:49:31 AEST; 25min ago

If the service is having trouble starting, you can view any related messages.

# journalctl -t neutron-openvswitch-agent.service

Confirm that the PMD CPU mask of the ovs-dpdk are pinned to the CPUs. In case of HT, use sibling CPUs.

For example, take CPU4:

# cat /sys/devices/system/cpu/cpu4/topology/thread_siblings_list
4,20

So, using CPU 4 and 20:

# ovs-vsctl set Open_vSwitch . other_config:pmd-cpu-mask=0x100010

Display their status:

# tuna -t ovs-vswitchd -CP
thread  ctxt_switches pid SCHED_ rtpri affinity voluntary nonvoluntary       cmd
3161	OTHER 	0    	6	765023      	614	ovs-vswitchd
3219   OTHER 	0    	6     	1        	0   	handler24
3220   OTHER 	0    	6     	1        	0   	handler21
3221   OTHER 	0    	6     	1        	0   	handler22
3222   OTHER 	0    	6     	1        	0   	handler23
3223   OTHER 	0    	6     	1        	0   	handler25
3224   OTHER 	0    	6     	1        	0   	handler26
3225   OTHER 	0    	6     	1        	0   	handler27
3226   OTHER 	0    	6     	1        	0   	handler28
3227   OTHER 	0    	6     	2        	0   	handler31
3228   OTHER 	0    	6     	2        	4   	handler30
3229   OTHER 	0    	6     	2        	5   	handler32
3230   OTHER 	0    	6	953538      	431   revalidator29
3231   OTHER 	0    	6   1424258      	976   revalidator33
3232   OTHER 	0    	6   1424693      	836   revalidator34
3233   OTHER 	0    	6	951678      	503   revalidator36
3234   OTHER 	0    	6   1425128      	498   revalidator35
*3235   OTHER 	0    	4	151123       	51       	pmd37*
*3236   OTHER 	0   	20	298967       	48       	pmd38*
3164   OTHER 	0    	6 	47575        	0  dpdk_watchdog3
3165   OTHER 	0    	6	237634        	0   vhost_thread1
3166   OTHER 	0    	6  	3665        	0       	urcu2