Chapter 10. Custom composable networks

This chapter follows on from the concepts and procedures outlined in Chapter 9, Basic network isolation and shows how to configure the overcloud with an additional composable network. This includes:

  • The environment file to enable network isolation (/usr/openstack-tripleo-heat-templates/environments/network-isolation.yaml).
  • The environment file to configure network defaults (/usr/openstack-tripleo-heat-templates/environments/network-environment.yaml).
  • A custom network_data file to create additional networks outside of the defaults.
  • A custom roles_data file to assign custom networks to roles.
  • Templates to define your NIC layout for each node. The overcloud core template collection contains a set of defaults for different use cases.
  • An environment file to enable NICs. This example uses a a default file located in the environments directory.
  • Any additional environment files to customize your networking parameters. This example uses an environment file to customize OpenStack service mappings to composable networks.

The following content in this chapter shows how to define each of these aspects.

10.1. Composable networks

The overcloud uses a pre-defined set of network segments by default. These network segments are:

  • Control Plane
  • Internal API
  • Storage
  • Storage Management
  • Tenant
  • External
  • Management (optional)

Composable networks allow you to add networks for various services. For example, if you have a network dedicated to NFS traffic, you can present it to multiple roles.

Director supports the creation of custom networks during the deployment and update phases. These additional networks can be used for ironic bare metal nodes, system management, or to create separate networks for different roles. You can also use them to create multiple sets of networks for split deployments where traffic is routed between networks.

A single data file (network_data.yaml) manages the list of networks to be deployed. You include this file with your deployment command using the -n option. Without this option, the deployment uses the default file (/usr/share/openstack-tripleo-heat-templates/network_data.yaml).

10.2. Adding a composable network

This procedure shows how to add an additional composable network to your overcloud.


  1. Copy the default network_data file:

    $ cp /usr/share/openstack-tripleo-heat-templates/network_data.yaml /home/stack/.
  2. Edit the local copy of the network_data.yaml file and add a section for your new network. For example:

    - name: StorageBackup
      vip: true
      name_lower: storage_backup
      ip_subnet: ''
      allocation_pools: [{'start': '', 'end': ''}]
      gateway_ip: ''
    • name is the only mandatory value, however you can also use name_lower to normalize names for readability. For example, changing InternalApi to internal_api.
    • vip: true creates a virtual IP address (VIP) on the new network. This IP is used as the target IP for services listed in the service-to-network mapping parameter (ServiceNetMap). Note that VIPs are only used by roles that use Pacemaker. The overcloud’s load balancing service redirects traffic from these IPs to their respective service endpoint.
    • ip_subnet, allocation_pools, and gateway_ip set the default IPv4 subnet, IP range, and gateway for the network.

Include the custom network_data file with your deployment using the -n option. Without the -n option, the deployment command uses the default set of networks.

10.3. Including a composable network in a role

You can assign composable networks to the roles defined in your environment. For example, you might include a custom StorageBackup network with your Ceph Storage nodes.

This procedure shows how to add composable networks to a role in your overcloud.


  1. If you do not already have a custon roles_data file, copy the default to your home directory:

    $ cp /usr/share/openstack-tripleo-heat-templates/roles_data.yaml /home/stack/.
  2. Edit the custom roles_data file.
  3. Scroll to the role you want to add the composable network and add the network name to the list of networks. For example, to add the network to the Ceph Storage role, use the following snippet as a guide:

    - name: CephStorage
      description: |
        Ceph OSD Storage node role
        - Storage
        - StorageMgmt
        - StorageBackup
  4. After adding custom networks to their respective roles, save the file.

When running the openstack overcloud deploy command, include the roles_data file using the -r option. Without the -r option, the deployment command uses the default set of roles with their respective assigned networks.

10.4. Assigning OpenStack services to composable networks

Each OpenStack service is assigned to a default network type in the resource registry. These services are then bound to IP addresses within the network type’s assigned network. Although the OpenStack services are divided among these networks, the number of actual physical networks might differ as defined in the network environment file. You can reassign OpenStack services to different network types by defining a new network map in an environment file (for example, /home/stack/templates/service-reassignments.yaml). The ServiceNetMap parameter determines the network types used for each service.

For example, you can reassign the Storage Management network services to the Storage Backup Network by modifying the highlighted sections:

    SwiftMgmtNetwork: storage_backup
    CephClusterNetwork: storage_backup

Changing these parameters to storage_backup will place these services on the Storage Backup network instead of the Storage Management network. This means you only need to define a set of parameter_defaults for the Storage Backup network and not the Storage Management network.

The director merges your custom ServiceNetMap parameter definitions into a pre-defined list of defaults taken from ServiceNetMapDefaults and overrides the defaults. The director then returns the full list including customizations back to ServiceNetMap, which is used to configure network assignments for various services.

Service mappings only apply to networks that use vip: true in the network_data file for nodes that use Pacemaker. The overcloud’s load balancer redirects traffic from the VIPs to the specific service endpoints.


A full list of default services can be found in the ServiceNetMapDefaults parameter within /usr/share/openstack-tripleo-heat-templates/network/service_net_map.j2.yaml.

10.5. Enabling custom composable networks

This procedure show how to enable custom composable networks using one of the default NIC templates. In this case, it is the Single NIC with VLANs (single-nic-vlans).


  1. When running the openstack overcloud deploy command, make sure to include:

    • The custom network_data file.
    • The custom roles_data file with network-to-role assignments.
    • The rendered file name of the default network isolation configuration.
    • The rendered file name of the default network environment file.
    • The rendered file name of the default network interface configuration.
    • Any additional environment files related to your network, such as the service reassignments.

For example:

$ openstack overcloud deploy --templates \
    -n /home/stack/network_data.yaml \
    -r /home/stack/roles_data.yaml \
    -e /usr/share/openstack-tripleo-heat-templates/environments/network-isolation.yaml \
    -e /usr/share/openstack-tripleo-heat-templates/environments/network-environment.yaml \
    -e /usr/share/openstack-tripleo-heat-templates/environments/net-single-nic-with-vlans.yaml \
    -e /home/stack/templates/service-reassignments.yaml \

This deploys the composable networks, including your additional custom networks, across nodes in your overcloud.