Chapter 10. Using Composable Networks

With composable networks, you are no longer constrained by the pre-defined network segments (Internal, Storage, Storage Management, Tenant, External, Control Plane), and instead you can now create your own networks and assign them to any role: default or custom. For example, if you have a network dedicated to NFS traffic, you can now present it to multiple different roles.

Director supports the creation of custom networks during the deployment and update phases. These additional networks can be used for ironic bare metal nodes, system management, or to create separate networks for different roles. They can also be used to create multiple sets of networks for split deployments, where traffic is routed between networks.

A single data file (network_data.yaml) manages the list of networks that will be deployed; the role definition process then assigns the networks to the required roles through network isolation (see Chapter 9, Isolating Networks for more information).

10.1. Defining a Composable Network

To create composable networks, edit a local copy of the /usr/share/openstack-tripleo-heat-templates/network_data.yaml Heat template. For example:

- name: StorageBackup
  vip: true
  name_lower: storage_backup
  ip_subnet: ''
  allocation_pools: [{'start': '', 'end': ''}]
  gateway_ip: ''
  ipv6_subnet: 'fd00:fd00:fd00:7000::/64'
  ipv6_allocation_pools: [{'start': 'fd00:fd00:fd00:7000::10', 'end': 'fd00:fd00:fd00:7000:ffff:ffff:ffff:fffe'}]
  gateway_ipv6: 'fd00:fd00:fd00:7000::1'
  • name - is the only mandatory value, however you can also use name_lower to normalize names for readability. For example, changing InternalApi to internal_api.
  • vip:true will create a virtual IP address (VIP) on the new network, with the remaining parameters setting the defaults for the new network.
  • ip_subnet and allocation_pools will set the default IPv4 subnet and IP range for the network.
  • ipv6_subnet and ipv6_allocation_pools will set the default IPv6 subnets for the network.

You can override these defaults using an environment file (usually named network-environment.yaml). The sample network-environment.yaml file can be created after modifying the network_data.yaml file by running this command from the root of the director’s core Heat templates you are using (local copy of /usr/share/openstack-tripleo-heat-templates/):

[stack@undercloud ~/templates] $ ./tools/

10.1.1. Define Network Interface Configuration for Composable Networks

When using composable networks, the parameter definition for the network IP address must be added to the NIC configuration template used for each role, even if the network is not used on the role. See the directories in /usr/share/openstack-tripleo-heat-templates/network/config for examples of these NIC configurations. For instance, if a StorageBackup network is added to only the Ceph nodes, the following would need to be added to the resource definitions in the NIC configuration templates for all roles:

    default: ''
    description: IP address/subnet on the external network
    type: string

You may also create resource definitions for VLAN IDs and/or gateway IP, if needed:

  StorageBackupNetworkVlanID: # Override this via parameter_defaults in network_environment.yaml
    default: 60
    description: Vlan ID for the management network traffic.
    type: number
  StorageBackupDefaultRoute: # Override this via parameter_defaults in network_environment.yaml
	description: The default route of the storage backup network.
	type: string

The IpSubnet parameter for the custom network appears in the parameter definitions for each role. However, since the Ceph role is the only role that makes use of the StorageBackup network in our example, only the NIC configuration template for the Ceph role would make use of the StorageBackup parameters in the network_config section of the template.

            - type: interface
              name: nic1
              use_dhcp: false
              - ip_netmask:
                  Get_param: StorageBackupIpSubnet

10.1.2. Assign Composable Networks to Services

If vip: true is specified in the custom network definition, then it is possible to assign services to the network using the ServiceNetMap parameters. The custom network chosen for the service must exist on the role hosting the service. You can override the default networks by overriding the ServiceNetMap that is defined in /usr/share/openstack-tripleo-heat-templates/network/service_net_map.j2.yaml in your network_environment.yaml (or in a different environment file):

	NeutronTenantNetwork: tenant
	CeilometerApiNetwork: internal_api
	AodhApiNetwork: internal_api
	GnocchiApiNetwork: internal_api
	MongoDbNetwork: internal_api
	CinderApiNetwork: internal_api
	CinderIscsiNetwork: storage
	GlanceApiNetwork: storage
	GlanceRegistryNetwork: internal_api
	KeystoneAdminApiNetwork: ctlplane # Admin connection for Undercloud
	KeystonePublicApiNetwork: internal_api
	NeutronApiNetwork: internal_api
	HeatApiNetwork: internal_api
	NovaApiNetwork: internal_api
	NovaMetadataNetwork: internal_api
	NovaVncProxyNetwork: internal_api
	SwiftMgmtNetwork: storage_backup # Changed from storage_mgmt
	SwiftProxyNetwork: storage
	SaharaApiNetwork: internal_api
	HorizonNetwork: internal_api
	MemcachedNetwork: internal_api
	RabbitMqNetwork: internal_api
	RedisNetwork: internal_api
	MysqlNetwork: internal_api
	CephClusterNetwork: storage_backup # Changed from storage_mgmt
	CephPublicNetwork: storage
	ControllerHostnameResolveNetwork: internal_api
	ComputeHostnameResolveNetwork: internal_api
	BlockStorageHostnameResolveNetwork: internal_api
	ObjectStorageHostnameResolveNetwork: internal_api
	CephStorageHostnameResolveNetwork: storage

10.1.3. Define the Routed Networks

When using composable networks to deploy routed networks, you define routes and router gateways for use in the network configuration. You can create network routes and supernet routes to define which interface to use when routing traffic between subnets. For example, in a deployment where traffic is routed between the Compute and Controller roles, you may want to define supernets for sets of isolated networks. For instance, is a supernet that contains all networks beginning with 172.17, so the Internal API network used on the controllers might use and the Internal API network used on the Compute nodes might use On both roles, you would define a route to the supernet through the router gateway that is specific to the network used on the role.

The available parameters in network-environment.yaml:

    default: ''
    description: Supernet that contains Internal API subnets for all roles.
    type: string
    default: ''
    description: Router gateway on Internal API network
    type: string
    default: ''
    description: Router gateway on Internal API 2 network
    Type: string

These parameters can be used in the NIC configuration templates for the roles.

The controller uses the parameters for the InternalApi network in controller.yaml:

            - type: interface
              name: nic3
              use_dhcp: false
              - ip_netmask:
                  get_param: InternalApiIpSubnet
              - routes:
                    get_param: InternalApiSupernet
                    Get_param: InternalApiGateway

The compute role uses the parameters for the InternalApi2 network in compute.yaml:

            - type: interface
              name: nic3
              use_dhcp: false
              - ip_netmask:
                  get_param: InternalApi2IpSubnet
              - routes:
                    get_param: InternalApiSupernet
                    Get_param: InternalApi2Gateway

If specific network routes are not applied on isolated networks, all traffic to non-local networks use the default gateway. This is generally undesirable from both a security and performance standpoint since it mixes different kinds of traffic and puts all outbound traffic on the same interface. In addition, if the routing is asymmetric (traffic is sent through a different interface than received), it might cause unreachable services. Using a route to the supernet on both the client and server directs traffic to use the correct interface on both sides.