Chapter 3. Configuring routed spine-leaf in the undercloud

Procedure

1. Log in to the undercloud as the stack user.

2. If you do not already have an undercloud.conf file, copy the sample template file:

   [stack@director ~]$ cp /usr/share/python-tripleoclient/undercloud.conf.sample ~/undercloud.conf

3. Edit the undercloud.conf file.

4. Set the following values in the [DEFAULT] section:

   1. Set local_ip to the undercloud IP on leaf0:

      local_ip = 192.168.10.1/24

   2. Set undercloud_public_host to the externally facing IP address of the undercloud:

      undercloud_public_host = 10.1.1.1

   3. Set undercloud_admin_host to the administration IP address of the undercloud. This IP address is usually on leaf0:

      undercloud_admin_host = 192.168.10.2

   4. Set local_interface to the interface to bridge for the local network:

      local_interface = eth1

   5. Set enable_routed_networks to true:

      enable_routed_networks = true

   6. Define your list of subnets using the subnets parameter. Define one subnet for each L2 segment in the routed spine and leaf:

      subnets = leaf0,leaf1,leaf2

   7. Specify the subnet associated with the physical L2 segment local to the undercloud using the local_subnet parameter:

      local_subnet = leaf0

   8. Set the value of undercloud_nameservers.

      undercloud_nameservers = 10.11.5.19,10.11.5.20

      Tip

      You can find the current IP addresses of the DNS servers that are used for the undercloud nameserver by looking in /etc/resolv.conf.

5. Create a new section for each subnet that you define in the subnets parameter:

   [leaf0]
   cidr = 192.168.10.0/24
   dhcp_start = 192.168.10.10
   dhcp_end = 192.168.10.90
   inspection_iprange = 192.168.10.100,192.168.10.190
   gateway = 192.168.10.1
   masquerade = False
   
   [leaf1]
   cidr = 192.168.11.0/24
   dhcp_start = 192.168.11.10
   dhcp_end = 192.168.11.90
   inspection_iprange = 192.168.11.100,192.168.11.190
   gateway = 192.168.11.1
   masquerade = False
   
   [leaf2]
   cidr = 192.168.12.0/24
   dhcp_start = 192.168.12.10
   dhcp_end = 192.168.12.90
   inspection_iprange = 192.168.12.100,192.168.12.190
   gateway = 192.168.12.1
   masquerade = False

6. Save the undercloud.conf file.

7. Run the undercloud installation command:

   [stack@director ~]$ openstack undercloud install

Procedure

1. Source the stackrc file:

   [stack@director ~]$ source ~/stackrc

2. Create flavors for each custom role:

   $ ROLES="control compute_leaf0 compute_leaf1 compute_leaf2 ceph-storage_leaf0 ceph-storage_leaf1 ceph-storage_leaf2"
   $ for ROLE in $ROLES; do openstack flavor create --id auto --ram <ram_size_mb> --disk <disk_size_gb> --vcpus <no_vcpus> $ROLE ; done
   $ for ROLE in $ROLES; do openstack flavor set --property "cpu_arch"="x86_64" --property "capabilities:boot_option"="local" --property resources:DISK_GB='0' --property resources:MEMORY_MB='0' --property resources:VCPU='0' $ROLE ; done

   • Replace <ram_size_mb> with the RAM of the bare metal node, in MB.
   • Replace <disk_size_gb> with the size of the disk on the bare metal node, in GB.
   • Replace <no_vcpus> with the number of CPUs on the bare metal node.

3. Retrieve a list of your nodes to identify their UUIDs:

   (undercloud)$ openstack baremetal node list

4. Tag each bare metal node to its leaf network and role by using a custom resource class:

   (undercloud)$ openstack baremetal node set \
    --resource-class baremetal.LEAF-ROLE <node>

   Replace <node> with the ID of the bare metal node.

   For example, enter the following command to tag a node with UUID 58c3d07e-24f2-48a7-bbb6-6843f0e8ee13 to the Compute role on Leaf2:

   (undercloud)$ openstack baremetal node set \
    --resource-class baremetal.COMPUTE-LEAF2 58c3d07e-24f2-48a7-bbb6-6843f0e8ee13

5. Associate each leaf network role flavor with the custom resource class:

   (undercloud)$ openstack flavor set \
    --property resources:CUSTOM_BAREMETAL_LEAF_ROLE=1 \
   <custom_role>

   To determine the name of a custom resource class that corresponds to a resource class of a Bare Metal Provisioning service node, convert the resource class to uppercase, replace each punctuation mark with an underscore, and prefix with CUSTOM_.

   Note

   A flavor can request only one instance of a bare metal resource class.

6. In the node-info.yaml file, specify the flavor that you want to use for each custom leaf role, and the number of nodes to allocate for each custom leaf role. For example, the following configuration specifies the flavor to use, and the number of nodes to allocate for the custom leaf roles compute_leaf0, compute_leaf1, compute_leaf2, ceph-storage_leaf0, ceph-storage_leaf1, and ceph-storage_leaf2:

   parameter_defaults:
     OvercloudControllerFlavor: control
     OvercloudComputeLeaf0Flavor: compute_leaf0
     OvercloudComputeLeaf1Flavor: compute_leaf1
     OvercloudComputeLeaf2Flavor: compute_leaf2
     OvercloudCephStorageLeaf0Flavor: ceph-storage_leaf0
     OvercloudCephStorageLeaf1Flavor: ceph-storage_leaf1
     OvercloudCephStorageLeaf2Flavor: ceph-storage_leaf2
     ControllerLeaf0Count: 3
     ComputeLeaf0Count: 3
     ComputeLeaf1Count: 3
     ComputeLeaf2Count: 3
     CephStorageLeaf0Count: 3
     CephStorageLeaf1Count: 3
     CephStorageLeaf2Count: 3

Procedure

1. Source the stackrc file:

   $ source ~/stackrc

2. Check the bare metal nodes:

   $ openstack baremetal node list

3. Ensure that the bare metal nodes are either in enroll or manageable state. If the bare metal node is not in one of these states, the command that sets the physical_network property on the baremetal port fails. To set all nodes to manageable state, run the following command:

   $ for node in $(openstack baremetal node list -f value -c Name); do openstack baremetal node manage $node --wait; done

4. Check which baremetal ports are associated with which baremetal node:

   $ openstack baremetal port list --node <node-uuid>

5. Set the physical-network parameter for the ports. In the example below, three subnets are defined in the configuration: leaf0, leaf1, and leaf2. The local_subnet is leaf0. Because the physical network for the local_subnet is always ctlplane, the baremetal port connected to leaf0 uses ctlplane. The remaining ports use the other leaf names:

   $ openstack baremetal port set --physical-network ctlplane <port-uuid>
   $ openstack baremetal port set --physical-network leaf1 <port-uuid>
   $ openstack baremetal port set --physical-network leaf2 <port-uuid>

6. Introspect the nodes before you deploy the overcloud. Include the --all-manageable and --provide options to set the nodes as available for deployment:

   $ openstack overcloud node introspect --all-manageable --provide

Procedure

1. Log in to the undercloud host as the stack user.

2. Source the undercloud credentials file:

   $ source ~/stackrc

3. In the /home/stack/undercloud.conf file, do the following:

   1. Locate the subnets parameter, and add a new subnet for the leaf that you are adding.

      A subnet represents an L2 segment in the routed spine and leaf:

      Example

      In this example, a new subnet (leaf3) is added for the new leaf (leaf3):

      subnets = leaf0,leaf1,leaf2,leaf3

   2. Create a section for the subnet that you added.

      Example

      In this example, the section [leaf3] is added for the new subnet (leaf3):

      [leaf0]
      cidr = 192.168.10.0/24
      dhcp_start = 192.168.10.10
      dhcp_end = 192.168.10.90
      inspection_iprange = 192.168.10.100,192.168.10.190
      gateway = 192.168.10.1
      masquerade = False
      
      [leaf1]
      cidr = 192.168.11.0/24
      dhcp_start = 192.168.11.10
      dhcp_end = 192.168.11.90
      inspection_iprange = 192.168.11.100,192.168.11.190
      gateway = 192.168.11.1
      masquerade = False
      
      [leaf2]
      cidr = 192.168.12.0/24
      dhcp_start = 192.168.12.10
      dhcp_end = 192.168.12.90
      inspection_iprange = 192.168.12.100,192.168.12.190
      gateway = 192.168.12.1
      masquerade = False
      
      [leaf3]
      cidr = 192.168.13.0/24
      dhcp_start = 192.168.13.10
      dhcp_end = 192.168.13.90
      inspection_iprange = 192.168.13.100,192.168.13.190
      gateway = 192.168.13.1
      masquerade = False

4. Save the undercloud.conf file.

5. Reinstall your undercloud:

   $ openstack undercloud install