Chapter 6. Scaling deployments with Compute cells

1. Extracting parameter information from the default first cell in the basic overcloud. This cell becomes the global Controller after you redeploy the overcloud.
2. Configuring a custom role and flavor for the cell.
3. Creating an environment file with cell-specific parameters.
4. Redeploying the overcloud with the new cell stack.

Extract parameter information from the overcloud

1. Create a new directory for the new cell and export the contents to the new directory. For example:

   $ source ~/stackrc
   (undercloud) $ mkdir cell1
   (undercloud) $ export DIR=cell1

2. Export the EndpointMap, HostsEntry, AllNodesConfig, GlobalConfig parameters, and the password information from the overcloud to a new environment file for the cell. For example:

   (undercloud) $ openstack overcloud cell export cell1 -o cell1/cell1-ctrl-input.yaml

   Note

   If the environment file already exists, run the command with the --force-overwrite or -f option.

Configure a custom role for a cell

1. Add the CellController role to your roles data file and regenerate the file. For example:

   (undercloud) $ openstack overcloud roles generate --roles-path \
                  /usr/share/openstack-tripleo-heat-templates/roles \
                  -o $DIR/cell_roles_data.yaml Compute CellController

   The CellController custom role includes the services from the default Compute role and additional configuration for the following services:

   • Galera database
   • RabbitMQ
   • nova-conductor
   • nova novnc proxy
   • nova metadata (only in case you set the NovaLocalMetadataPerCell parameter)

2. In case you want to divide your network between the global Controller and the cells, configure network access in the roles file that you created. For example:

   name: Compute
    description: |
      Basic Compute Node role
    CountDefault: 1
    # Create external Neutron bridge (unset if using ML2/OVS without DVR)
    tags:
      - external_bridge
    networks:
      InternalApi:
        subnet: internal_api_cell1
      Tenant:
        subnet: tenant_subnet
      Storage:
        subnet: storage_cell1
   ...
   - name: CellController
      description: |
        CellController role for the nova cell_v2 controller services
      CountDefault: 1
      tags:
        - primary
        - controller
      networks:
        External:
          subnet: external_cell1
        InternalApi:
          subnet: internal_api_cell1
        Storage:
          subnet: storage_cell1
        StorageMgmt:
          subnet: storage_mgmt_cell1
        Tenant:
          subnet: tenant_subnet

Configure a flavor and tag nodes to a cell

1. Create the cellcontroller flavor to tag nodes that you want to allocate to the cell. For example:

   (undercloud) $ openstack flavor create --id auto --ram 4096 --disk 40 --vcpus 1 cellcontroller
   (undercloud) $ openstack flavor set --property "cpu_arch"="x86_64" \
                   --property "capabilities:boot_option"="local" \
                   --property "capabilities:profile"="cellcontroller" \
                   --property "resources:CUSTOM_BAREMETAL=1" \
                   --property "resources:DISK_GB=0" \
                   --property "resources:MEMORY_MB=0" \
                   --property "resources:VCPU=0" \
                  cellcontroller

2. Tag each node that you want to assign to the cell with the cellcontroller profile.

   (undercloud) $ openstack baremetal node set --property \
                  capabilities='profile:cellcontroller,boot_option:local' <NODE_UUID>

   Replace <NODE_UUID> with the actual ID of the Compute node that you want to assign to the cell.

Create environment files with cell parameters

1. Create a new environment file in the directory for the cell, such as /cell1/cell1.yaml, and add the following parameters:

   resource_registry:
     # since the same networks are used in this example, the
     # creation of the different networks is omitted
     OS::TripleO::Network::External: OS::Heat::None
     OS::TripleO::Network::InternalApi: OS::Heat::None
     OS::TripleO::Network::Storage: OS::Heat::None
     OS::TripleO::Network::StorageMgmt: OS::Heat::None
     OS::TripleO::Network::Tenant: OS::Heat::None
     OS::TripleO::Network::Management: OS::Heat::None
     OS::TripleO::Network::Ports::OVNDBsVipPort: /usr/share/openstack-tripleo-heat-templates/network/ports/noop.yaml
     OS::TripleO::Network::Ports::RedisVipPort: /usr/share/openstack-tripleo-heat-templates/network/ports/noop.yaml
   
   parameter_defaults:
     # CELL Parameter to reflect that this is an additional CELL
     NovaAdditionalCell: True
   
     # mapping of the CellController flavor to the CellController role
     CellControllerFlavor: cellcontroller
   
     # The DNS names for the VIPs for the cell
     CloudName: cell1.ooo.test
     CloudNameInternal: cell1.internalapi.ooo.test
     CloudNameStorage: cell1.storage.ooo.test
     CloudNameStorageManagement: cell1.storagemgmt.ooo.test
     CloudNameCtlplane: cell1.ctlplane.ooo.test
   
     # Flavors used for the cell controller and computes
     OvercloudCellControllerFlavor: cellcontroller
     OvercloudComputeFlavor: compute
   
     # Number of controllers/computes in the cell
     CellControllerCount: 1
     ComputeCount: 1
   
     # Compute node name (must be unique)
     ComputeHostnameFormat: 'cell1-compute-%index%'
   
     # default gateway
     ControlPlaneStaticRoutes:
       - ip_netmask: 0.0.0.0/0
         next_hop: 192.168.24.1
         default: true
     DnsServers:
       - x.x.x.x

   Change the parameter values in this example according to your deployment needs.

2. Depending on your network configuration, you might need to allocate a network resource to the cell. Add the following parameter if you need to register cells to the network:

   resource_registry:
     OS::TripleO::CellController::Net::SoftwareConfig: single-nic-vlans/controller.yaml
     OS::TripleO::Compute::Net::SoftwareConfig: single-nic-vlans/compute.yaml

3. If you divide your network between the global Controller and the cells and want to run the Compute metadata API in each cell instead of in the global Controller, add the following parameter:

   parameter_defaults:
      NovaLocalMetadataPerCell: True

   Note

   • The parameters in this file restrict the overcloud to use a single network for all cells.
   • The Compute host names must be unique across all cells.

4. Copy the network_data.yaml file and name it according to the cell name. For example:

   (undercloud) $ cp /usr/share/openstack-tripleo-heat-templates/network_data.yaml cell1/network_data-ctrl.yaml

5. Add the UUIDs for the network components you want to reuse for the cells to the new network data file.

   external_resource_network_id: [EXISTING_NETWORK_UUID]
   external_resource_subnet_id: [EXISTING_SUBNET_UUID]
   external_resource_segment_id: [EXISTING_SEGMENT_UUID]
   external_resource_vip_id: [EXISTING_VIP_UUID]

1. Get the IP addresses of the control plane and cell controller.

   $ CTRL_IP=$(openstack server list -f value -c Networks --name overcloud-controller-0 | sed 's/ctlplane=//')
   $ CELL_CTRL_IP=$(openstack server list -f value -c Networks --name cellcontroller-0 | sed 's/ctlplane=//')

2. Add the cell information to all Controller nodes. This information is used to connect to the cell endpoint from the undercloud.

   (undercloud) [stack@undercloud ~]$ CELL_INTERNALAPI_INFO=$(ssh heat-admin@${CELL_CTRL_IP} egrep \
                                       cellcontrol.*\.internalapi /etc/hosts)
   (undercloud) [stack@undercloud ~]$ ansible -i /usr/bin/tripleo-ansible-inventory Controller -b \
                                       -m lineinfile -a "dest=/etc/hosts line=\"$CELL_INTERNALAPI_INFO\""

3. Get the transport_url and database.connection endpoint information from the controller cell.

   (undercloud) [stack@undercloud ~]$ CELL_TRANSPORT_URL=$(ssh heat-admin@${CELL_CTRL_IP} sudo \
                                       crudini --get /var/lib/config-data/nova/etc/nova/nova.conf DEFAULT transport_url)
   (undercloud) [stack@undercloud ~]$ CELL_MYSQL_VIP=$(ssh heat-admin@${CELL_CTRL_IP} sudo \
                                       crudini --get /var/lib/config-data/nova/etc/nova/nova.conf database connection \
                                       | perl -nle'/(\d+\.\d+\.\d+\.\d+)/ && print $1')

4. Log in to one of the global Controller nodes to create the cell based on the information that you retrieved in the previous steps. For example:

   $ export CONTAINERCLI='podman'
   
   $ ssh heat-admin@${CTRL_IP} sudo ${CONTAINERCLI} exec -i -u root nova_api \
     nova-manage cell_v2 create_cell --name computecell1 \
     --database_connection "{scheme}://{username}:{password}@$CELL_MYSQL_VIP/nova?{query}" \
     --transport-url "$CELL_TRANSPORT_URL"

5. Check that the cell is created and appears in the cell list.

   $ ssh heat-admin@${CTRL_IP} sudo ${CONTAINERCLI} exec -i -u root nova_api \
   nova-manage cell_v2 list_cells --verbose

6. Restart the Compute services on the Controller nodes.

   $ ansible -i /usr/bin/tripleo-ansible-inventory Controller -b -a \
   "systemctl restart tripleo_nova_api tripleo_nova_conductor tripleo_nova_scheduler"

7. Check that the cell controller services are provisioned.

   (overcloud) [stack@undercloud ~]$ nova service-list

1. Log into one of the Controller nodes.

2. Get the IP address of the control plane for the cell and run the host discovery command to expose and assign Compute hosts to the cell.

   $ CTRL=overcloud-controller-0
   $ CTRL_IP=$(openstack server list -f value -c Networks --name $CTRL | sed 's/ctlplane=//')
   
   $ export CONTAINERCLI='podman'
   
   $ ssh heat-admin@${CTRL_IP} sudo ${CONTAINERCLI} exec -i -u root nova_api \
     nova-manage cell_v2 discover_hosts --by-service --verbose

3. Verify that the Compute hosts were assigned to the cell.

   $ ssh heat-admin@${CTRL_IP} sudo ${CONTAINERCLI} exec -i -u root nova_api \
     nova-manage cell_v2 list_hosts

1. Delete all instances from the Compute nodes in the cell.

   Note

   Migrating instances between cells is not supported. You must delete the instances and re-create them in another cell.

2. On one of the global Controllers, delete all Compute nodes from the cell.

   $ CTRL=overcloud-controller-0
   $ CTRL_IP=$(openstack server list -f value -c Networks --name $CTRL | sed 's/ctlplane=//')
   
   $ export CONTAINERCLI='podman'
   
   $ ssh heat-admin@${CTRL_IP} sudo ${CONTAINERCLI} exec -i -u root nova_api \
     nova-manage cell_v2 list_hosts
   
   $ ssh heat-admin@${CTRL_IP} sudo ${CONTAINERCLI} exec -i -u root nova_api \
     nova-manage cell_v2 delete_host --cell_uuid <uuid> --host <compute>

3. Delete the resource providers for the cell from the Placement service, to ensure that the host name is available in case you want to add Compute nodes with the same host name to another cell later. For example:

   (undercloud) $ source ~/overcloudrc
   
   (overcloud) $ openstack resource provider list
               +--------------------------------------+---------------------------------------+------------+
               | uuid                                 | name                                  | generation |
               +--------------------------------------+---------------------------------------+------------+
               | 9cd04a8b-5e6c-428e-a643-397c9bebcc16 | computecell1-novacompute-0.site1.test |         11 |
               +--------------------------------------+---------------------------------------+------------+
   
   (overcloud) $ openstack resource provider delete 9cd04a8b-5e6c-428e-a643-397c9bebcc16

1. On one of the global Controllers, delete the cell.

   $ CTRL=overcloud-controller-0
   $ CTRL_IP=$(openstack server list -f value -c Networks --name $CTRL | sed 's/ctlplane=//')
   
   $ export CONTAINERCLI='podman'
   
   $ ssh heat-admin@${CTRL_IP} sudo ${CONTAINERCLI} exec -i -u root nova_api \
     nova-manage cell_v2 list_cells
   
   $ ssh heat-admin@${CTRL_IP} sudo ${CONTAINERCLI} exec -i -u root nova_api \
     nova-manage cell_v2 delete_cell --cell_uuid <uuid>

2. Delete the cell stack from the overcloud.

   $ openstack stack delete <stack name> --wait --yes && openstack overcloud plan delete <STACK_NAME>

   Note

   If you deployed separate cell stacks for a Controller and Compute cell, delete the Compute cell stack first and then the Controller cell stack.