Chapter 3. Integrating with the Existing Ceph Cluster

The Heat template collection provided by the director already contains the necessary templates and environment files to deploy an overcloud.

This environment file is invoked during deployment (Section 3.4, “Deploying the Overcloud”) to integrate an existing Ceph cluster to the overcloud being deployed.

  • /usr/share/openstack-tripleo-heat-templates/environments/ceph-ansible/ceph-ansible-external.yaml

The director uses ceph-ansible to integrate with an existing Ceph cluster, but ceph-ansible is not installed by default on the undercloud. Run the following command to install the ceph-ansible package on the undercloud:

sudo dnf install -y ceph-ansible

To configure the integration, you must supply the details of your Ceph cluster to the director. To do this, use a custom environment file, which will allow you to override the default settings used by /usr/share/openstack-tripleo-heat-templates/environments/ceph-ansible/ceph-ansible-external.yaml:

  1. Create the following custom environment file:


  2. Add a parameter_defaults: header to this file:

  3. Under this header, set all the parameters you want to override in /usr/share/openstack-tripleo-heat-templates/environments/ceph-ansible/ceph-ansible-external.yaml. At a minimum, you need to set:

    • CephClientKey: the Ceph client key of your Ceph Storage cluster. This is the value of key you retrieved earlier in Section 2.2, “Configuring the Existing Ceph Storage Cluster” (for example, AQDLOh1VgEp6FRAAFzT7Zw+Y9V6JJExQAsRnRQ==).
    • CephClusterFSID: the file system ID of your Ceph Storage cluster. This is the value of fsid in your Ceph Storage cluster configuration file, which you retrieved earlier in Section 2.2, “Configuring the Existing Ceph Storage Cluster” (for example, 4b5c8c0a-ff60-454b-a1b4-9747aa737d19).
    • CephExternalMonHost: a comma-delimited list of the IPs of all MON hosts in your Ceph Storage cluster (for example,,

For example:

  CephClientKey: AQDLOh1VgEp6FRAAFzT7Zw+Y9V6JJExQAsRnRQ==
  CephClusterFSID: 4b5c8c0a-ff60-454b-a1b4-9747aa737d19

If necessary, also set the name of the OpenStack pools and the client user using the following parameters and values:

  • CephClientUserName: openstack
  • NovaRbdPoolName: vms
  • CinderRbdPoolName: volumes
  • GlanceRbdPoolName: images
  • CinderBackupRbdPoolName: backups
  • GnocchiRbdPoolName: metrics

You can also add overcloud parameters to your custom environment file. For example, to set vxlan as the neutron network type, add the following to parameter_defaults:

  NeutronNetworkType: vxlan

3.1. Backwards Compatibility with Older Versions of Red Hat Ceph Storage

If you are integrating Red Hat OpenStack Platform with an external Ceph Storage Cluster from an earlier version, you might need to enable backwards compatibility.

If you are running Red Hat Ceph Storage 1.3, you need to add the following line to the parameter_defaults of your custom environment file (in this case, /home/stack/templates/ceph-config from Chapter 3, Integrating with the Existing Ceph Cluster):

  RbdDefaultFeatures: 1

If you are running an external Red Hat Ceph Storage 2.x cluster, then it is not necessary to use this parameter. Though this version of Red Hat OpenStack Platform uses Red Hat Ceph Storage 3.x clients, those clients remain compatible with a Red Hat Ceph Storage 2.x server.

3.2. Assigning Nodes and Flavors to Roles

Planning an overcloud deployment involves specifying how many nodes and which flavors to assign to each role. Like all Heat template parameters, these role specifications are declared in the parameter_defaults section of your custom environment file (in this case, /home/stack/templates/ceph-config from Chapter 3, Integrating with the Existing Ceph Cluster):

For this purpose, use the following parameters:

Table 3.1. Roles and Flavors for Overcloud Nodes

Heat Template ParameterDescription


The number of Controller nodes to scale out


The flavor to use for Controller nodes (control)


The number of Compute nodes to scale out


The flavor to use for Compute nodes (compute)

For example, to configure the overcloud to deploy three nodes for each role (Controller and Compute), add the following to your parameter_defaults:

  ControllerCount: 3
  ComputeCount: 3
  OvercloudControlFlavor: control
  OvercloudComputeFlavor: compute

See Creating the Overcloud with the CLI Tools from the Director Installation and Usage guide for a more complete list of Heat template parameters.

3.3. Ceph containers for Red Hat OpenStack Platform with Ceph Storage

A Ceph container is required to configure OpenStack Platform to use Ceph, even with an external Ceph cluster. To be compatible with Red Hat Enterprise Linux 8, OpenStack Platform 15 requires Red Hat Ceph Storage 4. The Ceph Storage 4 container is hosted at, a registry which requires authentication.

You can use the heat environment parameter ContainerImageRegistryCredentials to authenticate at, as described in Container image preparation parameters.

3.4. Deploying the Overcloud


During undercloud installation, set generate_service_certificate=false in the undercloud.conf file. Otherwise, you must inject a trust anchor when you deploy the overcloud. For more information about how to inject a trust anchor, see Enabling SSL/TLS on Overcloud Public Endpoints in the Advanced Overcloud Customization guide.

The creation of the overcloud requires additional arguments for the openstack overcloud deploy command. For example:

$ openstack overcloud deploy --templates \
  -e /usr/share/openstack-tripleo-heat-templates/environments/ceph-ansible/ceph-ansible-external.yaml \
  -e /home/stack/templates/ceph-config.yaml \
  -e --ntp-server \

The above command uses the following options:

  • --templates - Creates the overcloud from the default Heat template collection (namely, /usr/share/openstack-tripleo-heat-templates/).
  • -e /usr/share/openstack-tripleo-heat-templates/environments/ceph-ansible/ceph-ansible-external.yaml - Sets the director to integrate an existing Ceph cluster to the overcloud.
  • -e /home/stack/templates/ceph-config.yaml - Adds a custom environment file to override the defaults set by -e /usr/share/openstack-tripleo-heat-templates/environments/ceph-ansible/ceph-ansible-external.yaml. In this case, it is the custom environment file you created in Chapter 3, Integrating with the Existing Ceph Cluster.
  • --ntp-server - Sets our NTP server.

You can also use an answers file to invoke all your templates and environment files. For example, you can use the following command to deploy an identical overcloud:

$ openstack overcloud deploy \
  --answers-file /home/stack/templates/answers.yaml \

In this case, the answers file /home/stack/templates/answers.yaml contains:

templates: /usr/share/openstack-tripleo-heat-templates/
  - /usr/share/openstack-tripleo-heat-templates/environments/ceph-ansible/ceph-ansible-external.yaml \
  - /home/stack/templates/ceph-config.yaml \

See Including environment files in an overcloud deployment for more details.

For a full list of options, run:

$ openstack help overcloud deploy

For more information, see Configuring a basic overcloud with the CLI tools in the Director Installation and Usage guide.

The overcloud creation process begins and the director provisions your nodes. This process takes some time to complete. To view the status of the Overcloud creation, open a separate terminal as the stack user and run:

$ source ~/stackrc
$ openstack stack list --nested

This configures the Overcloud to use your external Ceph Storage cluster. Note that you manage this cluster independently from the Overcloud. For example, you scale the Ceph Storage cluster using the Ceph management tools and not through the OpenStack Platform director.