Chapter 17. Converting your existing deployment to use TLS

You can configure your existing overcloud and undercloud endpoints to use TLS encryption. This approach uses novajoin to integrate your deployment with Red Hat Identity Management (IdM), allowing access to DNS, Kerberos, and certmonger. Each overcloud node uses a certmonger client to retrieve certificates for each service.

For more information on TLS, see the Security and Hardening Guide.

17.1. Requirements

  • For Red Hat OpenStack Platform 13, you must be running version z8 or higher.
  • You must have an existing IdM deployment, and it must also supply DNS services to the OpenStack deployment.
  • The existing deployment must use FQDNs for public endpoints. Default configurations might use IP address-based endpoints, and will consequently generate IP address-based certificates; these must be changed to FQDNs before proceeding with these steps.
Important

The overcloud and undercloud services will be unavailable for the duration of this procedure.

17.2. Reviewing your endpoints

By default, your existing Red Hat OpenStack Platform 13 overcloud does not encrypt certain endpoints with TLS. For example, this output includes URLs that use http instead of https; these are not encrypted:

+----------------------------------+-----------+--------------+--------------+---------+-----------+--------------------------------------------------------+
| ID                               | Region    | Service Name | Service Type | Enabled | Interface | URL                                                    |
+----------------------------------+-----------+--------------+--------------+---------+-----------+--------------------------------------------------------+
| 0ad11e943e1f4ff988650cfba57b4031 | regionOne | nova         | compute      | True    | internal  | http://172.16.2.17:8774/v2.1                           |
| 1413eb9ef38a45b8bee1bee1b0dfe744 | regionOne | swift        | object-store | True    | public    | https://overcloud.lab.local:13808/v1/AUTH_%(tenant_id)s |
| 1a54f13f212044b0a20468861cd06f85 | regionOne | neutron      | network      | True    | public    | https://overcloud.lab.local:13696                       |
| 3477a3a052d2445697bb6642a8c26a91 | regionOne | placement    | placement    | True    | internal  | http://172.16.2.17:8778/placement                      |
| 3f56445c0dd14721ac830d6afb2c2cd4 | regionOne | nova         | compute      | True    | admin     | http://172.16.2.17:8774/v2.1                           |
| 425b1773a55c4245bcbe3d051772ebba | regionOne | glance       | image        | True    | internal  | http://172.16.2.17:9292                                |
| 57cf09fa33ed446f8736d4228bdfa881 | regionOne | placement    | placement    | True    | public    | https://overcloud.lab.local:13778/placement             |
| 58600f3751e54f7e9d0a50ba618e4c54 | regionOne | glance       | image        | True    | public    | https://overcloud.lab.local:13292                       |
| 5c52f273c3284b068f2dc885c77174ca | regionOne | neutron      | network      | True    | internal  | http://172.16.2.17:9696                                |
| 8792a4dd8bbb456d9dea4643e57c43dc | regionOne | nova         | compute      | True    | public    | https://overcloud.lab.local:13774/v2.1                  |
| 94bbea97580a4c4b844478aad5a85e84 | regionOne | keystone     | identity     | True    | public    | https://overcloud.lab.local:13000                       |
| acbf11b5c76d44198af49e3b78ffedcd | regionOne | swift        | object-store | True    | internal  | http://172.16.1.9:8080/v1/AUTH_%(tenant_id)s           |
| d4a1344f02a74f7ab0a50c5a7c13ca5c | regionOne | keystone     | identity     | True    | internal  | http://172.16.2.17:5000                                |
| d86c241dc97642419ddc12533447d73d | regionOne | placement    | placement    | True    | admin     | http://172.16.2.17:8778/placement                      |
| de7d6c34533e4298a2752852427a7030 | regionOne | glance       | image        | True    | admin     | http://172.16.2.17:9292                                |
| e82086062ebd4d4b9e03c7f1544bdd3b | regionOne | swift        | object-store | True    | admin     | http://172.16.1.9:8080                                 |
| f8134cd9746247bca6a06389b563c743 | regionOne | keystone     | identity     | True    | admin     | http://192.168.24.6:35357                              |
| fe29177bd29545ca8fdc0c777a7cf03f | regionOne | neutron      | network      | True    | admin     | http://172.16.2.17:9696                                |
+----------------------------------+-----------+--------------+--------------+---------+-----------+--------------------------------------------------------+

The following sections explain how to encrypt these endpoints using TLS.

17.3. Apply workaround for known issue

There is currently a known issue for TLS Everywhere in-place upgrades, where overcloud nodes are consequently unable to enroll in IdM. As a workaround, remove /etc/ipa/ca.crt/ from all overcloud nodes before running the overcloud deploy. For more information, see https://bugzilla.redhat.com/show_bug.cgi?id=1732564.

For example, the following script is one way of applying the workaround. You might need to amend this to suit your deployment.

[stack@undercloud-0 ~]$ vi rm-ca.crt-dir.sh
#!/bin/bash

source /home/stack/stackrc
NODES=$(openstack server list -f value -c Networks|sed s/ctlplane=//g)

for NODE in $NODES
do
    ssh heat-admin@$NODE sudo rm -rf /etc/ipa/ca.crt/
Done

[stack@undercloud-0 ~]$ bash rm-ca.crt-dir.sh

17.4. Configuring endpoints to use TLS

This section explains how to enable TLS endpoint encryption for an existing deployment, and then how to check that the endpoints have been correctly configured.

When enabling TLS everywhere, there are different upgrade paths available, depending on how your domains are structured. These examples use sample domain names to describe the upgrade paths:

  • Reuse the existing public endpoint certificates, and enable TLS everywhere on the internal and admin endpoints where the overcloud domain (lab.local) matches the IdM domain (lab.local).
  • Allow IdM to issue new public endpoints certificates, and enable TLS everywhere on the internal and admin endpoints where the overcloud domain (lab.local) matches the IdM domain (lab.local).
  • Reuse existing public endpoint certificates, and enable TLS everywhere on the internal and admin endpoints where the overcloud domain (site1.lab.local) is a subdomain of the IdM domain (lab.local).
  • Allow IdM to issue new public endpoints certificates, and enable TLS everywhere on the internal and admin endpoints where the overcloud domain (site1.lab.local) is a subdomain of the IdM domain (lab.local).

The following procedures in this section explain how to configure this integration using the various combinations described above.

17.4.1. Configuring undercloud integration for deployments using the same domain as IdM

This procedure describes how to configure undercloud integration for deployments that use the same domain as IdM.

Red Hat OpenStack Platform uses novajoin to integrate with Red Hat Identity Management (IdM), which then issues and manages encryption certificates. In this procedure, you register the undercloud with IdM, generate a token, enable the token in the undercloud configuration, then re-run the undercloud and overcloud deployment scripts. For example:

  1. Install python-novajoin for integration with IdM:

    [stack@undercloud-0 ~]$ sudo yum install python-novajoin
  2. Run the novajoin configuration script and supply the configuration details for your IdM deployment. For example:

    [stack@undercloud-0 ~]$ sudo novajoin-ipa-setup --principal admin --password ComplexRedactedPassword \
      --server ipa.lab.local --realm lab.local --domain lab.local \
      --hostname undercloud-0.lab.local --precreate
    ...
    0Uvua6NyIWVkfCSTOmwbdAobsqGH2GONRJrW24MoQ4wg

    This output includes a one time password (OTP) for IdM, which will be a different value for your deployment.

  3. Configure the undercloud to use novajoin, add the one-time password (OTP), use the IdM IP address for DNS, and describe the overcloud domain. You will need to adjust this example for your deployment:

    [stack@undercloud ~]$ vi undercloud.conf
    ...
    enable_novajoin = true
    ipa_otp = 0Uvua6NyIWVkfCSTOmwbdAobsqGH2GONRJrW24MoQ4wg
    undercloud_hostname = undercloud-0.lab.local
    undercloud_nameservers = X.X.X.X
    overcloud_domain_name = lab.local
    ...
  4. Install the novajoin services in the undercloud:

    [stack@undercloud ~]$ openstack undercloud install
  5. Add the overcloud IP address to DNS. You will need to amend this example to suit your deployment:

    Note: Check the overcloud’s network-environment.yaml, and choose a VIP within each network’s range.

    [root@ipa ~]$ ipa dnsrecord-add lab.local overcloud --a-rec=10.0.0.101
    [root@ipa ~]# ipa dnszone-add ctlplane.lab.local
    [root@ipa ~]# ipa dnsrecord-add ctlplane.lab.local overcloud --a-rec 192.168.24.101
    [root@ipa ~]# ipa dnszone-add internalapi.lab.local
    [root@ipa ~]# ipa dnsrecord-add internalapi.lab.local overcloud --a-rec 172.17.1.101
    [root@ipa ~]# ipa dnszone-add storage.lab.local
    [root@ipa ~]# ipa dnsrecord-add storage.lab.local overcloud --a-rec 172.17.3.101
    [root@ipa ~]# ipa dnszone-add storagemgmt.lab.local
    [root@ipa ~]# ipa dnsrecord-add storagemgmt.lab.local overcloud --a-rec 172.17.4.101
  6. Create a public_vip.yaml mapping for all the endpoints:

    Parameter_defaults:
        PublicVirtualFixedIPs: [{'ip_address':'10.0.0.101'}]
        ControlFixedIPs: [{'ip_address':'192.168.24.101'}]
        InternalApiVirtualFixedIPs: [{'ip_address':'172.17.1.101'}]
        StorageVirtualFixedIPs: [{'ip_address':'172.17.3.101'}]
        StorageMgmtVirtualFixedIPs: [{'ip_address':'172.17.4.101'}]
        RedisVirtualFixedIPs: [{'ip_address':'172.17.1.102'}]

17.4.2. Configuring overcloud integration for deployments that use the same domain as IdM, and retain the existing public endpoint certificates

  1. Make sure the following parameters exist in your openstack overcloud deploy command (with valid settings) and then re-run the deployment command:

    • ` --ntp-server` - If not already set, specify the NTP server to suit your environment. The IdM server should be running ntp.
    • cloud-names.yaml - Contains the FQDNs (not IPs) from the initial deployment command.
    • enable-tls.yaml - Contains the new overcloud certificate. For an example, see https://github.com/openstack/tripleo-heat-templates/blob/master/environments/ssl/enable-tls.yaml.
    • public_vip.yaml - The maps the endpoints to a specific ip so dns can match.
    • enable-internal-tls.yaml - Enables TLS for internal endpoints.
    • tls-everywhere-endpoints-dns.yaml - Configures TLS endpoints using DNS names. You can review the contents of this file to check the configuration scope.
    • haproxy-internal-tls-certmonger.yaml - certmonger will manage the internal certs in haproxy.
    • inject-trust-anchor.yaml - Adds the root certificate authority. This is only needed when the certificates rely on a CA chain that is not already part of the common set used by default; for example, when using self-signed.

      For example:

      [ stack@undercloud ~]$ openstack overcloud deploy \
      ...
        --ntp-server 10.13.57.78 \
        -e /home/stack/cloud-names.yaml \
        -e /home/stack/enable-tls.yaml \
        -e /home/stack/public_vip.yaml \
        -e <tripleo-heat-templates>/environments/ssl/enable-internal-tls.yaml \
        -e <tripleo-heat-templates>/environments/ssl/tls-everywhere-endpoints-dns.yaml \
        -e <tripleo-heat-templates>/environments/services/haproxy-internal-tls-certmonger.yaml \
        -e /home/stack/inject-trust-anchor.yaml
      ...
      Note

      Examples of these environment files can be found here: https://github.com/openstack/tripleo-heat-templates/tree/master/environments/ssl.

17.4.3. Configuring overcloud integration for deployments that use the same domain as IdM, and replace the existing public endpoint certificates with an IdM generated certificate

  1. Make sure the following parameters exist in your openstack overcloud deploy command (with valid settings) and then re-run the deployment command:

    • ` --ntp-server` - If not already set, specify the NTP server to suit your environment. The IdM server should be running ntp.
    • cloud-names.yaml - Contains the FQDNs (not IPs) from the initial deployment command.
    • enable-tls.yaml - Contains the new overcloud certificate. For an example, see https://github.com/openstack/tripleo-heat-templates/blob/master/environments/ssl/enable-tls.yaml.
    • public_vip.yaml - The maps the endpoints to a specific ip so dns can match.
    • enable-internal-tls.yaml - Enables TLS for internal endpoints.
    • tls-everywhere-endpoints-dns.yaml - Configures TLS endpoints using DNS names. You can review the contents of this file to check the configuration scope.
    • haproxy-public-tls-certmonger.yaml - certmonger will manage the internal and public certs in haproxy.
    • inject-trust-anchor.yaml - Adds the root certificate authority. This is only needed when the certificates rely on a CA chain that is not already part of the common set used by default; for example, when using self-signed.

      For example:

      [ stack@undercloud ~]$ openstack overcloud deploy \
      ...
        --ntp-server 10.13.57.78 \
        -e /home/stack/cloud-names.yaml \
        -e /home/stack/enable-tls.yaml \
        -e /home/stack/public_vip.yaml \
        -e <tripleo-heat-templates>/environments/ssl/enable-internal-tls.yaml \
        -e <tripleo-heat-templates>/environments/ssl/tls-everywhere-endpoints-dns.yaml \
        -e <tripleo-heat-templates>/environments/services/haproxy-public-tls-certmonger.yaml \
        -e /home/stack/inject-trust-anchor.yaml
      ...
      Note

      Examples of these environment files can be found at https://github.com/openstack/tripleo-heat-templates/tree/master/environments/ssl.

Note

The template enable-internal-tls.j2.yaml is referenced as enable-internal-tls.yaml in the overcloud deploy command.

In addition, the old public endpoint certificates in enable-tls.yaml will be replaced by certmonger with haproxy-public-tls-certmonger.yaml, however, this file must still be referenced in the upgrade process.

17.4.4. Configuring undercloud integration for deployments that use an IdM subdomain

This procedure explains how to configure undercloud integration for deployments that use an IdM subdomain.

Red Hat OpenStack Platform uses novajoin to integrate with Red Hat Identity Management (IdM), which then issues and manages encryption certificates. In this procedure, you register the undercloud with IdM, generate a token, enable the token in the undercloud configuration, then re-run the undercloud and overcloud deployment scripts. For example:

  1. Install python-novajoin for integration with IdM:

    [stack@undercloud-0 ~]$
  2. Run the novajoin configuration script and supply the configuration details for your IdM deployment. For example:

    [stack@undercloud-0 ~]$ sudo novajoin-ipa-setup --principal admin --password ComplexRedactedPassword \
      --server ipa.lab.local --realm lab.local --domain lab.local \
      --hostname undercloud-0.site1.lab.local --precreate
    ...
    0Uvua6NyIWVkfCSTOmwbdAobsqGH2GONRJrW24MoQ4wg

    This output includes a one time password (OTP) for IdM, which will be a different value for your deployment.

  3. Configure the undercloud to use novajoin, and add the OTP, IdM IP for DNS and NTP, and overcloud domain:

    [stack@undercloud ~]$ vi undercloud.conf
    …
    [DEFAULT]
    undercloud_ntp_servers=X.X.X.X
    hieradata_override = /home/stack/hiera_override.yaml
    enable_novajoin = true
    ipa_otp = 0Uvua6NyIWVkfCSTOmwbdAobsqGH2GONRJrW24MoQ4wg
    undercloud_hostname = undercloud-0.site1.lab.local
    undercloud_nameservers = X.X.X.X
    overcloud_domain_name = site1.lab.local
    ...
  4. Configure the undercloud to use novajoin, and add the OTP, IdM IP for DNS, and overcloud domain:

    [stack@undercloud-0 ~]$ vi hiera_override.yaml
    nova::metadata::novajoin::api::ipa_domain: site1.lab.local
    ...
  5. Install the novajoin services in the undercloud:

    [stack@undercloud ~]$ openstack undercloud install
  6. Add the overcloud IP address to DNS. You will need to amend this example to suit your deployment:

    Note: Check the overcloud’s network-environment.yaml, and choose a VIP within each network’s range.

    [root@ipa ~]$ ipa dnsrecord-add site1.lab.local overcloud --a-rec=10.0.0.101
    [root@ipa ~]# ipa dnszone-add site1.ctlplane.lab.local
    [root@ipa ~]# ipa dnsrecord-add site1.ctlplane.lab.local overcloud --a-rec 192.168.24.101
    [root@ipa ~]# ipa dnszone-add site1.internalapi.lab.local
    [root@ipa ~]# ipa dnsrecord-add site1.internalapi.lab.local overcloud --a-rec 172.17.1.101
    [root@ipa ~]# ipa dnszone-add site1.storage.lab.local
    [root@ipa ~]# ipa dnsrecord-add site1.storage.lab.local overcloud --a-rec 172.17.3.101
    [root@ipa ~]# ipa dnszone-add site1.storagemgmt.lab.local
    [root@ipa ~]# ipa dnsrecord-add site1.storagemgmt.lab.local overcloud --a-rec 172.17.4.101
  7. Create a public_vip.yaml mapping for each of the endpoints. For example:

    Parameter_defaults:
        PublicVirtualFixedIPs: [{'ip_address':'10.0.0.101'}]
        ControlFixedIPs: [{'ip_address':'192.168.24.101'}]
        InternalApiVirtualFixedIPs: [{'ip_address':'172.17.1.101'}]
        StorageVirtualFixedIPs: [{'ip_address':'172.17.3.101'}]
        StorageMgmtVirtualFixedIPs: [{'ip_address':'172.17.4.101'}]
        RedisVirtualFixedIPs: [{'ip_address':'172.17.1.102'}]
  8. Create the extras.yaml mapping for each of the endpoints. For example:

    parameter_defaults:
      MakeHomeDir: True
      IdMNoNtpSetup: false
      IdMDomain: redhat.local
      DnsSearchDomains: "site1.redhat.local,redhat.local"

17.4.5. Configuring undercloud integration for deployments that use an IdM subdomain, and retain the existing public endpoint certificates

This procedure explains how to configure undercloud integration for deployments that use an IdM subdomain, and still retain the existing public endpoint certificates.

  1. Make sure the following parameters exist in your openstack overcloud deploy command (with valid settings) and then re-run the deployment command:

    • ` --ntp-server` - If not already set, specify the NTP server to suit your environment. The IdM server should be running ntp.
    • cloud-names.yaml - Contains the FQDNs (not IPs) from the initial deployment command.
    • enable-tls.yaml - Contains the new overcloud certificate. For an example, see https://github.com/openstack/tripleo-heat-templates/blob/master/environments/ssl/enable-tls.yaml.
    • public_vip.yaml - Contains endpoint maps to a specific ip so dns can match.
    • `extras.yaml ` - Contains settings for pam to make home directorys on login, no ntp setup, the base IdM domain, and the dns search for resolv.conf.
    • enable-internal-tls.yaml - Enables TLS for internal endpoints.
    • tls-everywhere-endpoints-dns.yaml - Configures TLS endpoints using DNS names. You can review the contents of this file to check the configuration scope.
    • haproxy-internal-tls-certmonger.yaml - certmonger will manage the internal certs in haproxy.
    • inject-trust-anchor.yaml - Adds the root certificate authority. This is only needed when the certificates rely on a CA chain that is not already part of the common set used by default; for example, when using self-signed.

      For example:

      [ stack@undercloud ~]$ openstack overcloud deploy \
      ...
        --ntp-server 10.13.57.78 \
        -e /home/stack/cloud-names.yaml \
        -e /home/stack/enable-tls.yaml \
        -e /home/stack/public_vip.yaml \
        -e /home/stack/extras.yaml \
        -e <tripleo-heat-templates>/environments/ssl/enable-internal-tls.yaml \
        -e <tripleo-heat-templates>/environments/ssl/tls-everywhere-endpoints-dns.yaml \
        -e <tripleo-heat-templates>/environments/services/haproxy-internal-tls-certmonger.yaml \
        -e /home/stack/inject-trust-anchor.yaml
      ...
      Note

      Examples of these environment files can be found here: https://github.com/openstack/tripleo-heat-templates/tree/master/environments/ssl.

17.4.6. Configuring undercloud integration for deployments that use an IdM subdomain, and replace the existing public endpoint certificates with an IdM generated certificate

This procedure explains how to configure undercloud integration for deployments that use an IdM subdomain, and how to replace the existing public endpoint certificates with an IdM generated certificate.

  1. Make sure the following parameters exist in your openstack overcloud deploy command (with valid settings) and then re-run the deployment command:

    • ` --ntp-server` - If not already set, specify the NTP server to suit your environment. The IdM server should be running ntp.
    • cloud-names.yaml - Contains the FQDNs (not IPs) from the initial deployment command.
    • enable-tls.yaml - Contains the new overcloud certificate. For an example, see https://github.com/openstack/tripleo-heat-templates/blob/master/environments/ssl/enable-tls.yaml.
    • public_vip.yaml - The maps the endpoints to a specific ip so dns can match.
    • `extras.yaml ` - Contains settings for pam to make home directorys on login, no ntp setup, the base IdM domain, and the dns search for resolv.conf.
    • enable-internal-tls.yaml - Enables TLS for internal endpoints.
    • tls-everywhere-endpoints-dns.yaml - Configures TLS endpoints using DNS names. You can review the contents of this file to check the configuration scope.
    • haproxy-public-tls-certmonger.yaml - certmonger will manage the internal and public certs in haproxy.
    • inject-trust-anchor.yaml - Adds the root certificate authority. This is only needed when the certificates rely on a CA chain that is not already part of the common set used by default; for example, when using self-signed.

      For example:

      [ stack@undercloud ~]$ openstack overcloud deploy \
      ...
        --ntp-server 10.13.57.78 \
        -e /home/stack/cloud-names.yaml \
        -e /home/stack/enable-tls.yaml \
        -e /home/stack/public_vip.yaml \
        -e /home/stack/extras.yaml \
        -e <tripleo-heat-templates>/environments/ssl/enable-internal-tls.yaml \
        -e <tripleo-heat-templates>/environments/ssl/tls-everywhere-endpoints-dns.yaml \
        -e <tripleo-heat-templates>/environments/services/haproxy-public-tls-certmonger.yaml \
        -e /home/stack/inject-trust-anchor.yaml
      ...
      Note

      Examples of these environment files can be found here: https://github.com/openstack/tripleo-heat-templates/tree/master/environments/ssl.

Note

In this example, the template enable-internal-tls.j2.yaml is referenced as enable-internal-tls.yaml in the overcloud deploy command. In addition, the old public endpoint certificates in enable-tls.yaml will be replaced by certmonger using haproxy-public-tls-certmonger.yaml, however, this file must still be referenced in the upgrade process.

17.5. Checking TLS encryption

Once the overcloud re-deployment has completed, check that all endpoints are now encrypted with TLS. In this example, all endpoints are configured to use https, indicating that they are using TLS encryption:

+----------------------------------+-----------+--------------+----------------+---------+-----------+---------------------------------------------------------------+
| ID                               | Region    | Service Name | Service Type   | Enabled | Interface | URL                                                           |
+----------------------------------+-----------+--------------+----------------+---------+-----------+---------------------------------------------------------------+
| 0fee4efdc4ae4310b6a139a25d9c0d9c | regionOne | neutron      | network        | True    | public    | https://overcloud.lab.local:13696                              |
| 220558ab1d2445139952425961a0c89a | regionOne | glance       | image          | True    | public    | https://overcloud.lab.local:13292                              |
| 24d966109ffa419da850da946f19c4ca | regionOne | placement    | placement      | True    | admin     | https://overcloud.internalapi.lab.local:8778/placement         |
| 27ac9e0d22804ee5bd3cd8c0323db49c | regionOne | nova         | compute        | True    | internal  | https://overcloud.internalapi.lab.local:8774/v2.1              |
| 31d376853bd241c2ba1a27912fc896c6 | regionOne | swift        | object-store   | True    | admin     | https://overcloud.storage.lab.local:8080                       |
| 350806234c784332bfb8615e721057e3 | regionOne | nova         | compute        | True    | admin     | https://overcloud.internalapi.lab.local:8774/v2.1              |
| 49c312f4db6748429d27c60164779302 | regionOne | keystone     | identity       | True    | public    | https://overcloud.lab.local:13000                              |
| 4e535265c35e486e97bb5a8bc77708b6 | regionOne | nova         | compute        | True    | public    | https://overcloud.lab.local:13774/v2.1                         |
| 5e93dd46b45f40fe8d91d3a5d6e847d3 | regionOne | keystone     | identity       | True    | admin     | https://overcloud.ctlplane.lab.local:35357                     |
| 6561984a90c742a988bf3d0acf80d1b6 | regionOne | swift        | object-store   | True    | public    | https://overcloud.lab.local:13808/v1/AUTH_%(tenant_id)s        |
| 76b8aad0bdda4313a02e4342e6a19fd6 | regionOne | placement    | placement      | True    | public    | https://overcloud.lab.local:13778/placement                    |
| 96b004d5217c4d87a38cb780607bf9fb | regionOne | placement    | placement      | True    | internal  | https://overcloud.internalapi.lab.local:8778/placement         |
| 98489b4b107f4da596262b712c3fe883 | regionOne | glance       | image          | True    | internal  | https://overcloud.internalapi.lab.local:9292                   |
| bb7ab36f30b14b549178ef06ec74ff84 | regionOne | glance       | image          | True    | admin     | https://overcloud.internalapi.lab.local:9292                   |
| c1547f7bf9a14e9e85eaaaeea26413b7 | regionOne | neutron      | network        | True    | admin     | https://overcloud.internalapi.lab.local:9696                   |
| ca66f499ec544f42838eb78a515d9f1e | regionOne | keystone     | identity       | True    | internal  | https://overcloud.internalapi.lab.local:5000                   |
| df0181358c07431390bc66822176281d | regionOne | swift        | object-store   | True    | internal  | https://overcloud.storage.lab.local:8080/v1/AUTH_%(tenant_id)s |
| e420350ef856460991c3edbfbae917c1 | regionOne | neutron      | network        | True    | internal  | https://overcloud.internalapi.lab.local:9696                   |
+----------------------------------+-----------+--------------+----------------+---------+-----------+---------------------------------------------------------------+