Red Hat Training
A Red Hat training course is available for Red Hat OpenStack Platform
Chapter 3. Deploying Other Ceph Services on dedicated nodes
By default, the director deploys the Ceph MON and Ceph MDS services on the Controller nodes. This is suitable for small deployments. However, with larger deployments we advise that you deploy the Ceph MON and Ceph MDS services on dedicated nodes to improve the performance of your Ceph cluster. You can do this by creating a custom role for either one.
For more information about custom roles, see Creating a New Role in the Advanced Overcloud Customization guide.
The director uses the following file as a default reference for all overcloud roles:
-
/usr/share/openstack-tripleo-heat-templates/roles_data.yaml
Copy this file to /home/stack/templates/
so you can add custom roles to it:
$ cp /usr/share/openstack-tripleo-heat-templates/roles_data.yaml /home/stack/templates/roles_data_custom.yaml
You invoke the /home/stack/templates/roles_data_custom.yaml
file later during overcloud creation (Section 7.2, “Initiating overcloud deployment”). The following sub-sections describe how to configure custom roles for either Ceph MON and Ceph MDS services.
3.1. Creating a custom role and flavor for the Ceph MON service
This section describes how to create a custom role (named CephMon
) and flavor (named ceph-mon
) for the Ceph MON role. You should already have a copy of the default roles data file as described in Chapter 3, Deploying Other Ceph Services on dedicated nodes.
-
Open the
/home/stack/templates/roles_data_custom.yaml
file. - Remove the service entry for the Ceph MON service (namely, OS::TripleO::Services::CephMon) from under the Controller role.
Add the OS::TripleO::Services::CephClient service to the Controller role:
[...] - name: Controller # the 'primary' role goes first CountDefault: 1 ServicesDefault: - OS::TripleO::Services::CACerts - OS::TripleO::Services::CephMds - OS::TripleO::Services::CephClient - OS::TripleO::Services::CephExternal - OS::TripleO::Services::CephRbdMirror - OS::TripleO::Services::CephRgw - OS::TripleO::Services::CinderApi [...]
At the end of
roles_data_custom.yaml
, add a customCephMon
role containing the Ceph MON service and all the other required node services. For example:- name: CephMon ServicesDefault: # Common Services - OS::TripleO::Services::AuditD - OS::TripleO::Services::CACerts - OS::TripleO::Services::CertmongerUser - OS::TripleO::Services::Collectd - OS::TripleO::Services::Docker - OS::TripleO::Services::Fluentd - OS::TripleO::Services::Kernel - OS::TripleO::Services::Ntp - OS::TripleO::Services::ContainersLogrotateCrond - OS::TripleO::Services::SensuClient - OS::TripleO::Services::Snmp - OS::TripleO::Services::Timezone - OS::TripleO::Services::TripleoFirewall - OS::TripleO::Services::TripleoPackages - OS::TripleO::Services::Tuned # Role-Specific Services - OS::TripleO::Services::CephMon
Using the
openstack flavor create
command, define a new flavor namedceph-mon
for this role:$ openstack flavor create --id auto --ram 6144 --disk 40 --vcpus 4 ceph-mon
NoteFor more details about this command, run
openstack flavor create --help
.Map this flavor to a new profile, also named
ceph-mon
:$ openstack flavor set --property "cpu_arch"="x86_64" --property "capabilities:boot_option"="local" --property "capabilities:profile"="ceph-mon" ceph-mon
NoteFor more details about this command, run
openstack flavor set --help
.Tag nodes into the new
ceph-mon
profile:$ ironic node-update UUID add properties/capabilities='profile:ceph-mon,boot_option:local'
Add the following configuration to the
node-info.yaml
file to associate theceph-mon
flavor with the CephMon role:parameter_defaults: OvercloudCephMonFlavor: CephMon CephMonCount: 3
See Section 2.4, “Manually tagging the nodes” for more details about tagging nodes. See also Tagging Nodes Into Profiles for related information on custom role profiles.
3.2. Creating a custom role and flavor for the Ceph MDS service
This section describes how to create a custom role (named CephMDS
) and flavor (named ceph-mds
) for the Ceph MDS role. You should already have a copy of the default roles data file as described in Chapter 3, Deploying Other Ceph Services on dedicated nodes.
-
Open the
/home/stack/templates/roles_data_custom.yaml
file. Remove the service entry for the Ceph MDS service (namely, OS::TripleO::Services::CephMds) from under the Controller role:
[...] - name: Controller # the 'primary' role goes first CountDefault: 1 ServicesDefault: - OS::TripleO::Services::CACerts # - OS::TripleO::Services::CephMds 1 - OS::TripleO::Services::CephMon - OS::TripleO::Services::CephExternal - OS::TripleO::Services::CephRbdMirror - OS::TripleO::Services::CephRgw - OS::TripleO::Services::CinderApi [...]
- 1
- Comment out this line. This same service will be added to a custom role in the next step.
At the end of
roles_data_custom.yaml
, add a customCephMDS
role containing the Ceph MDS service and all the other required node services. For example:- name: CephMDS ServicesDefault: # Common Services - OS::TripleO::Services::AuditD - OS::TripleO::Services::CACerts - OS::TripleO::Services::CertmongerUser - OS::TripleO::Services::Collectd - OS::TripleO::Services::Docker - OS::TripleO::Services::Fluentd - OS::TripleO::Services::Kernel - OS::TripleO::Services::Ntp - OS::TripleO::Services::ContainersLogrotateCrond - OS::TripleO::Services::SensuClient - OS::TripleO::Services::Snmp - OS::TripleO::Services::Timezone - OS::TripleO::Services::TripleoFirewall - OS::TripleO::Services::TripleoPackages - OS::TripleO::Services::Tuned # Role-Specific Services - OS::TripleO::Services::CephMds - OS::TripleO::Services::CephClient 1
- 1
- The Ceph MDS service requires the admin keyring, which can be set by either Ceph MON or Ceph Client service. As we are deploying Ceph MDS on a dedicated node (without the Ceph MON service), include the Ceph Client service on the role as well.
Using the
openstack flavor create
command, define a new flavor namedceph-mds
for this role:$ openstack flavor create --id auto --ram 6144 --disk 40 --vcpus 4 ceph-mds
NoteFor more details about this command, run
openstack flavor create --help
.Map this flavor to a new profile, also named
ceph-mds
:$ openstack flavor set --property "cpu_arch"="x86_64" --property "capabilities:boot_option"="local" --property "capabilities:profile"="ceph-mds" ceph-mds
NoteFor more details about this command, run
openstack flavor set --help
.
Tag nodes into the new ceph-mds
profile:
$ ironic node-update UUID add properties/capabilities='profile:ceph-mds,boot_option:local'
See Section 2.4, “Manually tagging the nodes” for more details about tagging nodes. See also Tagging Nodes Into Profiles for related information on custom role profiles.