Chapter 2. Understanding Heat Templates

The custom configurations in this guide use Heat templates and environment files to define certain aspects of the Overcloud. This chapter provides a basic introduction to Heat templates so that you can understand the structure and format of these templates in the context of the Red Hat OpenStack Platform director.

2.1. Heat Templates

The director uses Heat Orchestration Templates (HOT) as a template format for its Overcloud deployment plan. Templates in HOT format are mostly expressed in YAML format. The purpose of a template is to define and create a stack, which is a collection of resources that heat creates, and the configuration of the resources. Resources are objects in OpenStack and can include compute resources, network configuration, security groups, scaling rules, and custom resources.

The structure of a Heat template has three main sections:

Parameters
These are settings passed to heat, which provides a way to customize a stack, and any default values for parameters without passed values. These are defined in the parameters section of a template.
Resources
These are the specific objects to create and configure as part of a stack. OpenStack contains a set of core resources that span across all components. These are defined in the resources section of a template.
Output
These are values passed from heat after the stack’s creation. You can access these values either through the heat API or client tools. These are defined in the output section of a template.

Here is an example of a basic heat template:

heat_template_version: 2013-05-23

description: > A very basic Heat template.

parameters:
  key_name:
    type: string
    default: lars
    description: Name of an existing key pair to use for the instance
  flavor:
    type: string
    description: Instance type for the instance to be created
    default: m1.small
  image:
    type: string
    default: cirros
    description: ID or name of the image to use for the instance

resources:
  my_instance:
    type: OS::Nova::Server
    properties:
      name: My Cirros Instance
      image: { get_param: image }
      flavor: { get_param: flavor }
      key_name: { get_param: key_name }

output:
  instance_name:
    description: Get the instance's name
    value: { get_attr: [ my_instance, name ] }

This template uses the resource type type: OS::Nova::Server to create an instance called my_instance with a particular flavor, image, and key. The stack can return the value of instance_name, which is called My Cirros Instance.

When Heat processes a template it creates a stack for the template and a set of child stacks for resource templates. This creates a hierarchy of stacks that descend from the main stack you define with your template. You can view the stack hierarchy using this following command:

$ heat stack-list --show-nested

2.2. Environment Files

An environment file is a special type of template that provides customization for your Heat templates. This includes three key parts:

Resource Registry
This section defines custom resource names, linked to other heat templates. This essentially provides a method to create custom resources that do not exist within the core resource collection. These are defined in the resource_registry section of an environment file.
Parameters
These are common settings you apply to the top-level template’s parameters. For example, if you have a template that deploys nested stacks, such as resource registry mappings, the parameters only apply to the top-level template and not templates for the nested resources. Parameters are defined in the parameters section of an environment file.
Parameter Defaults
These parameters modify the default values for parameters in all templates. For example, if you have a Heat template that deploys nested stacks, such as resource registry mappings,the parameter defaults apply to all templates. In other words, the top-level template and those defining all nested resources. The parameter defaults are defined in the parameter_defaults section of an environment file.
Important

It is recommended to use parameter_defaults instead of parameters When creating custom environment files for your Overcloud. This is so the parameters apply to all stack templates for the Overcloud.

An example of a basic environment file:

resource_registry:
  OS::Nova::Server::MyServer: myserver.yaml

parameter_defaults:
  NetworkName: my_network

parameters:
  MyIP: 192.168.0.1

For example, this environment file (my_env.yaml) might be included when creating a stack from a certain Heat template (my_template.yaml). The my_env.yaml files creates a new resource type called OS::Nova::Server::MyServer. The myserver.yaml file is a Heat template file that provides an implementation for this resource type that overrides any built-in ones. You can include the OS::Nova::Server::MyServer resource in your my_template.yaml file.

The MyIP applies a parameter only to the main Heat template that deploys along with this environment file. In this example, it only applies to the parameters in my_template.yaml.

The NetworkName applies to both the main Heat template (in this example, my_template.yaml) and the templates associated with resources included the main template, such as the OS::Nova::Server::MyServer resource and its myserver.yaml template in this example.

2.3. Core Overcloud Heat Templates

The director contains a core heat template collection for the Overcloud. This collection is stored in /usr/share/openstack-tripleo-heat-templates.

There are many heat templates and environment files in this collection. However, the main files and directories to note in this template collection are:

overcloud.j2.yaml
This is the main template file used to create the Overcloud environment. This file uses Jinja2 syntax to iterate over certain sections in the template to create custom roles. The Jinja2 formatting is rendered into YAML during the overcloud deployment process.
overcloud-resource-registry-puppet.j2.yaml
This is the main environment file used to create the Overcloud environment. It provides a set of configurations for Puppet modules stored on the Overcloud image. After the director writes the Overcloud image to each node, Heat starts the Puppet configuration for each node using the resources registered in this environment file. This file uses Jinja2 syntax to iterate over certain sections in the template to create custom roles. The Jinja2 formatting is rendered into YAML during the overcloud deployment process.
roles_data.yaml
A file that defines the roles in an overcloud and maps services to each role.
network_data.yaml
A file that defines the networks in an overcloud and their properties such as subnets, allocation pools, and VIP status. The default network_data file only contains the default networks: External, Internal Api, Storage, Storage Management, Tenant, and Management. You can create a custom network_data file and add it to your openstack overcloud deploy command with the -n option.
plan-environment.yaml
A file that defines the metadata for your overcloud plan. This includes the plan name, main template to use, and environment files to apply to the overcloud.
capabilities-map.yaml
A mapping of environment files for an overcloud plan. Use this file to describe and enable environment files through the director’s web UI. Custom environment files detected in the environments directory in an overcloud plan but not defined in the capabilities-map.yaml are listed in the Other subtab of 2 Specify Deployment Configuration > Overall Settings on the web UI.
environments
Contains additional Heat environment files that you can use with your Overcloud creation. These environment files enable extra functions for your resulting OpenStack environment. For example, the directory contains an environment file for enabling Cinder NetApp backend storage (cinder-netapp-config.yaml). Any environment files detected in this directory that are not defined in the capabilities-map.yaml file are listed in the Other subtab of 2 Specify Deployment Configuration > Overall Settings in the director’s web UI.
network
A set of Heat templates to help create isolated networks and ports.
puppet
Templates mostly driven by configuration with puppet. The aforementioned overcloud-resource-registry-puppet.j2.yaml environment file uses the files in this directory to drive the application of the Puppet configuration on each node.
puppet/services
A directory containing heat templates for all services in the composable service architecture.
extraconfig
Templates used to enable extra functionality. For example, the extraconfig/pre_deploy/rhel-registration director provides the ability to register your nodes' Red Hat Enterprise Linux operating systems to the Red Hat Content Delivery network or your own Red Hat Satellite server.
firstboot
Provides example first_boot scripts that the director uses when initially creating the nodes.

2.4. Plan Environment Metadata

A plan environment metadata file allows you to define metadata about your overcloud plan. This information is used when importing and exporting your overcloud plan, plus used during the overcloud creation from your plan.

A plan environment metadata file includes the following parameters:

version
The version of the template.
name
The name of the overcloud plan and the container in OpenStack Object Storage (swift) used to store the plan files.
template
The core parent template to use for the overcloud deployment. This is most often overcloud.yaml, which is the rendered version of the overcloud.yaml.j2 template.
environments
Defines a list of environment files to use. Specify the path of each environment file with the path sub-parameter.
parameter_defaults
A set of parameters to use in your overcloud. This functions in the same way as the parameter_defaults section in a standard environment file.
passwords
A set of parameters to use for overcloud passwords. This functions in the same way as the parameter_defaults section in a standard environment file. Normally, the director automatically populates this section with randomly generated passwords.
workflow_parameters
Allows you to provide a set of parameters to OpenStack Workflow (mistral) namespaces. You can use this to calculate and automatically generate certain overcloud parameters.

The following is an example of the syntax of a plan environment file:

version: 1.0
name: myovercloud
description: 'My Overcloud Plan'
template: overcloud.yaml
environments:
- path: overcloud-resource-registry-puppet.yaml
- path: environments/docker.yaml
- path: environments/docker-ha.yaml
- path: environments/containers-default-parameters.yaml
- path: user-environment.yaml
parameter_defaults:
  ControllerCount: 1
  ComputeCount: 1
  OvercloudComputeFlavor: compute
  OvercloudControllerFlavor: control
workflow_parameters:
  tripleo.derive_params.v1.derive_parameters:
    num_phy_cores_per_numa_node_for_pmd: 2

You can include the plan environment metadata file with the openstack overcloud deploy command using the -p option. For example:

(undercloud) $ openstack overcloud deploy --templates \
  -p /my-plan-environment.yaml \
  [OTHER OPTIONS]

You can also view plan metadata for an existing overcloud plan using the following command:

(undercloud) $ openstack object save overcloud plan-environment.yaml --file -

2.5. Capabilities Map

The capabilities map provides a mapping of environment files in your plan and their dependencies. Use this file to describe and enable environment files through the director’s web UI. Custom environment files detected in an overcloud plan but not listed in the capabilities-map.yaml are listed in the Other subtab of 2 Specify Deployment Configuration > Overall Settings on the web UI.

The default file is located at /usr/share/openstack-tripleo-heat-templates/capabilities-map.yaml.

The following is an example of the syntax for a capabilities map:

topics: 1
  - title: My Parent Section
    description: This contains a main section for different environment files
    environment_groups: 2
      - name: my-environment-group
        title: My Environment Group
        description: A list of environment files grouped together
        environments: 3
          - file: environment_file_1.yaml
            title: Environment File 1
            description: Enables environment file 1
            requires: 4
              - dependent_environment_file.yaml
          - file: environment_file_2.yaml
            title: Environment File 2
            description: Enables environment file 2
            requires: 5
              - dependent_environment_file.yaml
          - file: dependent_environment_file.yaml
            title: Dependent Environment File
            description: Enables the dependent environment file
1
The topics parameter contains a list of sections in the UI’s deployment configuration. Each topic is displayed as a single screen of environment options and contains multiple environment groups, which you define with the environment_groups parameter. Each topic can have a plain text title and description.
2
The environment_groups parameter lists groupings of environment files in the UI’s deployment configuration. For example, on a storage topic, you might have an environment group for Ceph-related environment files. Each environment group can have a plain text title and description.
3
The environments parameter shows all environment files that belong to an environment group. The file parameter is the location of the environment file. Each environment entry can have a plain text title and description.
4 5
The requires parameter is a list of dependencies for an environment file. In this example, both environment_file_1.yaml and environment_file_2.yaml require you to enable dependent_environment_file.yaml too.
Note

Red Hat OpenStack Platform uses this file to add access to features to the director UI. It is recommended not to modify this file as newer versions of Red Hat OpenStack Platform might override this file.

2.6. Including Environment Files in Overcloud Creation

The deployment command (openstack overcloud deploy) uses the -e option to include an environment file to customize your Overcloud. You can include as many environment files as necessary. However, the order of the environment files is important as the parameters and resources defined in subsequent environment files take precedence. For example, you might have two environment files:

environment-file-1.yaml

resource_registry:
  OS::TripleO::NodeExtraConfigPost: /home/stack/templates/template-1.yaml

parameter_defaults:
  RabbitFDLimit: 65536
  TimeZone: 'Japan'

environment-file-2.yaml

resource_registry:
  OS::TripleO::NodeExtraConfigPost: /home/stack/templates/template-2.yaml

parameter_defaults:
  TimeZone: 'Hongkong'

Then deploy with both environment files included:

$ openstack overcloud deploy --templates -e environment-file-1.yaml -e environment-file-2.yaml

In this example, both environment files contain a common resource type (OS::TripleO::NodeExtraConfigPost) and a common parameter (TimeZone). The openstack overcloud deploy command runs through the following process:

  1. Loads the default configuration from the core Heat template collection as per the --template option.
  2. Applies the configuration from environment-file-1.yaml, which overrides any common settings from the default configuration.
  3. Applies the configuration from environment-file-2.yaml, which overrides any common settings from the default configuration and environment-file-1.yaml.

This results in the following changes to the default configuration of the Overcloud:

  • OS::TripleO::NodeExtraConfigPost resource is set to /home/stack/templates/template-2.yaml as per environment-file-2.yaml.
  • TimeZone parameter is set to Hongkong as per environment-file-2.yaml.
  • RabbitFDLimit parameter is set to 65536 as per environment-file-1.yaml. environment-file-2.yaml does not change this value.

This provides a method for defining custom configuration to the your Overcloud without values from multiple environment files conflicting.

2.7. Using Customized Core Heat Templates

When creating the overcloud, the director uses a core set of Heat templates located in /usr/share/openstack-tripleo-heat-templates. If you want to customize this core template collection, use a Git workflow to track changes and merge updates. Use the following git processes to help manage your custom template collection:

Initializing a Custom Template Collection

Use the following procedure to create an initial Git repository containing the Heat template collection:

  1. Copy the template collection to the stack users directory. This example copies the collection to the ~/templates directory:

    $ cd ~/templates
    $ cp -r /usr/share/openstack-tripleo-heat-templates .
  2. Change to the custom template directory and initialize a Git repository:

    $ cd openstack-tripleo-heat-templates
    $ git init .
  3. Stage all templates for the initial commit:

    $ git add *
  4. Create an initial commit:

    $ git commit -m "Initial creation of custom core heat templates"

This creates an initial master branch containing the latest core template collection. Use this branch as the basis for your custom branch and merge new template versions to this branch.

Creating a Custom Branch and Committing Changes

Use a custom branch to store your changes to the core template collection. Use the following procedure to create a my-customizations branch and add customizations to it:

  1. Create the my-customizations branch and switch to it:

    $ git checkout -b my-customizations
  2. Edit the files in the custom branch.
  3. Stage the changes in git:

    $ git add [edited files]
  4. Commit the changes to the custom branch:

    $ git commit -m "[Commit message for custom changes]"

This adds your changes as commits to the my-customizations branch. When the master branch updates, you can rebase my-customizations off master, which causes git to add these commits on to the updated template collection. This helps track your customizations and replay them on future template updates.

Updating the Custom Template Collection:

When updating the undercloud, the openstack-tripleo-heat-templates package might also update. When this occurs, use the following procedure to update your custom template collection:

  1. Save the openstack-tripleo-heat-templates package version as an environment variable:

    $ export PACKAGE=$(rpm -qv openstack-tripleo-heat-templates)
  2. Change to your template collection directory and create a new branch for the updated templates:

    $ cd ~/templates/openstack-tripleo-heat-templates
    $ git checkout -b $PACKAGE
  3. Remove all files in the branch and replace them with the new versions:

    $ git rm -rf *
    $ cp -r /usr/share/openstack-tripleo-heat-templates/* .
  4. Add all templates for the initial commit:

    $ git add *
  5. Create a commit for the package update:

    $ git commit -m "Updates for $PACKAGE"
  6. Merge the branch into master. If using a Git management system (such as GitLab) use the management workflow. If using git locally, merge by switching to the master branch and run the git merge command:

    $ git checkout master
    $ git merge $PACKAGE

The master branch now contains the latest version of the core template collection. You can now rebase the my-customization branch from this updated collection.

Rebasing the Custom Branch

Use the following procedure to update the my-customization branch,:

  1. Change to the my-customizations branch:

    $ git checkout my-customizations
  2. Rebase the branch off master:

    $ git rebase master

This updates the my-customizations branch and replays the custom commits made to this branch.

If git reports any conflicts during the rebase, use this procedure:

  1. Check which files contain the conflicts:

    $ git status
  2. Resolve the conflicts of the template files identified.
  3. Add the resolved files

    $ git add [resolved files]
  4. Continue the rebase:

    $ git rebase --continue

Deploying Custom Templates

Use the following procedure to deploy the custom template collection:

  1. Make sure you have switched to the my-customization branch:

    git checkout my-customizations
  2. Run the openstack overcloud deploy command with the --templates option to specify your local template directory:

    $ openstack overcloud deploy --templates /home/stack/templates/openstack-tripleo-heat-templates [OTHER OPTIONS]
Note

The director uses the default template directory (/usr/share/openstack-tripleo-heat-templates) if you specify the --templates option without a directory.

Important

Red Hat recommends using the methods in Chapter 4, Configuration Hooks instead of modifying the heat template collection.