This guide aims to help partners integrate their software and hardware solutions in a manner that the director configures as a part of the Overcloud. This follows a workflow broken down into multiple sections that show how to perform certain integration tasks:

You must meet several prerequisites before meaningful integration work can be completed with the director. These requirements are not limited to technical integration and also include various levels of partner solution documentation. The goal is to have a complete shared understanding of the entire integration so that Red Hat engineering, partner managers, and support resources can effectively support the work.

The first requirement is related to Red Hat OpenStack Platform solution certification. To be included with OpenStack Platform director, the partner solution must first be certified with Red Hat OpenStack Platform.

This section examines some of the core components of the Red Hat OpenStack Platform director and describes how they contribute to Overcloud creation.

2.1.1. Ironic

Ironic provides dedicated bare metal hosts to end users through self-service provisioning. The director uses Ironic to manage the lifecycle of the bare metal hardware in our Overcloud. Ironic has its own native API for defining bare metal nodes. Administrators aiming to provision OpenStack environments with the director must register their nodes with Ironic using a specific driver. The main supported driver is The Intelligent Platform Management Interface (IPMI) as most hardware contains some support for IPMI power management functions. However, ironic also contains vendor specific equivalents such as HP iLO, Cisco UCS, or Dell DRAC. Ironic controls the power management of the nodes and gathers hardware information or facts using a introspection mechanism. The director uses the information obtained from the introspection process to match node to various OpenStack environment roles, such as Controller nodes, Compute nodes, and storage nodes. For example, a discovered node with 10 disks will more than likely be provisioned as a storage node.

Partners wishing to have director support for their hardware will need to have driver coverage in Ironic.

2.1.2. Heat

Heat acts as an application stack orchestration engine. This allows organizations to define elements for a given application before deploying it to a cloud. This involves creating a stack template that includes a number of infrastructure resources (e.g. instances, networks, storage volumes, elastic IPs, etc) along with a set of parameters for configuration. Heat creates these resources based on a given dependency chain, monitors them for availability, and scales them where necessary. These templates enable application stacks to become portable and achieve repeatable results.

The director uses the native OpenStack Heat APIs to provision and manage the resources associated with deploying an Overcloud. This includes precise details such as defining the number of nodes to provision per node role, the software components to configure for each node, and the order in which the director configures these components and node types. The director also uses Heat for troubleshooting a deployment and making changes post-deployment with ease.

The following example is a snippet from a Heat template that defines parameters of a Controller node:

NeutronExternalNetworkBridge:
    description: Name of bridge used for external network traffic.
    type: string
    default: 'br-ex'
NeutronBridgeMappings:
    description: >
      The OVS logical->physical bridge mappings to use. See the Neutron
      documentation for details. Defaults to mapping br-ex - the external
      bridge on hosts - to a physical name 'datacentre' which can be used
      to create provider networks (and we use this for the default floating
      network) - if changing this either use different post-install network
      scripts or be sure to keep 'datacentre' as a mapping network name.
    type: string
    default: "datacentre:br-ex"

Heat consumes templates included with the director to facilitate the creation of an Overcloud, which includes calling Ironic to power the nodes. We can view the resources (and their status) of an in-progress Overcloud using the standard Heat tools. For example, you can use the Heat tools to display the Overcloud as a nested application stack.

Heat provides a comprehensive and powerful syntax for declaring and creating production OpenStack clouds. However, it requires some prior understanding and proficiency for partner integration. Every partner integration use case requires Heat templates.

The director requires several disk images for provisioning Overcloud nodes. This includes:

Obtain these images from the rhosp-director-images and rhosp-director-images-ipa packages:

$ sudo yum install rhosp-director-images rhosp-director-images-ipa

Extract the archives to the images directory on the stack user’s home (/home/stack/images):

$ cd ~/images
$ for i in /usr/share/rhosp-director-images/overcloud-full-latest-10.0.tar /usr/share/rhosp-director-images/ironic-python-agent-latest-10.0.tar; do tar -xvf $i; done

Some situations might require you to modify the initial ramdisk. For example, you might require a certain driver available when you boot the nodes during the introspection or provisioning processes. The following procedure shows how to modify an initial ramdisk. In the context of the Overcloud, this includes either:

This procedure adds an additional RPM package to the ironic-python-agent.initramfs ramdisk as an example.

Log in as the root user and create a temporary directory for the ramdisk

# mkdir ~/ipa-tmp
# cd ~/ipa-tmp

Use the skipcpio and `cpio commands to extract the ramdisk to the temporary directory:

# /usr/lib/dracut/skipcpio ~/images/ironic-python-agent.initramfs | zcat | cpio -ivd | pax -r

Install an RPM package to the extracted contents:

# rpm2cpio ~/RPMs/python-proliantutils-2.1.7-1.el7ost.noarch.rpm | pax -r

Recreate the new ramdisk:

# find . 2>/dev/null | cpio --quiet -c -o | gzip -8  > /home/stack/images/ironic-python-agent.initramfs
# chown stack: /home/stack/images/ironic-python-agent.initramfs

Verify the new package now exists in the ramdisk:

# lsinitrd /home/stack/images/ironic-python-agent.initramfs | grep proliant

The libguestfs-tools package contains the virt-customize tool. Install the libguestfs-tools from the rhel-7-server-rpms repository:

$ sudo yum install libguestfs-tools

You might aim to explore the contents of the overcloud-full.qcow2. Create a virtual machine instance using either the qemu-system-x86_64 command:

$ sudo qemu-system-x86_64 --kernel overcloud-full.vmlinuz --initrd overcloud-full.initrd -m 1024 --append root=/dev/sda --enable-kvm overcloud-full.qcow2

Or using the following boot options in virt-manager:

Set the password for the root user on image:

$ virt-customize --selinux-relabel -a overcloud-full.qcow2 --root-password password:test
[  0.0] Examining the guest ...
[ 18.0] Setting a random seed
[ 18.0] Setting passwords
[ 19.0] Finishing off

This provides administration-level access for your nodes through the console.

Register your image temporarily to enable Red Hat repositories relevant to your customizations:

$ virt-customize --selinux-relabel -a overcloud-full.qcow2 --run-command 'subscription-manager register --username=[username] --password=[password]'
[  0.0] Examining the guest ...
[ 10.0] Setting a random seed
[ 10.0] Running: subscription-manager register --username=[username] --password=[password]
[ 24.0] Finishing off

Make sure to replace the [username] and [password] with your Red Hat customer account details. This runs the following command on the image:

subscription-manager register --username=[username] --password=[password]

This registers your Overcloud image to the Red Hat Content Delivery Network:

Find a list of pool ID from your account’s subscriptions:

$ sudo subscription-manager list

Choose a subscription pool ID and attach it to the image:

$ virt-customize --selinux-relabel -a overcloud-full.qcow2 --run-command 'subscription-manager attach --pool [subscription-pool]'
[  0.0] Examining the guest ...
[ 12.0] Setting a random seed
[ 12.0] Running: subscription-manager attach --pool [subscription-pool]
[ 52.0] Finishing off

Make sure to replace the [subscription-pool] with your chosen subscription pool ID. This runs the following command on the image:

subscription-manager attach --pool [subscription-pool]

This adds the pool to the image, which allows you to enable Red Hat repositories with the following command:

$ subscription-manager repos --enable=[repo-id]

Adding third-party software to the image requires additional repositories. For example, the following is an example repo file that contains configuration to use the OpenDaylight repository content:

$ cat opendaylight.repo

[opendaylight]
name=OpenDaylight Repository
baseurl=https://nexus.opendaylight.org/content/repositories/opendaylight-yum-epel-6-x86_64/
gpgcheck=0

Copy the repository file on to the image:

$ virt-customize --selinux-relabel -a overcloud-full.qcow2 --upload opendaylight.repo:/etc/yum.repos.d/
[  0.0] Examining the guest ...
[ 12.0] Setting a random seed
[ 12.0] Copying: opendaylight.repo to /etc/yum.repos.d/
[ 13.0] Finishing off

The --copy-in option copies the repository file to /etc/yum.repos.d/ on the Overcloud image.

Important: Red Hat does not offer support for software from non-certified vendors. Check with your Red Hat support representative that the software you aim to install is supported.

Use the virt-customize command to install packages to the image:

$ virt-customize --selinux-relabel -a overcloud-full.qcow2 --install opendaylight
[  0.0] Examining the guest ...
[ 11.0] Setting a random seed
[ 11.0] Installing packages: opendaylight
[ 91.0] Finishing off

The --install option allows you to specify a package to install.

After installing the necessary packages to customize the image, we now remove our subscriptions and unregister the image:

$ virt-customize --selinux-relabel -a overcloud-full.qcow2 --run-command 'subscription-manager remove --all'
[  0.0] Examining the guest ...
[ 12.0] Setting a random seed
[ 12.0] Running: subscription-manager remove --all
[ 18.0] Finishing off

This removes all subscription pools from the image.

Finally, unregister the image. This is so the Overcloud deployment process can deploy the image to your nodes and register each of them individually.

$ virt-customize --selinux-relabel -a overcloud-full.qcow2 --run-command 'subscription-manager unregister'
[  0.0] Examining the guest ...
[ 11.0] Setting a random seed
[ 11.0] Running: subscription-manager unregister
[ 17.0] Finishing off

After modifying the image, upload it to the director. Make sure to source the stackrc file so that you can access the director from the command line:

$ source stackrc
$ openstack overcloud image upload --image-path /home/stack/images/

This uploads the following images into the director: bm-deploy-kernel, bm-deploy-ramdisk, overcloud-full, overcloud-full-initrd, and overcloud-full-vmlinuz. These are the images for deployment and the Overcloud. The script also installs the introspection images on the director’s PXE server. View a list of the images in the CLI using the following command:

$ openstack image list
+--------------------------------------+------------------------+
| ID                                   | Name                   |
+--------------------------------------+------------------------+
| 765a46af-4417-4592-91e5-a300ead3faf6 | bm-deploy-ramdisk      |
| 09b40e3d-0382-4925-a356-3a4b4f36b514 | bm-deploy-kernel       |
| ef793cd0-e65c-456a-a675-63cd57610bd5 | overcloud-full         |
| 9a51a6cb-4670-40de-b64b-b70f4dd44152 | overcloud-full-initrd  |
| 4f7e33f4-d617-47c1-b36f-cbe90f132e5d | overcloud-full-vmlinuz |
+--------------------------------------+------------------------+

This list will not show the introspection PXE images (agent.*). The director copies these files to /httpboot.

[stack@host1 ~]$ ls /httpboot -l
total 151636
-rw-r--r--. 1 ironic ironic       269 Sep 19 02:43 boot.ipxe
-rw-r--r--. 1 root   root         252 Sep 10 15:35 inspector.ipxe
-rwxr-xr-x. 1 root   root     5027584 Sep 10 16:32 agent.kernel
-rw-r--r--. 1 root   root   150230861 Sep 10 16:32 agent.ramdisk
drwxr-xr-x. 2 ironic ironic      4096 Sep 19 02:45 pxelinux.cfg

The following section provide a few basic to help you understand Puppet’s syntax and the structure of a Puppet module.

class mymodule::httpd {
  package { 'httpd':
    ensure => installed,
  }
}

class mymodule::httpd {
  package { 'httpd':
    ensure => installed,
  }
  service { 'httpd':
    ensure => running,
    enable => true,
    require => Package["httpd"],
  }
}

Listen <%= @httpd_port %>
NameVirtualHost *:<%= @httpd_port %>
<VirtualHost *:<%= @httpd_port %>>
  DocumentRoot /var/www/myserver/
  ServerName *:<%= @fqdn %>>
  <Directory "/var/www/myserver/">
    Options All Indexes FollowSymLinks
    Order allow,deny
    Allow from all
  </Directory>
</VirtualHost>

class mymodule::httpd {
  package { 'httpd':
    ensure => installed,
  }
  service { 'httpd':
    ensure => running,
    enable => true,
    require => Package["httpd"],
  }
  file {'/etc/httpd/conf.d/myserver.conf':
  notify => Service["httpd"],
    ensure => file,
    require => Package["httpd"],
    content => template("mymodule/myserver.conf.erb"),
  }
  file { "/var/www/myserver":
    ensure => "directory",
  }
}

The Red Hat OpenStack Platform uses the official OpenStack Puppet modules, which you obtain from the openstack group on Github. Navigate your browser to https://github.com/openstack and in the filters section search for puppet. All Puppet module use the prefix puppet-.

For this example, we will examine the official OpenStack Block Storage (cinder), which you can clone using the following command:

$ git clone https://github.com/openstack/puppet-cinder.git

This creates a clone of the Puppet module for Cinder.

The OpenStack modules primarily aim to configure the core service. Most also contain additional manifests to configure additional services, sometimes known as backends, agents, or plugins. For example, the cinder module contains a directory called backends, which contains configuration options for different storage devices including NFS, iSCSI, Red Hat Ceph Storage, and others.

For example, the manifests/backends/nfs.pp file contains the following configuration

define cinder::backend::nfs (
  $volume_backend_name  = $name,
  $nfs_servers          = [],
  $nfs_mount_options    = undef,
  $nfs_disk_util        = undef,
  $nfs_sparsed_volumes  = undef,
  $nfs_mount_point_base = undef,
  $nfs_shares_config    = '/etc/cinder/shares.conf',
  $nfs_used_ratio       = '0.95',
  $nfs_oversub_ratio    = '1.0',
  $extra_options        = {},
) {

  file {$nfs_shares_config:
    content => join($nfs_servers, "\n"),
    require => Package['cinder'],
    notify  => Service['cinder-volume']
  }

  cinder_config {
    "${name}/volume_backend_name":  value => $volume_backend_name;
    "${name}/volume_driver":        value =>
      'cinder.volume.drivers.nfs.NfsDriver';
    "${name}/nfs_shares_config":    value => $nfs_shares_config;
    "${name}/nfs_mount_options":    value => $nfs_mount_options;
    "${name}/nfs_disk_util":        value => $nfs_disk_util;
    "${name}/nfs_sparsed_volumes":  value => $nfs_sparsed_volumes;
    "${name}/nfs_mount_point_base": value => $nfs_mount_point_base;
    "${name}/nfs_used_ratio":       value => $nfs_used_ratio;
    "${name}/nfs_oversub_ratio":    value => $nfs_oversub_ratio;
  }

  create_resources('cinder_config', $extra_options)

}

This achieves a couple of things:

This shows the importance of including a manifest to configure your hardware’s OpenStack driver. The manifest provides a simple method for the director to include configuration options relevant to your hardware. This acts as a main integration point for the director to configure your Overcloud to use your hardware.

Puppet contains a tool called Hiera, which acts as a key/value systems that provides node-specific configuration. These keys and their values are usually stored in files located in /etc/puppet/hieradata. The /etc/puppet/hiera.yaml file defines the order that Puppet reads the files in the hieradata directory.

When configuring the Overcloud, Puppet uses this data to overwrite the default values for certain Puppet classes. For example, the default NFS mount options for cinder::backend::nfs in puppet-cinder are undefined:

  $nfs_mount_options    = undef,

However, you can create your own manifest that calls the cinder::backend::nfs defined type and replace this option with Hiera data:

  cinder::backend::nfs { $cinder_nfs_backend:
    nfs_mount_options   => hiera('cinder_nfs_mount_options'),
  }

This means the nfs_mount_options parameter takes uses Hiera data value from the cinder_nfs_mount_options key:

cinder_nfs_mount_options: rsize=8192,wsize=8192

Alternatively, you can use the Hiera data to overwrite cinder::backend::nfs::nfs_mount_options parameter directly so that it applies to all evalutations of the NFS configuration. For example:

cinder::backend::nfs::nfs_mount_options: rsize=8192,wsize=8192

The above Hiera data overwrites this parameter on each evaluation of cinder::backend::nfs.

The director uses an advanced Heat template collection used to create an Overcloud. This collection is available from the openstack group on Github in the openstack-tripleo-heat-templates repository. To obtain a clone of this template collection, run the following command:

$ git clone https://github.com/openstack/tripleo-heat-templates.git

There are many heat templates and environment files in this collection. However, the main files and directories to note in this template collection are:

This provides a general overview of the templates the director uses for orchestrating the Overcloud creation. The next few sections show how to create your own custom templates and environment files that you can add to an Overcloud deployment.

The director provides a mechanism to perform configuration on all nodes upon the initial creation of the Overcloud. The director achieves this through cloud-init, which you can call using the OS::TripleO::NodeUserData resource type.

In this example, you will update the nameserver with a custom IP address on all nodes. You must first create a basic heat template (/home/stack/templates/nameserver.yaml) that runs a script to append each node’s resolv.conf with a specific nameserver. You can use the OS::TripleO::MultipartMime resource type to send the configuration script.

heat_template_version: 2014-10-16

description: >
  Extra hostname configuration

resources:
  userdata:
    type: OS::Heat::MultipartMime
    properties:
      parts:
      - config: {get_resource: nameserver_config}

  nameserver_config:
    type: OS::Heat::SoftwareConfig
    properties:
      config: |
        #!/bin/bash
        echo "nameserver 192.168.1.1" >> /etc/resolv.conf

outputs:
  OS::stack_id:
    value: {get_resource: userdata}

Next, create an environment file (/home/stack/templates/firstboot.yaml) that registers your heat template as the OS::TripleO::NodeUserData resource type.

resource_registry:
  OS::TripleO::NodeUserData: /home/stack/templates/nameserver.yaml

To add the first boot configuration, add the environment file to the stack along with your other environment files when first creating the Overcloud. For example:

$ openstack overcloud deploy --templates \
    ...
    -e /home/stack/templates/firstboot.yaml \
    ...

The -e applies the environment file to the Overcloud stack.

This adds the configuration to all nodes when they are first created and boot for the first time. Subsequent inclusions of these templates, such as updating the Overcloud stack, does not run these scripts.

This achieves the following:

As a result, each node runs the following Bash script on its first boot:

#!/bin/bash
echo "nameserver 192.168.1.1" >> /etc/resolve.conf

This example shows how Heat template pass and modfy configuration from one resource to another. It also shows how to use environment files to register new Heat resources or modify existing ones.

The Overcloud uses Puppet for the core configuration of OpenStack components. The director provides a set of hooks to provide custom configuration for specific node roles after the first boot completes and before the core configuration begins. These hooks include:

In this example, you first create a basic heat template (/home/stack/templates/nameserver.yaml) that runs a script to write to a node’s resolv.conf with a variable nameserver.

heat_template_version: 2014-10-16

description: >
  Extra hostname configuration

  parameters:
    server:
      type: json
    nameserver_ip:
      type: string
    DeployIdentifier:
      type: string

resources:
  CustomExtraConfigPre:
    type: OS::Heat::SoftwareConfig
    properties:
      group: script
      config:
        str_replace:
          template: |
            #!/bin/sh
            echo "nameserver _NAMESERVER_IP_" > /etc/resolv.conf
          params:
            _NAMESERVER_IP_: {get_param: nameserver_ip}

  CustomExtraDeploymentPre:
    type: OS::Heat::SoftwareDeployment
    properties:
      server: {get_param: server}
      config: {get_resource: CustomExtraConfigPre}
      actions: ['CREATE','UPDATE']
      input_values:
        deploy_identifier: {get_param: DeployIdentifier}

outputs:
  deploy_stdout:
    description: Deployment reference, used to trigger pre-deploy on changes
    value: {get_attr: [CustomExtraDeploymentPre, deploy_stdout]}

In this example, the resources section contains the following:

Next, create an environment file (/home/stack/templates/pre_config.yaml) that registers your heat template to the role-based resource type. For example, to apply only to Controller nodes, use the ControllerExtraConfigPre hook:

resource_registry:
  OS::TripleO::ControllerExtraConfigPre: /home/stack/templates/nameserver.yaml

parameter_defaults:
  nameserver_ip: 192.168.1.1

To apply the configuration, add the environment file to the stack along with your other environment files when creating or updating the Overcloud. For example:

$ openstack overcloud deploy --templates \
    ...
    -e /home/stack/templates/pre_config.yaml \
    ...

This applies the configuration to all Controller nodes before the core configuration begins on either the initial Overcloud creation or subsequent updates.

This achieves the following:

This example shows how to create a Heat template that defines a configuration and deploys it using the OS::Heat::SoftwareConfig and OS::Heat::SoftwareDeployments before the core configuration. It also shows how to define parameters in your environment file and pass them to templates in the configuration.

The Overcloud uses Puppet for the core configuration of OpenStack components. The director provides a hook to configure all node types after the first boot completes and before the core configuration begins:

In this example, you first create a basic heat template (/home/stack/templates/nameserver.yaml) that runs a script to append each node’s resolv.conf with a variable nameserver.

heat_template_version: 2014-10-16

description: >
  Extra hostname configuration

parameters:
  server:
    type: string
  nameserver_ip:
    type: string
  DeployIdentifier:
    type: string

resources:
  CustomExtraConfigPre:
    type: OS::Heat::SoftwareConfig
    properties:
      group: script
      config:
        str_replace:
          template: |
            #!/bin/sh
            echo "nameserver _NAMESERVER_IP_" >> /etc/resolv.conf
          params:
            _NAMESERVER_IP_: {get_param: nameserver_ip}

  CustomExtraDeploymentPre:
    type: OS::Heat::SoftwareDeployment
    properties:
      server: {get_param: server}
      config: {get_resource: CustomExtraConfigPre}
      actions: ['CREATE','UPDATE']
      input_values:
        deploy_identifier: {get_param: DeployIdentifier}

outputs:
  deploy_stdout:
    description: Deployment reference, used to trigger pre-deploy on changes
    value: {get_attr: [CustomExtraDeploymentPre, deploy_stdout]}

In this example, the resources section contains the following:

Next, create an environment file (/home/stack/templates/pre_config.yaml) that registers your heat template as the OS::TripleO::NodeExtraConfig resource type.

resource_registry:
  OS::TripleO::NodeExtraConfig: /home/stack/templates/nameserver.yaml

parameter_defaults:
  nameserver_ip: 192.168.1.1

To apply the configuration, add the environment file to the stack along with your other environment files when creating or updating the Overcloud. For example:

$ openstack overcloud deploy --templates \
    ...
    -e /home/stack/templates/pre_config.yaml \
    ...

This applies the configuration to all nodes before the core configuration begins on either the initial Overcloud creation or subsequent updates.

This achieves the following:

This example shows how to create a Heat template that defines a configuration and deploys it using the OS::Heat::SoftwareConfig and OS::Heat::SoftwareDeployments before the core configuration. It also shows how to define parameters in your environment file and pass them to templates in the configuration.

A situation might occur where you have completed the creation of your Overcloud but want to add additional configuration to all roles, either on initial creation or on a subsequent update of the Overcloud. In this case, you use the following post-configuration hook:

In this example, you first create a basic heat template (/home/stack/templates/nameserver.yaml) that runs a script to append each node’s resolv.conf with a variable nameserver.

heat_template_version: 2014-10-16

description: >
  Extra hostname configuration

parameters:
  servers:
    type: json
  nameserver_ip:
    type: string
  DeployIdentifier:
    type: string

resources:
  CustomExtraConfig:
    type: OS::Heat::SoftwareConfig
    properties:
      group: script
      config:
        str_replace:
          template: |
            #!/bin/sh
            echo "nameserver _NAMESERVER_IP_" >> /etc/resolv.conf
          params:
            _NAMESERVER_IP_: {get_param: nameserver_ip}

  CustomExtraDeployments:
    type: OS::Heat::SoftwareDeploymentGroup
    properties:
      servers:  {get_param: servers}
      config: {get_resource: CustomExtraConfig}
      actions: ['CREATE','UPDATE']
      input_values:
        deploy_identifier: {get_param: DeployIdentifier}

In this example, the resources section contains the following:

Next, create an environment file (/home/stack/templates/post_config.yaml) that registers your heat template as the OS::TripleO::NodeExtraConfigPost: resource type.

resource_registry:
  OS::TripleO::NodeExtraConfigPost: /home/stack/templates/nameserver.yaml

parameter_defaults:
  nameserver_ip: 192.168.1.1

To apply the configuration, add the environment file to the stack along with your other environment files when creating or updating the Overcloud. For example:

$ openstack overcloud deploy --templates \
    ...
    -e /home/stack/templates/post_config.yaml \
    ...

This applies the configuration to all nodes after the core configuration completes on either initial Overcloud creation or subsequent updates.

This achieves the following:

This example shows how to create a Heat template that defines a configuration and deploys it using the OS::Heat::SoftwareConfig and OS::Heat::SoftwareDeployments. It also shows how to define parameters in your environment file and pass them to templates in the configuration.

Previously, we discussed adding configuration for a new backend to OpenStack Puppet modules. This section show how the director executes the application of new configuration.

Heat templates provide a hook allowing you to apply Puppet configuration with a OS::Heat::SoftwareConfig resource. The process is similar to how we include and execute Bash scripts. However, instead of the group: script hook, we use the group: puppet hook.

For example, you might have a Puppet manifest (example-puppet-manifest.pp) that enables an NFS Cinder backend using the official Cinder Puppet Module:

cinder::backend::nfs { 'mynfsserver':
  nfs_servers          => ['192.168.1.200:/storage'],
}

This Puppet configuration creates a new resource using the cinder::backend::nfs defined type. To apply this resource through Heat, create a basic Heat template (puppet-config.yaml) that runs our Puppet manifest:

heat_template_version: 2014-10-16

parameters:
  servers:
    type: json

resources:
  ExtraPuppetConfig:
    type: OS::Heat::SoftwareConfig
    properties:
      group: puppet
      config:
        get_file: example-puppet-manifest.pp
      options:
        enable_hiera: True
        enable_facter: False

  ExtraPuppetDeployment:
    type: OS::Heat::SoftwareDeployments
    properties:
      config: {get_resource: ExtraPuppetConfig}
      servers: {get_param: servers}
      actions: ['CREATE','UPDATE']

Next, create an environment file (puppet_config.yaml) that registers our Heat template as the OS::TripleO::NodeExtraConfigPost resource type.

resource_registry:
  OS::TripleO::NodeExtraConfigPost: puppet_config.yaml

This example is similar to using SoftwareConfig and SoftwareDeployments from the script hook example in the previous section. However, there are some differences in this example:

This example shows how to include a Puppet manifest as part of the software configuration for the Overcloud. This provides a way to apply certain configuration classes from existing Puppet modules on the Overcloud images, which helps you customize your Overcloud to use certain software and hardware.

The Heat template collection contains a set of parameters to pass extra configuration to certain node types. These parameters save the configuration as hieradata for the node’s Puppet configuration. These parameters are:

To add extra configuration to the post-deployment configuration process, create an environment file that contains these parameters in the parameter_defaults section. For example, to increase the reserved memory for Compute hosts to 1024 MB and set the VNC keymap to Japanese:

parameter_defaults:
  NovaComputeExtraConfig:
    nova::compute::reserved_host_memory: 1024
    nova::compute::vnc_keymap: ja

Include this environment file when running openstack overcloud deploy.

After developing a set of environment files relevant to your custom configuration, include these files in your Overcloud deployment. This means running the openstack overcloud deploy command with the -e option, followed by the environment file. You can specify the -e option as many times as necessary for your customization. For example:

$ openstack overcloud deploy --templates -e network-configuration.yaml -e storage-configuration.yaml -e first-boot.yaml

The core Heat template collection contains a collection of composable service templates in the puppet/services subdirectory. You can view these services with the following command:

$ ls /usr/share/openstack-tripleo-heat-templates/puppet/services

Each service template contains a description that identifies its purpose. For example, the keystone.yaml service template contains the following description:

description: >
 OpenStack Identity (`keystone`) service configured with Puppet

These service templates are registered as resources specific to a Red Hat OpenStack Platform deployment. This means you can call each resource using a unique Heat resource namespace defined in the overcloud-resource-registry-puppet.j2.yaml file. All services use the OS::TripleO::Services namespace for their resource type. For example, the keystone.yaml service template is registered to the OS::TripleO::Services::Keystone resource type:

grep "OS::TripleO::Services::Keystone" /usr/share/openstack-tripleo-heat-templates/overcloud-resource-registry-puppet.j2.yaml
  OS::TripleO::Services::Keystone: puppet/services/keystone.yaml

The overcloud.j2.yaml Heat template includes a section of Jinja2-based code to define a service list for each custom role in the roles_data.yaml file:

{{role.name}}Services:
  description: A list of service resources (configured in the Heat
               resource_registry) which represent nested stacks
               for each service that should get installed on the {{role.name}} role.
  type: comma_delimited_list
  default: {{role.ServicesDefault|default([])}}

For the default roles, this creates the following service list parameters: ControllerServices, ComputeServices, BlockStorageServices, ObjectStorageServices, and CephStorageServices.

You define the default services for each custom role in the roles_data.yaml file. For example, the default Controller role contains the following content:

- name: Controller
  CountDefault: 1
  ServicesDefault:
    - OS::TripleO::Services::CACerts
    - OS::TripleO::Services::CephMon
    - OS::TripleO::Services::CephExternal
    - OS::TripleO::Services::CephRgw
    - OS::TripleO::Services::CinderApi
    - OS::TripleO::Services::CinderBackup
    - OS::TripleO::Services::CinderScheduler
    - OS::TripleO::Services::CinderVolume
    - OS::TripleO::Services::Core
    - OS::TripleO::Services::Kernel
    - OS::TripleO::Services::Keystone
    - OS::TripleO::Services::GlanceApi
    - OS::TripleO::Services::GlanceRegistry
...

These services are then defined as the default list for the ControllerServices parameter.

You can also use an environment file to override the default list for the service parameters. For example, you can define ControllerServices as a parameter_default in an environment file to override the services list from the roles_data.yaml file.

This example examines how to create a user-defined composable service and focuses on implementing a message of the day (motd) service. This example assumes the overcloud image contains a custom motd Puppet module loaded either through a configuration hook or through modifying the overcloud images as per Chapter 3, Overcloud Images.

When creating your own service, there are specific items to define in the service’s Heat template:

The following is an example Heat template (service.yaml) for the motd service:

heat_template_version: 2016-04-08

description: >
  Message of the day service configured with Puppet

parameters:
  ServiceNetMap:
    default: {}
    type: json
  DefaultPasswords:
    default: {}
    type: json
  EndpointMap:
    default: {}
    type: json
  MotdMessage: 1
    default: |
      Welcome to my Red Hat OpenStack Platform environment!

    type: string
    description: The message to include in the motd

outputs:
  role_data:
    description: Motd role using composable services.
    value:
      service_name: motd
      config_settings: 2
        motd::content: {get_param: MotdMessage}
      step_config: | 3
        if hiera('step') >= 2 {
          include ::motd
        }

The aim for this example is to configure the custom motd service only on our overcloud’s Controller nodes. This requires a custom environment file and custom roles data file included with our deployment.

First, add the new service to an environment file (env-motd.yaml) as a registered Heat resource within the OS::TripleO::Services namespace. For this example, the resource name for our motd service is OS::TripleO::Services::Motd:

resource_registry:
  OS::TripleO::Services::Motd: /home/stack/templates/motd.yaml

parameter_defaults:
  MotdMessage: |
    You have successfully accessed my Red Hat OpenStack Platform environment!

Note that our custom environment file also includes a new message that overrides the default for MotdMessage.

The deployment will now include the motd service. However, each role that requires this new service must have an updated ServicesDefault listing in a custom roles_data.yaml file. In this example, we aim to only configure the service on Controller nodes.

Create a copy of the default roles_data.yaml file:

$ cp /usr/share/openstack-tripleo-heat-templates/roles_data.yaml ~/custom_roles_data.yaml

Edit this file, scroll to the Controller role, and include the service in the ServicesDefault listing:

- name: Controller
  CountDefault: 1
  ServicesDefault:
    - OS::TripleO::Services::CACerts
    - OS::TripleO::Services::CephMon
    - OS::TripleO::Services::CephExternal
...
    - OS::TripleO::Services::FluentdClient
    - OS::TripleO::Services::VipHosts
    - OS::TripleO::Services::Motd           # Add the service to the end

When creating an overcloud, include the resulting environment file and the custom_roles_data.yaml file with your other environment files and deployment options:

$ openstack overcloud deploy --templates -e /home/stack/templates/env-motd.yaml -r ~/custom_roles_data.yaml [OTHER OPTIONS]

This includes our custom motd service in our deployment and configures the service on Controller nodes only.

The OpenStack Bare Metal Provisioning (Ironic) component is used within the director to control the power state of the nodes. The director uses a set of back-end drivers to interface with specific bare metal power controllers. These drivers are the key to enabling hardware and vendor specific extensions and capabilities. The most common driver is the IPMI driver (pxe_ipmitool) which controls the power state for any server that supports the Intelligent Platform Management Interface (IPMI).

Integrating with Ironic starts with the upstream OpenStack community first. Ironic drivers accepted upstream are automatically included in the core Red Hat OpenStack Platform product and the director by default. However, they might not be supported as per certification requirements.

Hardware drivers must undergo continuous integration testing to ensure their continued functionality. For information on third party driver testing and suitability, please see the OpenStack community page on Ironic Testing.

Upstream Repositories:

Upstream Blueprints:

Puppet Module:

Bugzilla components:

Integration Notes:

OpenStack Networking (Neutron) provides the ability to create a network architecture within your cloud environment. The project provides several integration points for Software Defined Networking (SDN) vendors. These integration points usually fall into the categories of plugins or agents

A plugin allows extension and customization of pre-existing Neutron functions. Vendors can write plugins to ensure interoperability between Neutron and certified software and hardware. Most vendors should aim to develop a driver for Neutron’s Modular Layer 2 (ml2) plugin, which provides a modular backend for integrating your own drivers.

An agent provides a specific network function. The main Neutron server (and its plugins) communicate with Neutron agents. Existing examples include agents for DHCP, Layer 3 support, and bridging support.

For both plugins and agents, you can either:

It is recommended to analyze the functionality of existing plugins and agents so you can determine how to integrate your own certified hardware and software. In particular, it is recommended to first develop a driver as a part of the ml2 plugin.

Upstream Repositories:

Upstream Blueprints:

Puppet Module:

Bugzilla components:

Integration Notes:

OpenStack Block Storage (Cinder) provides an API that interacts with block storage devices, which OpenStack uses to create volumes. For example, Cinder provides virtual storage devices for instances. Cinder provides a core set of drivers to support different storage hardware and protocols. For example, some of the core drivers include support for NFS, iSCSI, and Red Hat Ceph Storage. Vendors can include drivers to support additional certified hardware.

Vendors have two main options with the drivers and configuration they develop:

It is recommended to analyze the functionality of existing drivers so you can determine how to integrate your own certified hardware and software.

Upstream Repositories:

Upstream Blueprints:

Puppet Module:

Bugzilla components:

Integration Notes:

OpenStack Image Storage (Cinder) provides an API that interacts with storage types to provide storage for images. Glance provides a core set of drivers to support different storage hardware and protocols. For example, the core drivers include support for file, OpenStack Object Storage (Swift), OpenStack Block Storage (Cinder), and Red Hat Ceph Storage. Vendors can include drivers to support additional certified hardware.

Upstream Repositories:

Upstream Blueprints:

Puppet Module:

Bugzilla components:

Integration Notes:

OpenStack Shared File System Service (Manila) provides an API for shared and distributed file system services. Vendors can include drivers to support additional certified hardware.

Upstream Repositories:

Upstream Blueprints:

Puppet Module:

Bugzilla components:

Integration Notes:

The Cisco Nexus 1000V is a network switch designed for virtual machine access. It also provides advanced switching and security using VXLANs, ACLs, and IGMP snooping. The ml2 driver for the Cisco Nexus 1000V is contained in the networking-cisco repository, which you can install alongside the Neutron service.

The Overcloud image contains the Neutron Puppet module (puppet-neutron), which includes a class (neutron::plugins::ml2::cisco::nexus1000v) to configure Neutron to use the Cisco Nexus 1000V. This class is located in the manifests/plugins/ml2/cisco/nexus1000v.pp manifest from the module. The class uses a set of default parameters, which you can override, and then uses the neutron_plugin_ml2 library to configure the ml2 plugin to use the Cisco Nexus 1000V:

neutron_plugin_ml2 {
  'ml2/extension_drivers'                          : value => $extension_drivers;
  'ml2_cisco_n1kv/n1kv_vsm_ips'                    : value => $n1kv_vsm_ip;
  'ml2_cisco_n1kv/username'                        : value => $n1kv_vsm_username;
  'ml2_cisco_n1kv/password'                        : value => $n1kv_vsm_password;
  'ml2_cisco_n1kv/default_policy_profile'          : value => $default_policy_profile;
  'ml2_cisco_n1kv/default_vlan_network_profile'    : value => $default_vlan_network_profile;
  'ml2_cisco_n1kv/default_vxlan_network_profile'   : value => $default_vxlan_network_profile;
  'ml2_cisco_n1kv/poll_duration'                   : value => $poll_duration;
  'ml2_cisco_n1kv/http_pool_size'                  : value => $http_pool_size;
  'ml2_cisco_n1kv/http_timeout'                    : value => $http_timeout;
  'ml2_cisco_n1kv/sync_interval'                   : value => $sync_interval;
  'ml2_cisco_n1kv/max_vsm_retries'                 : value => $max_vsm_retries;
  'ml2_cisco_n1kv/restrict_policy_profiles'        : value => $restrict_policy_profiles;
  'ml2_cisco_n1kv/enable_vif_type_n1kv'            : value => $enable_vif_type_n1kv;
}

The director’s Heat template collection contains an environment file and registered templates to configure the Hiera data for the Cisco Nexus 1000V. The environment file is located in environments/cisco-n1kv-config.yaml and contains the following default content:

resource_registry:
  OS::TripleO::ControllerExtraConfigPre: ../puppet/extraconfig/pre_deploy/controller/cisco-n1kv.yaml
  OS::TripleO::ComputeExtraConfigPre: ../puppet/extraconfig/pre_deploy/controller/cisco-n1kv.yaml

parameter_defaults:
  N1000vVSMIP: '192.0.2.50'
  N1000vMgmtGatewayIP: '192.0.2.1'
  N1000vVSMDomainID: '100'
  N1000vVSMHostMgmtIntf: 'br-ex'

The resource_registry sets the preconfiguration resources for Controller and Compute nodes (OS::TripleO::ControllerExtraConfigPre and OS::TripleO::ComputeExtraConfigPre) to use puppet/extraconfig/pre_deploy/controller/cisco-n1kv.yaml as the template to use for preconfiguration. The parameter_defaults section includes some parameters to pass to these resources.

Including this environment file in the deployment defines the Hiera data, which the Puppet uses for the Neutron Puppet module’s parameters during configuration.

Starting the actual application of the Puppet configuration is automatic. The Heat template collection contains a set of core Puppet manifests for configuring the Controller and Compute nodes. These files contain logic that detects if the Cisco Nexus 1000V Hiera data is set. If so (by including cisco-n1kv.yaml in your deployment), the manifest includes the neutron::plugins::ml2::cisco::nexus1000v class as well as the Cisco Nexus 1000V’s VEM and VSM agents:

  if 'cisco_n1kv' in hiera('neutron_mechanism_drivers') {
    include neutron::plugins::ml2::cisco::nexus1000v

    class { 'neutron::agents::n1kv_vem':
      n1kv_source          => hiera('n1kv_vem_source', undef),
      n1kv_version         => hiera('n1kv_vem_version', undef),
    }

    class { 'n1k_vsm':
      n1kv_source       => hiera('n1kv_vsm_source', undef),
      n1kv_version      => hiera('n1kv_vsm_version', undef),
    }
  }

This means configuring the Overcloud to use Cisco Nexus 1000V only requires a few steps:

This defines the Cisco Nexus 1000V configuration as a part of the Overcloud’s Hiera data. Then the Overcloud uses this Hieradata to configure Neutron’s Nexus 1000V ml2 driver during the core configuration.

This example demonstrates how the director integrates network components from a certified vendor with the Overcloud’s Neutron service.

NetApp provides several solutions for integration with OpenStack storage components. This example shows the how NetApp Storage integrates with Cinder to provide a backend for block storage.

The drivers for Cinder are contained within the project itself, which is publically available on GitHub at https://github.com/openstack/cinder. The drivers for NetApp Storage are located in the cinder/volume/drivers/netapp/ directory of the repository. This means the drivers are automatically included with Red Hat OpenStack Platform.

The configuration for NetApp is contained in the Puppet module for cinder (puppet-cinder), which the Overcloud image also contains. The manifest in the Puppet modules that contains the configuration is located at manifests/backend/netapp.pp. This manifest uses the cinder_config library to add netapp settings to the Cinder configuration files:

cinder_config {
  "${name}/nfs_mount_options":            value => $nfs_mount_options;
  "${name}/volume_backend_name":          value => $volume_backend_name;
  "${name}/volume_driver":                value => 'cinder.volume.drivers.netapp.common.NetAppDriver';
  "${name}/netapp_login":                 value => $netapp_login;
  "${name}/netapp_password":              value => $netapp_password, secret => true;
  "${name}/netapp_server_hostname":       value => $netapp_server_hostname;
  "${name}/netapp_server_port":           value => $netapp_server_port;
  "${name}/netapp_size_multiplier":       value => $netapp_size_multiplier;
  "${name}/netapp_storage_family":        value => $netapp_storage_family;
  "${name}/netapp_storage_protocol":      value => $netapp_storage_protocol;
  "${name}/netapp_transport_type":        value => $netapp_transport_type;
  "${name}/netapp_vfiler":                value => $netapp_vfiler;
  "${name}/netapp_volume_list":           value => $netapp_volume_list;
  "${name}/netapp_vserver":               value => $netapp_vserver;
  "${name}/netapp_partner_backend_name":  value => $netapp_partner_backend_name;
  "${name}/expiry_thres_minutes":         value => $expiry_thres_minutes;
  "${name}/thres_avl_size_perc_start":    value => $thres_avl_size_perc_start;
  "${name}/thres_avl_size_perc_stop":     value => $thres_avl_size_perc_stop;
  "${name}/nfs_shares_config":            value => $nfs_shares_config;
  "${name}/netapp_copyoffload_tool_path": value => $netapp_copyoffload_tool_path;
  "${name}/netapp_controller_ips":        value => $netapp_controller_ips;
  "${name}/netapp_sa_password":           value => $netapp_sa_password, secret => true;
  "${name}/netapp_storage_pools":         value => $netapp_storage_pools;
  "${name}/netapp_eseries_host_type":     value => $netapp_eseries_host_type;
  "${name}/netapp_webservice_path":       value => $netapp_webservice_path;
}

The director’s Heat template collection contains an environment file and registered templates to configure the Hiera data for a NetApp Storage backend. The environment file is located in environments/cinder-netapp-config.yaml and contains the following default content:

resource_registry:
  OS::TripleO::ControllerExtraConfigPre: ../puppet/extraconfig/pre_deploy/controller/cinder-netapp.yaml

parameter_defaults:
  CinderEnableNetappBackend: true
  CinderNetappBackendName: 'tripleo_netapp'
  CinderNetappLogin: ''
  CinderNetappPassword: ''
  CinderNetappServerHostname: ''
  CinderNetappServerPort: '80'
  CinderNetappSizeMultiplier: '1.2'
  CinderNetappStorageFamily: 'ontap_cluster'
  CinderNetappStorageProtocol: 'nfs'
  CinderNetappTransportType: 'http'
  CinderNetappVfiler: ''
  CinderNetappVolumeList: ''
  CinderNetappVserver: ''
  CinderNetappPartnerBackendName: ''
  CinderNetappNfsShares: ''
  CinderNetappNfsSharesConfig: '/etc/cinder/shares.conf'
  CinderNetappNfsMountOptions: ''
  CinderNetappCopyOffloadToolPath: ''
  CinderNetappControllerIps: ''
  CinderNetappSaPassword: ''
  CinderNetappStoragePools: ''
  CinderNetappEseriesHostType: 'linux_dm_mp'
  CinderNetappWebservicePath: '/devmgr/v2'

The resource_registry sets the preconfiguration resources for Controller nodes (OS::TripleO::ControllerExtraConfigPre) to use puppet/extraconfig/pre_deploy/controller/cinder-netapp.yaml as the template to use for preconfiguration. The parameter_defaults section includes some parameters to pass to these resources.

Including this environment file in the deployment defines the Hiera data, which the Puppet uses for the Cinder Puppet module’s parameters during configuration.

Starting the actual application of the Puppet configuration depends on the CinderEnableNetappBackend parameter. The Heat template collection contains a set of core Puppet manifests for configuring Controller nodes. These files contain logic that detects if the cinder_enable_netapp_backend Hiera data is set. The Hiera data is set using the CinderEnableNetappBackend parameter in the preconfiguration. Including cinder-netapp-config.yaml in your deployment and leaving the CinderEnableNetappBackend: true as is means the Controller Puppet manifest includes the cinder::backend::netapp class and passes the Hiera data values from the environment file:

  if hiera('cinder_enable_netapp_backend', false) {
    $cinder_netapp_backend = hiera('cinder::backend::netapp::title')

    cinder_config {
      "${cinder_netapp_backend}/host": value => 'hostgroup';
    }

    if hiera('cinder::backend::netapp::nfs_shares', undef) {
      $cinder_netapp_nfs_shares = split(hiera('cinder::backend::netapp::nfs_shares', undef), ',')
    }

    cinder::backend::netapp { $cinder_netapp_backend :
      netapp_login                 => hiera('cinder::backend::netapp::netapp_login', undef),
      netapp_password              => hiera('cinder::backend::netapp::netapp_password', undef),
      netapp_server_hostname       => hiera('cinder::backend::netapp::netapp_server_hostname', undef),
      netapp_server_port           => hiera('cinder::backend::netapp::netapp_server_port', undef),
      netapp_size_multiplier       => hiera('cinder::backend::netapp::netapp_size_multiplier', undef),
      netapp_storage_family        => hiera('cinder::backend::netapp::netapp_storage_family', undef),
      netapp_storage_protocol      => hiera('cinder::backend::netapp::netapp_storage_protocol', undef),
      netapp_transport_type        => hiera('cinder::backend::netapp::netapp_transport_type', undef),
      netapp_vfiler                => hiera('cinder::backend::netapp::netapp_vfiler', undef),
      netapp_volume_list           => hiera('cinder::backend::netapp::netapp_volume_list', undef),
      netapp_vserver               => hiera('cinder::backend::netapp::netapp_vserver', undef),
      netapp_partner_backend_name  => hiera('cinder::backend::netapp::netapp_partner_backend_name', undef),
      nfs_shares                   => $cinder_netapp_nfs_shares,
      nfs_shares_config            => hiera('cinder::backend::netapp::nfs_shares_config', undef),
      netapp_copyoffload_tool_path => hiera('cinder::backend::netapp::netapp_copyoffload_tool_path', undef),
      netapp_controller_ips        => hiera('cinder::backend::netapp::netapp_controller_ips', undef),
      netapp_sa_password           => hiera('cinder::backend::netapp::netapp_sa_password', undef),
      netapp_storage_pools         => hiera('cinder::backend::netapp::netapp_storage_pools', undef),
      netapp_eseries_host_type     => hiera('cinder::backend::netapp::netapp_eseries_host_type', undef),
      netapp_webservice_path       => hiera('cinder::backend::netapp::netapp_webservice_path', undef),
    }
  }

This means configuring the Overcloud to use NetApp Storage only requires a few steps:

This defines the NetApp Storage configuration as a part of the Overcloud’s Hiera data. Then the Overcloud uses this Hieradata to configure Cinder’s NetApp backend during the core configuration.

This example demonstrates how the director integrates storage components from a certified vendor with the Overcloud’s Cinder service.