Red Hat Training

A Red Hat training course is available for Red Hat OpenStack Platform

Chapter 4. Configuring additions to the OpenStack Puppet modules

This chapter explores how to provide additions to the OpenStack Puppet modules. This includes some basic guidelines on developing Puppet modules.

4.1. Puppet syntax and module structure

The following section provides a few basics to help you understand Puppet syntax and the structure of a Puppet module.

4.1.1. Anatomy of a Puppet module

Before you contribute to the OpenStack modules, you must understand the components that create a Puppet module.

⁠Manifests

Manifests are files that contain code to define a set of resources and their attributes. A resource is any configurable part of a system. Examples of resources include packages, services, files, users, and groups, SELinux configuration, SSH key authentication, cron jobs, and more. A manifest defines each required resource by using a set of key-value pairs for their attributes. 

  package { 'httpd':
    ensure => installed,
  }

For example, this declaration checks if the httpd package is installed. If not, the manifest executes dnf and installs it. Manifests are located in the manifest directory of a module. Puppet modules also use a test directory for test manifests. These manifests are used to test certain classes that are in your official manifests.

Classes
Classes unify multiple resources in a manifest. For example, if you install and configure a HTTP server, you might create a class with three resources: one to install the HTTP server packages, one to configure the HTTP server, and one to start or enable the server. You can also refer to classes from other modules, which applies their configuration. For example, if you want to configure an application that also required a webserver, you can refer to the previously mentioned class for the HTTP server.
⁠Static Files

Modules can contain static files that Puppet can copy to certain locations on your system. Define locations, and other attributes such as permissions, by using file resource declarations in manifests.

Static files are located in the files directory of a module.

⁠Templates

Sometimes configuration files require custom content. In this situation, users create a template instead of a static file. Like static files, templates are defined in manifests and copied to locations on a system. The difference is that templates allow Ruby expressions to define customized content and variable input. For example, if you want to configure httpd with a customizable port then the template for the configuration file includes: 

Listen <%= @httpd_port %>

The httpd_port variable in this case is defined in the manifest that references this template.

Templates are located in the templates directory of a module.

⁠Plugins

Use plugins for aspects that extend beyond the core functionality of Puppet. For example, you can use plugins to define custom facts, custom resources, or new functions. For example, a database administrator might need a resource type for PostgreSQL databases. This can help the database administrator populate PostgreSQL with a set of new databases after they install PostgreSQL. As a result, the database administrator must create only a Puppet manifest that ensures PostgreSQL installs and the databases are created afterwards.

Plugins are located in the lib directory of a module. This includes a set of subdirectories depending on the plugin type:

  • /lib/facter - Location for custom facts.
  • /lib/puppet/type - Location for custom resource type definitions, which outline the key-value pairs for attributes.
  • /lib/puppet/provider - Location for custom resource providers, which are used in conjunction with resource type definitions to control resources.
  • /lib/puppet/parser/functions - Location for custom functions.

4.1.2. Installing a service

Some software requires package installations. This is one function that a Puppet module can perform. This requires a resource definition that defines configurations for a certain package.

For example, to install the httpd package through the mymodule module, add the following content to a Puppet manifest in the mymodule module: 

class mymodule::httpd {
  package { 'httpd':
    ensure => installed,
  }
}

This code defines a subclass of mymodule called httpd, then defines a package resource declaration for the httpd package. The ensure => installed attribute tells Puppet to check if the package is installed. If it is not installed, Puppet executes yum to install it.

4.1.3. Starting and enabling a service

After you install a package, you might want to start the service. Use another resource declaration called service. Edit the manifest with the following content: 

class mymodule::httpd {
  package { 'httpd':
    ensure => installed,
  }
  service { 'httpd':
    ensure => running,
    enable => true,
    require => Package["httpd"],
  }
}

Result:

  • The ensure => running attribute checks if the service is running. If not, Puppet enables it.
  • The enable => true attribute sets the service to run when the system boots.
  • The require => Package["httpd"] attribute defines an ordering relationship between one resource declaration and another. In this case, it ensures that the httpd service starts after the httpd package installs. This creates a dependency between the service and its respective package.

4.1.4. Configuring a service

The HTTP server provides some default configuration in /etc/httpd/conf/httpd.conf, which provides a web host on port 80. However, you can add extra configuration to provide an additional web host on a user-specified port.

Procedure

  1. You must use a template file to store the HTTP configuration file because the user-defined port requires variable input. In the module templates directory, add a file called myserver.conf.erb with the following contents: 

    Listen <%= @httpd_port %>
NameVirtualHost *:<%= @httpd_port %>
<VirtualHost *:<%= @httpd_port %>>
  DocumentRoot /var/www/myserver/
  ServerName *:<%= @fqdn %>>
  <Directory "/var/www/myserver/">
    Options All Indexes FollowSymLinks
    Order allow,deny
    Allow from all
  </Directory>
</VirtualHost>

    This template follows the standard syntax for Apache web server configuration. The only difference is the inclusion of Ruby escape characters to inject variables from the module. For example, httpd_port, which you use to specify the web server port.

    The inclusion of fqdn is a variable that stores the fully qualified domain name of the system. This is known as a system fact. System facts are collected from each system before generating each Puppet catalog of a system. Puppet uses the facter command to gather these system facts and you can also run facter to view a list of these facts.

  2. Save myserver.conf.erb.

  3. Add the resource to the Puppet manifest of the module: 

    class mymodule::httpd {
  package { 'httpd':
    ensure => installed,
  }
  service { 'httpd':
    ensure => running,
    enable => true,
    require => Package["httpd"],
  }
  file {'/etc/httpd/conf.d/myserver.conf':
  notify => Service["httpd"],
    ensure => file,
    require => Package["httpd"],
    content => template("mymodule/myserver.conf.erb"),
  }
  file { "/var/www/myserver":
    ensure => "directory",
  }
}

Result:

  • You add a file resource declaration for the server configuration file, (/etc/httpd/conf.d/myserver.conf. The content for this file is the myserver.conf.erb template that you created.
  • You check the httpd package is installed before you add this file.
  • You add a second file resource declaration that creates a directory, /var/www/myserver, for your web server.
  • You add a relationship between the configuration file and the httpd service using the notify => Service["httpd"] attribute. This checks your configuration file for any changes. If the file has changed, Puppet restarts the service.

4.2. Obtaining OpenStack Puppet modules

The Red Hat OpenStack Platform uses the official OpenStack Puppet modules. To obtain OpenStack Puppet modules, see the openstack group on Github.

Procedure

  1. In your browser, go to https://github.com/openstack.
  2. In the filters section, search for Puppet. All Puppet modules use the prefix puppet-.

  3. Clone the Puppet module that you want. For example, the official OpenStack Block Storage (cinder) module: 

    $ git clone https://github.com/openstack/puppet-cinder.git

4.3. Example configuration of a Puppet module

The OpenStack modules primarily aim to configure the core service. Most modules also contain additional manifests to configure additional services, sometimes known as backends, agents, or plugins. For example, the cinder module contains a directory called backends, which contains configuration options for different storage devices including NFS, iSCSI, Red Hat Ceph Storage, and others.

For example, the manifests/backends/nfs.pp file contains the following configuration: 

define cinder::backend::nfs (
  $volume_backend_name  = $name,
  $nfs_servers          = [],
  $nfs_mount_options    = undef,
  $nfs_disk_util        = undef,
  $nfs_sparsed_volumes  = undef,
  $nfs_mount_point_base = undef,
  $nfs_shares_config    = '/etc/cinder/shares.conf',
  $nfs_used_ratio       = '0.95',
  $nfs_oversub_ratio    = '1.0',
  $extra_options        = {},
) {

  file {$nfs_shares_config:
    content => join($nfs_servers, "\n"),
    require => Package['cinder'],
    notify  => Service['cinder-volume']
  }

  cinder_config {
    "${name}/volume_backend_name":  value => $volume_backend_name;
    "${name}/volume_driver":        value =>
      'cinder.volume.drivers.nfs.NfsDriver';
    "${name}/nfs_shares_config":    value => $nfs_shares_config;
    "${name}/nfs_mount_options":    value => $nfs_mount_options;
    "${name}/nfs_disk_util":        value => $nfs_disk_util;
    "${name}/nfs_sparsed_volumes":  value => $nfs_sparsed_volumes;
    "${name}/nfs_mount_point_base": value => $nfs_mount_point_base;
    "${name}/nfs_used_ratio":       value => $nfs_used_ratio;
    "${name}/nfs_oversub_ratio":    value => $nfs_oversub_ratio;
  }

  create_resources('cinder_config', $extra_options)

}

Result:

  • The define statement creates a defined type called cinder::backend::nfs. A defined type is similar to a class; the main difference is Puppet evaluates a defined type multiple times. For example, you might require multiple NFS back ends and as such the configuration requires multiple evaluations for each NFS share.
  • The next few lines define the parameters in this configuration and their default values. The default values are overwritten if the user passes new values to the cinder::backend::nfs defined type.

  • The file function is a resource declaration that calls for the creation of a file. This file contains a list of the NFS shares and the name for this file is defined in the parameters $nfs_shares_config = '/etc/cinder/shares.conf. Note the additional attributes:

    • The content attribute creates a list by using the $nfs_servers parameter.
    • The require attribute ensures that the cinder package is installed.
    • The notify attribute tells the cinder-volume service to reset.

  • The cinder_config function is a resource declaration that uses a plugin from the lib/puppet/ directory in the module. This plugin adds configuration to the /etc/cinder/cinder.conf file. Each line in this resource adds a configuration options to the relevant section in the cinder.conf file. For example, if the $name parameter is mynfs, then the following attributes: 

      "${name}/volume_backend_name":  value => $volume_backend_name;
  "${name}/volume_driver":        value =>
    'cinder.volume.drivers.nfs.NfsDriver';
  "${name}/nfs_shares_config":    value => $nfs_shares_config;

    Save the following snippet to the cinder.conf file: 

    [mynfs]
volume_backend_name=mynfs
volume_driver=cinder.volume.drivers.nfs.NfsDriver
nfs_shares_config=/etc/cinder/shares.conf
  • The create_resources function converts a hash into a set of resources. In this case, the manifest converts the $extra_options hash to a set of additional configuration options for the backend. This provides a flexible method to add further configuration options that are not included in the core parameters of the manifest.

This shows the importance of including a manifest to configure the OpenStack driver of your hardware. The manifest provides a method for director to include configuration options that are relevant to your hardware. This acts as a main integration point for director to configure your overcloud to use your hardware.

4.4. Example of adding hiera data to a Puppet configuration

Puppet contains a tool called hiera, which acts as a key value system that provides node-specific configuration. These keys and their values are usually stored in files located in /etc/puppet/hieradata. The /etc/puppet/hiera.yaml file defines the order that Puppet reads the files in the hieradata directory.

During overcloud configuration, Puppet uses hiera data to overwrite the default values for certain Puppet classes. For example, the default NFS mount options for cinder::backend::nfs in puppet-cinder are undefined: 

  $nfs_mount_options    = undef,

However, you can create your own manifest that calls the cinder::backend::nfs defined type and replace this option with hiera data: 

  cinder::backend::nfs { $cinder_nfs_backend:
    nfs_mount_options   => hiera('cinder_nfs_mount_options'),
  }

This means the nfs_mount_options parameter takes uses hiera data value from the cinder_nfs_mount_options key: 

cinder_nfs_mount_options: rsize=8192,wsize=8192

Alternatively, you can use the hiera data to overwrite cinder::backend::nfs::nfs_mount_options parameter directly so that it applies to all evaluations of the NFS configuration: 

cinder::backend::nfs::nfs_mount_options: rsize=8192,wsize=8192

The above hiera data overwrites this parameter on each evaluation of cinder::backend::nfs.