Red Hat Training

A Red Hat training course is available for Red Hat OpenStack Platform

Chapter 3. Install the Client-Side Tools

Before you deploy the overcloud, you need to determine the configuration settings to apply to each client. Copy the example environment files from the director’s Heat template collection and modify them to suit your environment.

Note

If your deployment uses containerized services, the environment files are available at /usr/share/openstack-tripleo-heat-templates/docker/services/*.

3.1. Set Centralized Logging Client Parameters

For Fluentd configuration settings, copy /usr/share/openstack-tripleo-heat-templates/environments/logging-environment.yaml and modify the file to suit your environment. For example:

Simple configuration

resource_registry:
  OS::TripleO::Services::Fluentd: ../puppet/services/logging/fluentd.yaml

parameter_defaults:
  LoggingServers:
    - host: log0.example.com
      port: 24224
    - host: log1.example.com
      port: 24224

Example SSL configuration

## (note the use of port 24284 for ssl connections)

resource_registry:
  OS::TripleO::Services::Fluentd: ../puppet/services/logging/fluentd.yaml

parameter_defaults:
  LoggingServers:
    - host: 192.0.2.11
      port: 24284
  LoggingUsesSSL: true
  LoggingSharedKey: secret
  LoggingSSLCertificate: |
    -----BEGIN CERTIFICATE-----
    ...certificate data here...
    -----END CERTIFICATE-----

  • LoggingServers - The destination system that will receive Fluentd log messages.
  • LoggingUsesSSL - Setting that determines whether secure_forward is used when forwarding log messages.
  • LoggingSharedKey - The shared secret used by secure_forward.
  • LoggingSSLCertificate - The PEM-encoded contents of the SSL CA certificate.

3.2. Set Availability Monitoring Client Parameters

For the Sensu client configuration settings, copy /usr/share/openstack-tripleo-heat-templates/environments/monitoring-environment.yaml and modify the file to suit your environment. For example:

resource_registry:
  OS::TripleO::Services::SensuClient: ../puppet/services/monitoring/sensu-client.yaml

parameter_defaults:
  MonitoringRabbitHost: 10.10.10.10
  MonitoringRabbitPort: 5672
  MonitoringRabbitUserName: sensu
  MonitoringRabbitPassword: sensu
  MonitoringRabbitUseSSL: true
  MonitoringRabbitVhost: "/sensu"
  SensuClientCustomConfig:
    api:
      warning: 10
      critical: 20
  • MonitoringRabbit - These parameters connect the Sensu client services to the RabbitMQ instance that runs on the monitoring server.
  • MonitoringRabbitUseSSL - Enables SSL for the RabbitMQ client. Uses SSL transport if the private key or certificate chain are not specified, as below.
  • MonitoringRabbitSSLPrivateKey - Defines the path to the private key file, or can contain the contents of that file.
  • MonitoringRabbitSSLCertChain - Defines the private SSL certificate chain to use.
  • SensuClientCustomConfig - Specify additional Sensu client configuration. Defines the OpenStack credentials to be used, including username/password, auth_url, tenant, and region.

3.3. Set Performance Monitoring Client Parameters

Performance monitoring collects system information periodically and provides the mechanism to store and monitor the values in a variety of ways using the collectd daemon. The collectd daemon stores the data it collects, like operating system and log files, or makes it available over the network. You can use these statistics to monitor systems, find performance bottlenecks, and predict future system load.

Red Hat OpenStack Platform supports performance monitoring (collectd) only on the client side (the overcloud nodes).

  1. Make a copy of /usr/share/openstack-tripleo-heat-templates/environments/collectd-environment.yaml file for the monitoring server and modify it to include the parameters defaults as follows:

    parameter_defaults:
      CollectdServer: <MONITORING_SERVER_IP_ADDRESS>
      CollectdServerPort: 25826
      CollectdSecurityLevel: None
    
      CollectdDefaultPlugins:
          - load
          - memory
          - processes
    • CollectdServer - Address of remote collectd server where the metrics are sent.
    • CollectdServerPort - Port for collectd server.
    • CollectdSecurityLevel - Security level setting for remote collectd connection. By default, the security level is None.

      If the CollectdSecurityLevel parameter is set to Encrypt or Sign, you need to set the CollectdUsername: user and CollectdPassword: password parameters for authentication.

    • CollectdDefaultPlugins - By default, collectd comes with the disk, interface, load, memory, processes, and tcpconns plugins. You can add extra plugins using the CollectdExtraPlugins parameter.

3.4. Install Operational Tools on Overcloud Nodes

Include the modified YAML files with your openstack overcloud deploy command to install the Sensu client, Fluentd tools, and collectd daemon on all overcloud nodes. For example:

$ openstack overcloud deploy \
--templates /usr/share/openstack-tripleo-heat-templates \
-r /home/stack/roles_data.yaml \
-e /home/stack/templates/overcloud_images.yaml \
-e /home/stack/parameter.yaml
-e /usr/share/openstack-tripleo-heat-templates/environments/network-isolation.yaml \
-e network-environment.yaml \
-e ~/templates/monitoring-environment.yaml \
-e ~/templates/logging-environment.yaml --control-scale 3 --compute-scale 1 \
-e ~/templates/collectd-environment.yaml
--ntp-server 192.168.122.10

3.5. Filter and Tranform Logging Data

You can filter and transform events sent to Fluentd by setting the LoggingDefaultFilters parameter in your environment file. For example, the record_transformer type can modify incoming events:

parameter_defaults:

  LoggingDefaultFilters:
      - tag_pattern: '**'
        type: record_transformer
        enable_ruby: true
        record:
          openstack: '{"hostname": "${hostname}","tag": "${tag}","region": "regionOne","inputname":"fluent-plugin-in_tail","name":"fluentd openstack","fluentd version":"0.12.26", "pipeline_metadata": {"collector":{"ipaddr4":"#{ begin Socket.ip_address_list.select { |a| a.ipv4? && !a.ipv4_loopback? }.map { |a| a.ip_address } end}","ipaddr6":"#{ begin Socket.ip_address_list.select { |a| a.ipv6? && !a.ipv6_loopback? }.map { |a| a.ip_address } end}"}}}'
        remove_keys: host

      - tag_pattern: 'openstack.**'
        type: record_transformer
        record:
          service: '${tag_parts[1]}'

      - tag_pattern: 'system.messages**'
        type: record_transformer
        enable_ruby: true
        record:
          openstack: '{"info": "${record[''message'']}", "systemd":{"t":{"PID":"${record[''pid'']}"},"u":{"SYSLOG_IDENTIFIER":"${record[''ident'']}"}}}'
        remove_keys: 'ident,message,pid'

As a result, the data received by Kibana has been transformed accordingly:

{
  "_index": "logstash-2017.06.29",
  "_type": "fluentd",
  "_id": "AVz132QmRtyd8nnlv_11",
  "_score": null,
  "_source": {
    "pid": "22691",
    "priority": "INFO",
    "message": "cinder.api.openstack.wsgi [req-04bc2808-f86f-4443-86e6-bfc596969937 - - - - -] OPTIONS http://overcloud-controller-0.lab.local/",
    "openstack": {
      "hostname": "overcloud-controller-0",
      "tag": "openstack.cinder.api",
      "region": "regionOne",
      "inputname": "fluent-plugin-in_tail",
      "name": "fluentd openstack",
      "fluentd version": "0.12.26",
      "pipeline_metadata": {
        "collector": {
          "ipaddr4": "[\"192.168.24.14\", \"192.168.24.8\", \"10.0.0.4\", \"10.0.0.5\", \"172.16.2.8\", \"172.16.2.4\", \"172.16.2.14\", \"172.16.1.7\", \"172.16.1.9\", \"172.16.3.10\", \"172.16.3.11\", \"172.16.0.14\"]",
          "ipaddr6": "[\"fe80::293:33ff:fed8:2228%eth0\", \"fe80::293:33ff:fed8:2228%br-ex\", \"fe80::b86c:79ff:fe8f:9fb8%vlan10\", \"fe80::4c78:6fff:feff:14fc%vlan20\", \"fe80::9ced:dfff:fe8c:2d62%vlan30\", \"fe80::ecde:1bff:fe5d:e362%vlan40\", \"fe80::549c:51ff:feea:dfa8%vlan50\", \"fe80::e093:8fff:fef9:69b6%vxlan_sys_4789\"]"
        }
      }
    },
    "service": "cinder",
    "@timestamp": "2017-06-29T21:59:38+00:00"
  },
  "fields": {
    "@timestamp": [
      1498773578000
    ]
  },
  "sort": [
    1498773578000
  ]
}