Chapter 2. Planning for operational measurements

The resources that you monitor depend on your business requirements. You can use Ceilometer or collectd to monitor your resources.

2.1. Ceilometer measurements

For a full list of Ceilometer measures, see https://docs.openstack.org/ceilometer/train/admin/telemetry-measurements.html

2.2. Collectd measurements

The following measurements are the most commonly used collectd metrics:

  • disk
  • interface
  • load
  • memory
  • processes
  • tcpconns

For a full list of measurements, see collectd metrics and events.

2.3. Planning for data storage

Gnocchi stores a collection of data points, where each data point is an aggregate. The storage format is compressed using different techniques. As a result, to calculate the size of a time-series database, you must estimate the size based on the worst-case scenario.

Procedure

  1. Calculate the number of data points:

    number of points = timespan / granularity

    For example, if you want to retain a year of data with one-minute resolution, use the formula:

    number of data points = (365 days X 24 hours X 60 minutes) / 1 minute number of data points = 525600

  2. Calculate the size of the time-series database:

    size in bytes = number of data points X 8 bytes

    If you apply this formula to the example, the result is 4.1 MB:

    size in bytes = 525600 points X 8 bytes = 4204800 bytes = 4.1 MB

    This value is an estimated storage requirement for a single aggregated time-series database. If your archive policy uses multiple aggregation methods (min, max, mean, sum, std, count), multiply this value by the number of aggregation methods you use.

Additional resources

2.4. Planning and managing archive policies

An archive policy defines how you aggregate the metrics and for how long you store the metrics in the time-series database. An archive policy is defined as the number of points over a timespan.

If your archive policy defines a policy of 10 points with a granularity of 1 second, the time series archive keeps up to 10 seconds, each representing an aggregation over 1 second. This means that the time series retains, at a maximum, 10 seconds of data between the more recent point and the older point. The archive policy also defines the aggregate method to use. The default is set to the parameter default_aggregation_methods, where the default values are set to mean, min, max. sum, std, count. So, depending on the use case, the archive policy and the granularity can vary.

To plan an archive policy, ensure that you are familiar with the following concepts:

To create and manage an archive police, complete the following tasks:

  1. Create an archive policy. For more information, see Section 2.4.6, “Creating an archive policy”.
  2. Manage an archive policy. For more information, see Section 2.4.7, “Managing archive policies”.
  3. Create an archive policy rule. For more information, see Section 2.4.8, “Creating an archive policy rule”.

2.4.1. Metrics

Gnocchi provides an object type called metric. A metric is anything that you can measure, for example, the CPU usage of a server, the temperature of a room, or the number of bytes sent by a network interface. A metric has the following properties:

  • A UUID to identify it
  • A name
  • The archive policy used to store and aggregate the measures

Additional resources

2.4.1.1. Creating a metric

Procedure

  1. Create a resource. Replace <resource_name> with the name of the resource: 

    $ openstack metric resource create <resource_name>

  2. Create the metric. Replace <resource_name> with the name of the resource and <metric_name> with the name of the metric: 

    $ openstack metric metric create -r <resource_name> <metric_name>

    When you create the metric, the archive policy attribute is fixed and unchangeable. You can change the definition of the archive policy through the archive_policy endpoint.

2.4.2. Creating custom measures

A measure is an incoming datapoint tuple that the API sends to Gnocchi. It is composed of a timestamp and a value. You can create your own custom measures.

Procedure

  • Create a custom measure: 

    $ openstack metric measures add -m <MEASURE1> -m <MEASURE2> .. -r <RESOURCE_NAME> <METRIC_NAME>

2.4.3. Default archive policies

By default, Gnocchi has the following archive policies:

  • low

    • 5 minutes granularity over 30 days
    • aggregation methods used: default_aggregation_methods
    • maximum estimated size per metric: 406 KiB

  • medium

    • 1 minute granularity over 7 days
    • 1 hour granularity over 365 days
    • aggregation methods used: default_aggregation_methods
    • maximum estimated size per metric: 887 KiB

  • high

    • 1 second granularity over 1 hour
    • 1 minute granularity over 1 week
    • 1 hour granularity over 1 year
    • aggregation methods used: default_aggregation_methods
    • maximum estimated size per metric: 1 057 KiB

  • bool

    • 1 second granularity over 1 year
    • aggregation methods used: last
    • maximum optimistic size per metric: 1 539 KiB
    • maximum pessimistic size per metric: 277 172 KiB

2.4.4. Calculating the size of a time-series aggregate

Gnocchi stores a collection of data points, where each point is an aggregate. The storage format is compressed using different techniques. As a result, calculating the size of a time-series is estimated based on a worst case scenario, as shown in the following example.

Procedure

  1. Use this formula to calculate the number of points:

    number of points = timespan / granularity

    For example, if you want to keep a year of data with one-minute resolution:

    number of points = (365 days X 24 hours X 60 minutes) / 1 minute

    number of points = 525600

  2. To calculate the point size in bytes, use this formula:

    size in bytes = number of points X 8 bytes

    size in bytes = 525600 points X 8 bytes = 4204800 bytes = 4.1 MB

    This value is an estimated storage requirement for a single aggregated time-series. If your archive policy uses multiple aggregation methods - min, max, mean, sum, std, count - multiply this value by the number of aggregation methods you use.

2.4.5. Metricd workers

You can use the metricd daemon to processes measures, create aggregates, store measures in aggregate storage and delete metrics. The metricd daemon is responsible for most CPU usage and I/O jobs in Gnocchi. The archive policy of each metric determines how fast the metricd daemon performs. Metricd checks the incoming storage for new measures periodically. To configure the delay between each check, you can use the [metricd]metric_processing_delay configuration option.

2.4.6. Creating an archive policy

Procedure

  • Create an archive policy. Replace <archive-policy-name> with the name of the policy and <aggregation-method> with the method of aggregation. 

    # openstack metric archive policy create <archive-policy-name> --definition <definition> \
--aggregation-method <aggregation-method>
    Note

    <definition> is the policy definition. Separate multiple attributes with a comma (,). Separate the name and value of the archive policy definition with a colon (:).

2.4.7. Managing archive policies

  • To delete an archive policy: 

    openstack metric archive policy delete <archive-policy-name>

  • To view all archive policies: 

    # openstack metric archive policy list

  • To view the details of an archive policy: 

    # openstack metric archive-policy show <archive-policy-name>

2.4.8. Creating an archive policy rule

An archive policy rule defines a mapping between a metric and an archive policy. This gives users the ability to predefine rules so an archive policy is assigned to metrics based on a matched pattern.

Procedure

  • Create an archive policy rule. Replace <rule-name> with the name of the rule and <archive-policy-name> with the name of the archive policy: 

    # openstack metric archive-policy-rule create <rule-name> /
--archive-policy-name  <archive-policy-name>

2.5. Verifying the Red Hat OpenStack Platform deployment

You can use the openstack metric command to verify a successful deployment.

Procedure

  • Verify the deployment: 

    (overcloud) [stack@undercloud-0 ~]$ openstack metric status
+-----------------------------------------------------+-------+
| Field                                           	| Value |
+-----------------------------------------------------+-------+
| storage/number of metric having measures to process | 0 	|
| storage/total number of measures to process     	| 0 	|
+-----------------------------------------------------+-------+