Chapter 5. Monitoring broker runtime metrics

When you install AMQ Broker, a Prometheus metrics plugin is included in your installation. Prometheus is software built for monitoring large, scalable systems and storing historical runtime data over an extended time period. You must modify the broker configuration to enable the plugin. When enabled, the plugin collects runtime metrics for the broker and exports these to Prometheus format. You can then use Prometheus to review the metrics. You might also use a graphical tool such as Grafana to configure more advanced visualizations of the data.

Note

The Prometheus metrics plugin enables you to collect and export broker metrics in Prometheus format. However, Red Hat does not provide support for installation or configuration of Prometheus itself, nor of visualization tools such as Grafana. If you require support with installing, configuring, or running Prometheus or Grafana, visit the product websites for resources such as community support and documentation.

In addition to the broker metrics collected by the Prometheus plugin, you can modify the broker configuration to capture standard sets of metrics relating to the host Java Virtual Machine (JVM) for the broker. Specifically, you can capture JVM metrics for Garbage Collection (GC), memory, and threads.

The sections that follow describe how to:

5.1. Overview of broker metrics

To monitor the health and performance of your broker instances, you can use the Prometheus plugin for AMQ Broker to monitor and store broker runtime metrics. The AMQ Broker Prometheus plugin exports the broker runtime metrics to Prometheus format, enabling you to use Prometheus itself to visualize and run queries on the data.

You can also use a graphical tool, such as Grafana, to configure more advanced visualizations and dashboards for the metrics that the Prometheus plugin collects.

The metrics that the plugin exports to Prometheus format are listed below. A description of each metric is exported along with the metric itself.

Broker Metrics

  • address.memory.usage
  • connection.count
  • total.connection.count

Address Metrics

  • routed.message.count
  • unrouted.message.count

Queue Metrics

  • consumer.count
  • delivering.durable.message.count
  • delivering.durable.persistent.size
  • delivering.message.count
  • delivering.persistent.size
  • durable.message.count
  • durable.persistent.size
  • messages.acknowledged
  • messages.added
  • message.count
  • messages.killed
  • messages.expired
  • persistent.size
  • scheduled.durable.message.count
  • scheduled.durable.persistent.size
  • scheduled.message.count
  • scheduled.persistent.size

For higher-level broker metrics that are not listed above, you can calculate these by aggregating lower-level metrics. For example, to calculate total message count, you can aggregate the message.count metrics from all queues in your broker deployment.

Java Virtual Machine (JVM) metrics are also exported to Prometheus format.

5.2. Enabling the Prometheus metrics plugin for AMQ Broker

When you install AMQ Broker, a Prometheus metrics plugin is included in your installation. Although the plugin is already configured for use, you need to enable the plugin in your broker configuration. When enabled, the plugin collects runtime metrics for the broker and exports these to Prometheus format.

The following procedure shows how to enable the Prometheus plugin for AMQ Broker.

Procedure

  1. Copy the Prometheus metrics plugin .jar file from your AMQ Broker 7.7 extracted archive to the lib directory of your broker instance.

    $ cp amq-broker-7.7.0/lib/artemis-prometheus-metrics-plugin-1.0.0.CR1-redhat-00010.jar <broker-instance-dir>/lib
  2. Open the <broker-instance-dir>/etc/broker.xml configuration file.
  3. Enable the Prometheus plugin in the broker configuration. Add a <metrics> element with a <plugin> sub-element, configured as shown below.

    <metrics>
        <plugin class-name="org.apache.activemq.artemis.core.server.metrics.plugins.ArtemisPrometheusMetricsPlugin"/>
    </metrics>
  4. Save the broker.xml configuration file. The metrics plugin starts to gather broker runtime metrics in Prometheus format.

5.3. Configuring the broker to collect JVM metrics

The following procedure shows how to configure the broker to collect Java Virtual Machine (JVM) metrics for Garbage Collection (GC), memory, and threads.

Prerequisites

Procedure

  1. Open the <broker-instance-dir>/etc/broker.xml configuration file.
  2. In the <metrics> element that you added to the configuration when enabling the Prometheus metrics plugin, specify whether the broker collects JVM metrics for Garbage Collection (GC), memory, and threads. For example:

    <metrics>
        <jvm-gc>true</jvm-gc>
        <jvm-memory>true</jvm-memory
        <jvm-threads>true</jvm-threads>
        <plugin class-name="org.apache.activemq.artemis.core.server.metrics.plugins.ArtemisPrometheusMetricsPlugin"/>
    </metrics>
    Note

    If you do not explicitly add the jvm-memory parameter to your configuration and specify a value, the broker uses a default value of true. This means that the broker exports JVM memory metrics by default. The default values of the jvm-gc and jvm-threads parameters are false.

  3. Save the broker.xml configuration file. The broker starts to gather the JVM metrics that you have enabled. These metrics are also exported to Prometheus format.

5.4. Disabling metrics collection for specific addresses

When you configure a metrics plugin for AMQ Broker (for example, the Prometheus metrics plugin), metrics collection is enabled by default. However, within the address-setting configuration element of a specific address or set of addresses, you can explicitly disable metrics collection.

The following procedure shows how disable metrics collection for a specific address or set of addresses.

Procedure

  1. Open the <broker-instance-dir>/etc/broker.xml configuration file.
  2. In the address-setting element of a matching address or set of addresses, add the enable-metrics parameter and set the value of the parameter to false. For example, the following configuration disables metrics collection for an address called orders.

    <configuration>
      <core>
        ...
        <address-settings>
           <address-setting match="orders">
              ...
              <enable-metrics>false</enable-metrics>
              ...
           </address-setting>
        </address-settings>
        ...
      </core>
    </configuration>

5.5. Accessing broker runtime data using Prometheus

Prerequisites

  • To query and visualize the broker runtime data collected by the Prometheus plugin, you need to install Prometheus. For more information, see Installing Prometheus in the Prometheus documentation.

Procedure

  1. From your Prometheus installation directory, open the prometheus.yml configuration file.
  2. In the static_configs section of the configuration file, change the targets element to localhost:8161. This location is where the broker runs its web server. By default, /metrics is appended to this host name, forming the full path to the metrics stored on the broker web server.
  3. To view the broker runtime metrics collected by the Prometheus plugin, open localhost:8161/metrics in a web browser.

    On the resulting web page, you see the current values of the metrics collected by the plugin, based on the queues and addresses that you have configured on the broker. If you have more than one running broker instance in your JVM, you see metrics for each broker.

  4. From your Prometheus installation directory, run Prometheus.

    $ ./prometheus

    When Prometheus starts, the shell output includes the following line:

    component=web, msg=”Start listening for connections” address=0.0.0.0:9090

    The preceding line indicates that Prometheus is listening for HTTP traffic on port 9090.

  5. To access the Prometheus web console, open 127.0.0.1:9090 in a web browser.
  6. In the Prometheus web console, you can use the Expression field to create a query on your broker data. The queries you create are based on the Prometheus query language, PromQL. Broker metrics that are available to insert in your query are in the Insert metric drop-down list.

    As a simple example, suppose you want to query the message count on the DLQ queue, over time. In this case, select artemis_message_count from the metrics drop-down list. Complete your query by specifying the DLQ queue name and address. This example query is shown below.

    artemis_message_count{address=“DLQ”, queue=“DLQ”}

    For more advanced visualizations, you can use regular expressions to create complex queries that overlay several metrics, for example. Or, you can perform mathematical operations on a number of metrics, such as aggregating them. For more information about creating Prometheus queries, see Querying Prometheus in the Prometheus documentation.

Revised on 2020-07-02 13:59:35 UTC