Chapter 5. Monitoring broker runtime data using Prometheus

5.1. Overview

To monitor the health and performance of your broker instances, you can use the Prometheus plugin for AMQ Broker to monitor and store broker runtime metrics. Prometheus is software built for monitoring large, scalable systems and storing historical runtime data over an extended time period. The AMQ Broker Prometheus plugin exports the broker runtime metrics to Prometheus format, enabling you to use Prometheus itself to visualize and run queries on the data.

You can also use a graphical tool, such as Grafana, to configure more advanced visualizations and dashboards for the metrics that the Prometheus plugin collects.

For AMQ Broker, the metrics exported to Prometheus format by the plugin are listed below. A description of each metric is exported along with the metric itself.

Broker Metrics

  • address.memory.usage
  • connection.count
  • total.connection.count

Address Metrics

  • routed.message.count
  • unrouted.message.count

Queue Metrics

  • consumer.count
  • delivering.durable.message.count
  • delivering.durable.persistent.size
  • delivering.message.count
  • delivering.persistent.size
  • durable.message.count
  • durable.persistent.size
  • messages.acknowledged
  • messages.added
  • message.count
  • messages.killed
  • messages.expired
  • persistent.size
  • scheduled.durable.message.count
  • scheduled.durable.persistent.size
  • scheduled.message.count
  • scheduled.persistent.size

For higher-level broker metrics that are not listed above, you can calculate these by aggregating lower-level metrics. For example, to calculate total message count, you can aggregate the message.count metrics from all queues in your broker deployment.

Java Virtual Machine (JVM) memory metrics are also exported to Prometheus format.

5.2. Enabling the Prometheus plugin for AMQ Broker

When you install AMQ Broker, the Prometheus plugin is included in your installation and is already configured for use. However, you need to enable the plugin in your broker configuration.

The following procedure shows you how to enable the Prometheus plugin and start gathering metrics for running broker instances.

Procedure

  1. Copy the Prometheus metrics plugin .jar file from your AMQ Broker 7.4 extracted archive to the lib directory of your broker instance.

    $ cp amq-broker-7.4.0/lib/artemis-prometheus-metrics-plugin-1.0.0.CR1-redhat-00002.jar BROKER_INSTANCE_DIR/lib
  2. Add the following line to your broker.xml configuration file.

    <metrics-plugin class-name="org.apache.activemq.artemis.core.server.metrics.plugins.ArtemisPrometheusMetricsPlugin"/>

When you have enabled the Prometheus plugin, the plugin starts to gather the metrics listed in the previous section, for any broker that you start.

5.3. Accessing broker runtime data using Prometheus

Prerequisites

  • To query and visualize the broker runtime data collected by the Prometheus plugin, you need to install Prometheus. For more information, see Installing Prometheus.

Procedure

  1. From your Prometheus installation directory, open the prometheus.yml configuration file.
  2. In the static_configs section of the configuration file, change the targets element to localhost:8161. This location is where the broker runs its web server. By default, /metrics is appended to this host name, forming the full path to the metrics stored on the broker web server.
  3. To view the broker runtime metrics collected by the Prometheus plugin, open localhost:8161/metrics in a web browser.

    On the resulting web page, you see the current values of the metrics collected by the plugin, based on the queues and addresses that you have configured on the broker. If you have more than one running broker instance in your JVM, you see metrics for each broker.

  4. From your Prometheus installation directory, run Prometheus.

    $ ./prometheus

    When Prometheus starts, the shell output includes the following line:

    component=web, msg=”Start listening for connections” address=0.0.0.0:9090

    The preceding line indicates that Prometheus is listening for HTTP traffic on port 9090.

  5. To access the Prometheus web console, open 127.0.0.1:9090 in a web browser.
  6. In the Prometheus web console, you can use the Expression field to create a query on your broker data. The queries you create are based on the Prometheus query language, PromQL. Broker metrics that are available to insert in your query are in the Insert metric drop-down list.

    As a simple example, suppose you want to query the message count on the DLQ queue, over time. In this case, select artemis_message_count from the metrics drop-down list. Complete your query by specifying the DLQ queue name and address. This example query is shown below.

    artemis_message_count{address=“DLQ”, queue=“DLQ”}

    For more advanced visualizations, you can use regular expressions to create complex queries that overlay several metrics, for example. Or, you can perform mathematical operations on a number of metrics, such as aggregating them. For more information about creating Prometheus queries, see Querying Prometheus.

Revised on 2021-05-14 10:16:09 UTC