Chapter 7. Using Prometheus and Grafana to monitor the router network

Prometheus is container-native software built for storing historical data and for monitoring large, scalable systems such as AMQ Interconnect. It gathers data over an extended time, rather than just for the currently running session.

You use Prometheus and Alertmanager to monitor and store AMQ Interconnect data so that you can use a graphical tool, such as Grafana, to visualize and run queries on the data.

7.1. Setting up Prometheus and Grafana

Before you can view AMQ Interconnect dashboards, you must deploy and configure Prometheus, Alertmanager, and Grafana in the OpenShift project in which AMQ Interconnect is deployed. All of the required configuration files are provided in a GitHub repository.

Procedure

  1. Clone the qdr-monitoring GitHub repository.

    This repository contains the configuration files needed to set up Prometheus and Grafana to monitor AMQ Interconnect. 

    $ git clone https://github.com/interconnectedcloud/qdr-monitoring

  2. Open the deploy-monitoring.sh script and set the NAMESPACE variable.

    Set NAMESPACE to be the name of the project into which you have deployed AMQ Interconnect. 

    #!/bin/bash

# Change the namespace to that of your project
NAMESPACE=myproject
...

  3. Run the deploy-monitoring.sh script.

    This script creates and configures the OpenShift resources needed to deploy Prometheus, Alertmanager, and Grafana in your OpenShift project. It also configures two dashboards that provide metrics for the router network. 

    $ ./deploy-monitoring.sh

  4. Create a Route for the prometheus, alertmanager, and grafana Services. 

    $ oc expose service prometheus

$ oc expose service alertmanager

$ oc expose service grafana

Additional resources

7.2. Viewing AMQ Interconnect dashboards in Grafana

After setting up Prometheus and Grafana, you can visualize the AMQ Interconnect data on the following Grafana dashboards:

Qpid Dispatch Router

Shows metrics for:

  • Deliveries ingress
  • Deliveries egress
  • Deliveries ingress route container
  • Deliveries egress route container
  • Deliveries redirected to fallback destination
  • Dropped presettled deliveries
  • Presettled deliveries
  • Auto links
  • Link routes
  • Address count
  • Connection count
  • Link count
Qpid Dispatch Router - Delayed Deliveries

Shows metrics for:

  • Cumulative delayed 10 seconds
  • Cumulative delayed 1 second
  • Rate of new delayed deliveries

Procedure

  1. In the OpenShift web console, switch to Networking → Routes, and click the URL for the grafana Route.

    The Grafana Log In page appears.

  2. Enter your user name and password, and then click Log In.

    The default Grafana user name and password are both admin. After logging in for the first time, you can change the password.

  3. On the top header, click the dashboard drop-down menu, and then select the Qpid Dispatch Router or Qpid Dispatch Router - Delayed Deliveries dashboard.

    Figure 7.1. Delayed Deliveries dashboard

    Grafana dashboard showing delayed deliveries