Menu Close
Settings Close

Language and Page Formatting Options

Chapter 5. Monitoring AMQ Online

You can monitor AMQ Online by deploying built-in monitoring tools or using your pre-existing monitoring infrastructure by deploying the required service monitors and Prometheus rules.

5.1. (Optional) Deploying the Application Monitoring Operator

To monitor AMQ Online, an operator that acts on the monitoring Custom Resource Definitions must be deployed. You may skip this step if you have such an operator installed on your OpenShift cluster.

Procedure

  1. Log in as a user with cluster-admin privileges:

    oc login -u system:admin
  2. (Optional) If you want to deploy to a namespace other than amq-online-monitoring you must run the following command and substitute amq-online-monitoring in subsequent steps:

    sed -i 's/amq-online-monitoring/my-namespace/' install/bundles/amq-online/*.yaml
  3. Create the amq-online-monitoring namespace:

    oc new-project amq-online-monitoring
  4. Deploy the monitoring-operator component:

    oc apply -f install/components/monitoring-operator

5.2. (Optional) Deploying the kube-state-metrics agent

You can monitor AMQ Online pods using the kube-state-metrics agent.

Procedure

  1. Log in as a user with cluster-admin privileges:

    oc login -u system:admin
  2. Select the amq-online-infra project:

    oc project amq-online-infra
  3. Deploy the kube-state-metrics component:

    oc apply -f install/components/kube-state-metrics

5.3. Deploying monitoring using a YAML bundle

The simplest way to deploy monitoring is to use a predefined YAML bundle.

Prerequisites

Procedure

  1. Label the amq-online-infra namespace:

    oc label namespace amq-online-infra monitoring-key=middleware
  2. Select the amq-online-infra project:

    oc project amq-online-infra
  3. Deploy the monitoring bundle:

    oc apply -f install/bundles/monitoring

5.4. Deploying monitoring using Ansible

Monitoring can also be deployed during the AMQ Online installation using Ansible using the required configuration settings.

5.5. Configuring alert notifications

To configure alert notifications, such as emails, you must change the default configuration of Alertmanager.

Prerequisites

  • Create an Alertmanager configuration file following the Alertmanager documentation. An example configuration file for email notifications is shown:

    apiVersion: v1
    kind: ConfigMap
    metadata:
      labels:
        app: enmasse
      name: alertmanager-config
    data:
      alertmanager.yml: |
        global:
          resolve_timeout: 5m
          smtp_smarthost: localhost
          smtp_from: alerts@localhost
          smtp_auth_username: admin
          smtp_auth_password: password
        route:
          group_by: ['alertname']
          group_wait: 60s
          group_interval: 60s
          repeat_interval: 1h
          receiver: 'sysadmins'
        receivers:
        - name: 'sysadmins'
          email_configs:
          - to: sysadmin@localhost
        inhibit_rules:
          - source_match:
              severity: 'critical'
            target_match:
              severity: 'warning'
            equal: ['alertname']
  • Your Alertmanager configuration file must be named alertmanager.yaml so it can be read by the Prometheus Operator.

Procedure

  1. Delete the secret containing the default configuration:

    oc delete secret alertmanager-application-monitoring
  2. Create a secret containing your new configuration:

    oc create secret generic alertmanager-application-monitoring --from-file=alertmanager.yaml

5.6. Using qdstat

You can use qdstat to monitor the AMQ Online service.

5.6.1. Viewing router connections using qdstat

You can view the router connections using qdstat.

Procedure

  1. On the command line, run the following command to obtain the podname value needed in the following step:

    oc get pods
  2. On the command line, run the following command:

    oc exec -n namespace -it qdrouterd-podname -- qdstat -b 127.0.0.1:7777 -c
    
    Connections
      id   host                 container                             role    dir  security                              authentication                tenant
      =========================================================================================================================================================
      3    172.17.0.9:34998     admin-78794c68c8-9jdd6                normal  in   TLSv1.2(ECDHE-RSA-AES128-GCM-SHA256)  CN=admin,O=io.enmasse(x.509)
      12   172.30.188.174:5671  27803a14-42d2-6148-9491-a6c1e69e875a  normal  out  TLSv1.2(ECDHE-RSA-AES128-GCM-SHA256)  x.509
      567  127.0.0.1:43546      b240c652-82df-48dd-b54e-3b8bbaef16c6  normal  in   no-security                           PLAIN

5.6.2. Viewing router addresses using qdstat

You can view the router addresses using qdstat.

Procedure

  1. On the command line, run the following command to obtain the podname value needed in the following step:

    oc get pods
  2. Run the following command:

    oc exec -n namespace -it qdrouterd-podname -- qdstat -b 127.0.0.1:7777 -a
    
    Router Addresses
      class     addr                   phs  distrib       in-proc  local  remote  cntnr  in     out    thru  to-proc  from-proc
      ===========================================================================================================================
      local     $_management_internal       closest       1        0      0       0      0      0      0     588      588
      link-in   $lwt                        linkBalanced  0        0      0       0      0      0      0     0        0
      link-out  $lwt                        linkBalanced  0        0      0       0      0      0      0     0        0
      mobile    $management            0    closest       1        0      0       0      601    0      0     601      0
      local     $management                 closest       1        0      0       0      2,925  0      0     2,925    0
      local     qdhello                     flood         1        0      0       0      0      0      0     0        5,856
      local     qdrouter                    flood         1        0      0       0      0      0      0     0        0
      topo      qdrouter                    flood         1        0      0       0      0      0      0     0        196
      local     qdrouter.ma                 multicast     1        0      0       0      0      0      0     0        0
      topo      qdrouter.ma                 multicast     1        0      0       0      0      0      0     0        0
      local     temp.VTXOKyyWsq7OEei        balanced      0        1      0       0      0      0      0     0        0
      local     temp.k2RGQNPe6sDMvz4        balanced      0        1      0       0      0      3,511  0     0        3,511
      local     temp.xg+y8I_Tr4Y94LA        balanced      0        1      0       0      0      5      0     0        5