Chapter 9. Monitoring Debezium

You can use the JMX metrics provided by Zookeeper and Kafka to monitor Debezium. To use these metrics, you must enable them when you start the Zookeeper, Kafka, and Kafka Connect services. Enabling JMX involves setting the correct environment variables.

Note

If you are running multiple services on the same machine, be sure to use distinct JMX ports for each service.

9.1. Metrics for monitoring Debezium connectors

In addition to the built-in support for JMX metrics in Kafka, Zookeeper, and Kafka Connect, each connector provides additional metrics that you can use to monitor their activities.

9.2. Enabling JMX in local installations

With Zookeeper, Kafka, and Kafka Connect, you enable JMX by setting the appropriate environment variables when you start each service.

9.2.1. Zookeeper JMX environment variables

Zookeeper has built-in support for JMX. When running Zookeeper using a local installation, the zkServer.sh script recognizes the following environment variables:

JMXPORT
Enables JMX and specifies the port number that will be used for JMX. The value is used to specify the JVM parameter -Dcom.sun.management.jmxremote.port=$JMXPORT.
JMXAUTH
Whether JMX clients must use password authentication when connecting. Must be either true or false. The default is false. The value is used to specify the JVM parameter -Dcom.sun.management.jmxremote.authenticate=$JMXAUTH.
JMXSSL
Whether JMX clients connect using SSL/TLS. Must be either true or false. The default is false. The value is used to specify the JVM parameter -Dcom.sun.management.jmxremote.ssl=$JMXSSL.
JMXLOG4J
Whether the Log4J JMX MBeans should be disabled. Must be either true (default) or false. The default is true. The value is used to specify the JVM parameter -Dzookeeper.jmx.log4j.disable=$JMXLOG4J.

9.2.2. Kafka JMX environment variables

When running Kafka using a local installation, the kafka-server-start.sh script recognizes the following environment variables:

JMX_PORT
Enables JMX and specifies the port number that will be used for JMX. The value is used to specify the JVM parameter -Dcom.sun.management.jmxremote.port=$JMX_PORT.
KAFKA_JMX_OPTS

The JMX options, which are passed directly to the JVM during startup. The default options are:

  • -Dcom.sun.management.jmxremote
  • -Dcom.sun.management.jmxremote.authenticate=false
  • -Dcom.sun.management.jmxremote.ssl=false

9.2.3. Kafka Connect JMX environment variables

When running Kafka using a local installation, the connect-distributed.sh script recognizes the following environment variables:

JMX_PORT
Enables JMX and specifies the port number that will be used for JMX. The value is used to specify the JVM parameter -Dcom.sun.management.jmxremote.port=$JMX_PORT.
KAFKA_JMX_OPTS

The JMX options, which are passed directly to the JVM during startup. The default options are:

  • -Dcom.sun.management.jmxremote
  • -Dcom.sun.management.jmxremote.authenticate=false
  • -Dcom.sun.management.jmxremote.ssl=false

9.3. Monitoring Debezium on OpenShift

If you are using Debezium on OpenShift, you can obtain JMX metrics by opening a JMX port on 9999. For more information, see JMX Options in Using AMQ Streams on OpenShift.

In addition, you can use Prometheus and Grafana to monitor the JMX metrics. For more information, see Setting up metrics and dashboards for AMQ Streams, in Deploying and Upgrading AMQ Streams on OpenShift.