Procedure

1. Edit the AMQ Streams installation files to use the namespace the Cluster Operator is going to be installed into.

   For example, in this procedure the Cluster Operator is installed into the namespace my-cluster-operator-namespace.

   On Linux, use:

   sed -i 's/namespace: .*/namespace: my-cluster-operator-namespace/' install/cluster-operator/*RoleBinding*.yaml

   On MacOS, use:

   sed -i '' 's/namespace: .*/namespace: my-cluster-operator-namespace/' install/cluster-operator/*RoleBinding*.yaml

2. Deploy the Cluster Operator:

   oc apply -f install/cluster-operator -n my-cluster-operator-namespace

3. Verify that the Cluster Operator was successfully deployed:

   oc get deployments

Procedure

1. Edit the AMQ Streams installation files to use the namespace the Cluster Operator is going to be installed into.

   For example, in this procedure the Cluster Operator is installed into the namespace my-cluster-operator-namespace.

   On Linux, use:

   sed -i 's/namespace: .*/namespace: my-cluster-operator-namespace/' install/cluster-operator/*RoleBinding*.yaml

   On MacOS, use:

   sed -i '' 's/namespace: .*/namespace: my-cluster-operator-namespace/' install/cluster-operator/*RoleBinding*.yaml

2. Edit the install/cluster-operator/050-Deployment-strimzi-cluster-operator.yaml file to add a list of all the namespaces the Cluster Operator will watch to the STRIMZI_NAMESPACE environment variable.

   For example, in this procedure the Cluster Operator will watch the namespaces watched-namespace-1, watched-namespace-2, watched-namespace-3.

   apiVersion: apps/v1
   kind: Deployment
   spec:
     # ...
     template:
       spec:
         serviceAccountName: strimzi-cluster-operator
         containers:
         - name: strimzi-cluster-operator
           image: registry.redhat.io/amq7/amq-streams-rhel7-operator:1.5.0
           imagePullPolicy: IfNotPresent
           env:
           - name: STRIMZI_NAMESPACE
             value: watched-namespace-1,watched-namespace-2,watched-namespace-3

3. For each namespace listed, install the RoleBindings.

   In this example, we replace watched-namespace in these commands with the namespaces listed in the previous step, repeating them for watched-namespace-1, watched-namespace-2, watched-namespace-3:

   oc apply -f install/cluster-operator/020-RoleBinding-strimzi-cluster-operator.yaml -n watched-namespace
   oc apply -f install/cluster-operator/031-RoleBinding-strimzi-cluster-operator-entity-operator-delegation.yaml -n watched-namespace
   oc apply -f install/cluster-operator/032-RoleBinding-strimzi-cluster-operator-topic-operator-delegation.yaml -n watched-namespace

4. Deploy the Cluster Operator:

   oc apply -f install/cluster-operator -n my-cluster-operator-namespace

5. Verify that the Cluster Operator was successfully deployed:

   oc get deployments

Procedure

1. Edit the AMQ Streams installation files to use the namespace the Cluster Operator is going to be installed into.

   For example, in this procedure the Cluster Operator is installed into the namespace my-cluster-operator-namespace.

   On Linux, use:

   sed -i 's/namespace: .*/namespace: my-cluster-operator-namespace/' install/cluster-operator/*RoleBinding*.yaml

   On MacOS, use:

   sed -i '' 's/namespace: .*/namespace: my-cluster-operator-namespace/' install/cluster-operator/*RoleBinding*.yaml

2. Edit the install/cluster-operator/050-Deployment-strimzi-cluster-operator.yaml file to set the value of the STRIMZI_NAMESPACE environment variable to *.

   apiVersion: apps/v1
   kind: Deployment
   spec:
     # ...
     template:
       spec:
         # ...
         serviceAccountName: strimzi-cluster-operator
         containers:
         - name: strimzi-cluster-operator
           image: registry.redhat.io/amq7/amq-streams-rhel7-operator:1.5.0
           imagePullPolicy: IfNotPresent
           env:
           - name: STRIMZI_NAMESPACE
             value: "*"
           # ...

3. Create ClusterRoleBindings that grant cluster-wide access for all namespaces to the Cluster Operator.

   oc create clusterrolebinding strimzi-cluster-operator-namespaced --clusterrole=strimzi-cluster-operator-namespaced --serviceaccount my-cluster-operator-namespace:strimzi-cluster-operator
   oc create clusterrolebinding strimzi-cluster-operator-entity-operator-delegation --clusterrole=strimzi-entity-operator --serviceaccount my-cluster-operator-namespace:strimzi-cluster-operator
   oc create clusterrolebinding strimzi-cluster-operator-topic-operator-delegation --clusterrole=strimzi-topic-operator --serviceaccount my-cluster-operator-namespace:strimzi-cluster-operator

   Replace my-cluster-operator-namespace with the namespace you want to install the Cluster Operator into.

4. Deploy the Cluster Operator to your OpenShift cluster.

   oc apply -f install/cluster-operator -n my-cluster-operator-namespace

5. Verify that the Cluster Operator was successfully deployed:

   oc get deployments

Procedure

1. In the OpenShift 4 web console, click Operators > OperatorHub.

2. Search or browse for the AMQ Streams Operator, in the Streaming & Messaging category.

   
3. Click the AMQ Streams tile and then, in the sidebar on the right, click Install.

4. On the Create Operator Subscription screen, choose from the following installation and update options:

   • Installation Mode: Choose to install the AMQ Streams Operator to all (projects) namespaces in the cluster (the default option) or a specific (project) namespace. It is good practice to use namespaces to separate functions. We recommend that you dedicate a specific namespace to the Kafka cluster and other AMQ Streams components.
   • Approval Strategy: By default, the AMQ Streams Operator is automatically upgraded to the latest AMQ Streams version by the Operator Lifecycle Manager (OLM). Optionally, select Manual if you want to manually approve future upgrades. For more information, see the Operators guide in the OpenShift documentation.

5. Click Subscribe; the AMQ Streams Operator is installed to your OpenShift cluster.

   The AMQ Streams Operator deploys the Cluster Operator, CRDs, and role-based access control (RBAC) resources to the selected namespace, or to all namespaces.

6. On the Installed Operators screen, check the progress of the installation. The AMQ Streams Operator is ready to use when its status changes to InstallSucceeded.

   

Procedure

1. Create and deploy an ephemeral or persistent cluster.

   For development or testing, you might prefer to use an ephemeral cluster. You can use a persistent cluster in any situation.

   • To create and deploy an ephemeral cluster:

     oc apply -f examples/kafka/kafka-ephemeral.yaml

   • To create and deploy a persistent cluster:

     oc apply -f examples/kafka/kafka-persistent.yaml

2. Verify that the Kafka cluster was successfully deployed:

   oc get deployments

Procedure

1. Edit the entityOperator properties of the Kafka resource to include topicOperator:

   apiVersion: kafka.strimzi.io/v1beta1
   kind: Kafka
   metadata:
     name: my-cluster
   spec:
     #...
     entityOperator:
       topicOperator: {}
       userOperator: {}

2. Configure the Topic Operator spec using the properties described in EntityTopicOperatorSpec schema reference.

   Use an empty object ({}) if you want all properties to use their default values.

3. Create or update the resource:

   Use oc apply:

   oc apply -f <your-file>

Procedure

1. Edit the entityOperator properties of the Kafka resource to include userOperator:

   apiVersion: kafka.strimzi.io/v1beta1
   kind: Kafka
   metadata:
     name: my-cluster
   spec:
     #...
     entityOperator:
       topicOperator: {}
       userOperator: {}

2. Configure the User Operator spec using the properties described in EntityUserOperatorSpec schema reference.

   Use an empty object ({}) if you want all properties to use their default values.

3. Create or update the resource:

   oc apply -f <your-file>

Procedure

1. Edit the Deployment.spec.template.spec.containers[0].env properties in the install/topic-operator/05-Deployment-strimzi-topic-operator.yaml file by setting:

   1. STRIMZI_KAFKA_BOOTSTRAP_SERVERS to list the bootstrap brokers in your Kafka cluster, given as a comma-separated list of hostname:‍port pairs.
   2. STRIMZI_ZOOKEEPER_CONNECT to list the ZooKeeper nodes, given as a comma-separated list of hostname:‍port pairs. This should be the same ZooKeeper cluster that your Kafka cluster is using.
   3. STRIMZI_NAMESPACE to the OpenShift namespace in which you want the operator to watch for KafkaTopic resources.
   4. STRIMZI_RESOURCE_LABELS to the label selector used to identify the KafkaTopic resources managed by the operator.
   5. STRIMZI_FULL_RECONCILIATION_INTERVAL_MS to specify the interval between periodic reconciliations, in milliseconds.
   6. STRIMZI_TOPIC_METADATA_MAX_ATTEMPTS to specify the number of attempts at getting topic metadata from Kafka. The time between each attempt is defined as an exponential back-off. Consider increasing this value when topic creation could take more time due to the number of partitions or replicas. Default 6.
   7. STRIMZI_ZOOKEEPER_SESSION_TIMEOUT_MS to the ZooKeeper session timeout, in milliseconds. For example, 10000. Default 20000 (20 seconds).
   8. STRIMZI_TOPICS_PATH to the Zookeeper node path where the Topic Operator stores its metadata. Default /strimzi/topics.
   9. STRIMZI_TLS_ENABLED to enable TLS support for encrypting the communication with Kafka brokers. Default true.
   10. STRIMZI_TRUSTSTORE_LOCATION to the path to the truststore containing certificates for enabling TLS based communication. Mandatory only if TLS is enabled through STRIMZI_TLS_ENABLED.
   11. STRIMZI_TRUSTSTORE_PASSWORD to the password for accessing the truststore defined by STRIMZI_TRUSTSTORE_LOCATION. Mandatory only if TLS is enabled through STRIMZI_TLS_ENABLED.
   12. STRIMZI_KEYSTORE_LOCATION to the path to the keystore containing private keys for enabling TLS based communication. Mandatory only if TLS is enabled through STRIMZI_TLS_ENABLED.
   13. STRIMZI_KEYSTORE_PASSWORD to the password for accessing the keystore defined by STRIMZI_KEYSTORE_LOCATION. Mandatory only if TLS is enabled through STRIMZI_TLS_ENABLED.
   14. STRIMZI_LOG_LEVEL to the level for printing logging messages. The value can be set to: ERROR, WARNING, INFO, DEBUG, and TRACE. Default INFO.
   15. STRIMZI_JAVA_OPTS (optional) to the Java options used for the JVM running the Topic Operator. An example is -Xmx=512M -Xms=256M.
   16. STRIMZI_JAVA_SYSTEM_PROPERTIES (optional) to list the -D options which are set to the Topic Operator. An example is -Djavax.net.debug=verbose -DpropertyName=value.

2. Deploy the Topic Operator:

   oc apply -f install/topic-operator

3. Verify that the Topic Operator has been deployed successfully:

   oc describe deployment strimzi-topic-operator

   The Topic Operator is deployed when the Replicas: entry shows 1 available.

   Note

   You may experience a delay with the deployment if you have a slow connection to the OpenShift cluster and the images have not been downloaded before.

Procedure

1. Edit the following Deployment.spec.template.spec.containers[0].env properties in the install/user-operator/05-Deployment-strimzi-user-operator.yaml file by setting:

   1. STRIMZI_KAFKA_BOOTSTRAP_SERVERS to list the Kafka brokers, given as a comma-separated list of hostname:‍port pairs.
   2. STRIMZI_ZOOKEEPER_CONNECT to list the ZooKeeper nodes, given as a comma-separated list of hostname:‍port pairs. This must be the same ZooKeeper cluster that your Kafka cluster is using. Connecting to ZooKeeper nodes with TLS encryption is not supported.
   3. STRIMZI_NAMESPACE to the OpenShift namespace in which you want the operator to watch for KafkaUser resources.
   4. STRIMZI_LABELS to the label selector used to identify the KafkaUser resources managed by the operator.
   5. STRIMZI_FULL_RECONCILIATION_INTERVAL_MS to specify the interval between periodic reconciliations, in milliseconds.
   6. STRIMZI_ZOOKEEPER_SESSION_TIMEOUT_MS to the ZooKeeper session timeout, in milliseconds. For example, 10000. Default 20000 (20 seconds).
   7. STRIMZI_CA_CERT_NAME to point to an OpenShift Secret that contains the public key of the Certificate Authority for signing new user certificates for TLS client authentication. The Secret must contain the public key of the Certificate Authority under the key ca.crt.
   8. STRIMZI_CA_KEY_NAME to point to an OpenShift Secret that contains the private key of the Certificate Authority for signing new user certificates for TLS client authentication. The Secret must contain the private key of the Certificate Authority under the key ca.key.
   9. STRIMZI_CLUSTER_CA_CERT_SECRET_NAME to point to an OpenShift Secret containing the public key of the Certificate Authority used for signing Kafka brokers certificates for enabling TLS-based communication. The Secret must contain the public key of the Certificate Authority under the key ca.crt. This environment variable is optional and should be set only if the communication with the Kafka cluster is TLS based.
   10. STRIMZI_EO_KEY_SECRET_NAME to point to an OpenShift Secret containing the private key and related certificate for TLS client authentication against the Kafka cluster. The Secret must contain the keystore with the private key and certificate under the key entity-operator.p12, and the related password under the key entity-operator.password. This environment variable is optional and should be set only if TLS client authentication is needed when the communication with the Kafka cluster is TLS based.
   11. STRIMZI_CA_VALIDITY the validity period for the Certificate Authority. Default is 365 days.
   12. STRIMZI_CA_RENEWAL the renewal period for the Certificate Authority.
   13. STRIMZI_LOG_LEVEL to the level for printing logging messages. The value can be set to: ERROR, WARNING, INFO, DEBUG, and TRACE. Default INFO.
   14. STRIMZI_GC_LOG_ENABLED to enable garbage collection (GC) logging. Default true. Default is 30 days to initiate certificate renewal before the old certificates expire.
   15. STRIMZI_JAVA_OPTS (optional) to the Java options used for the JVM running User Operator. An example is -Xmx=512M -Xms=256M.
   16. STRIMZI_JAVA_SYSTEM_PROPERTIES (optional) to list the -D options which are set to the User Operator. An example is -Djavax.net.debug=verbose -DpropertyName=value.

2. Deploy the User Operator:

   oc apply -f install/user-operator

3. Verify that the User Operator has been deployed successfully:

   oc describe deployment strimzi-user-operator

   The User Operator is deployed when the Replicas: entry shows 1 available.

   Note

   You may experience a delay with the deployment if you have a slow connection to the OpenShift cluster and the images have not been downloaded before.

Procedure

1. Create a new Dockerfile using registry.redhat.io/amq7/amq-streams-kafka-25-rhel7:1.5.0 as the base image:

   FROM registry.redhat.io/amq7/amq-streams-kafka-25-rhel7:1.5.0
   USER root:root
   COPY ./my-plugins/ /opt/kafka/plugins/
   USER 1001

   Example plug-in file

   $ tree ./my-plugins/
   ./my-plugins/
   ├── debezium-connector-mongodb
   │   ├── bson-3.4.2.jar
   │   ├── CHANGELOG.md
   │   ├── CONTRIBUTE.md
   │   ├── COPYRIGHT.txt
   │   ├── debezium-connector-mongodb-0.7.1.jar
   │   ├── debezium-core-0.7.1.jar
   │   ├── LICENSE.txt
   │   ├── mongodb-driver-3.4.2.jar
   │   ├── mongodb-driver-core-3.4.2.jar
   │   └── README.md
   ├── debezium-connector-mysql
   │   ├── CHANGELOG.md
   │   ├── CONTRIBUTE.md
   │   ├── COPYRIGHT.txt
   │   ├── debezium-connector-mysql-0.7.1.jar
   │   ├── debezium-core-0.7.1.jar
   │   ├── LICENSE.txt
   │   ├── mysql-binlog-connector-java-0.13.0.jar
   │   ├── mysql-connector-java-5.1.40.jar
   │   ├── README.md
   │   └── wkb-1.0.2.jar
   └── debezium-connector-postgres
       ├── CHANGELOG.md
       ├── CONTRIBUTE.md
       ├── COPYRIGHT.txt
       ├── debezium-connector-postgres-0.7.1.jar
       ├── debezium-core-0.7.1.jar
       ├── LICENSE.txt
       ├── postgresql-42.0.0.jar
       ├── protobuf-java-2.6.1.jar
       └── README.md

2. Build the container image.
3. Push your custom image to your container registry.

4. Point to the new container image.

   You can either:

   • Edit the KafkaConnect.spec.image property of the KafkaConnect custom resource.

     If set, this property overrides the STRIMZI_KAFKA_CONNECT_IMAGES variable in the Cluster Operator.

     apiVersion: kafka.strimzi.io/v1beta1
     kind: KafkaConnect
     metadata:
       name: my-connect-cluster
     spec: 1
       #...
       image: my-new-container-image 2
       config: 3
         #...

     1

     The specification for the Kafka Connect cluster.

     2

     The docker image for the pods.

     3

     Configuration of the Kafka Connect workers (not connectors).

     or

   • In the install/cluster-operator/050-Deployment-strimzi-cluster-operator.yaml file, edit the STRIMZI_KAFKA_CONNECT_IMAGES variable to point to the new container image, and then reinstall the Cluster Operator.

Procedure

1. On the command line, use the oc apply command to create and deploy a Kafka Connect S2I cluster:

   oc apply -f examples/connect/kafka-connect-s2i.yaml

2. Create a directory with Kafka Connect plug-ins:

   $ tree ./my-plugins/
   ./my-plugins/
   ├── debezium-connector-mongodb
   │   ├── bson-3.4.2.jar
   │   ├── CHANGELOG.md
   │   ├── CONTRIBUTE.md
   │   ├── COPYRIGHT.txt
   │   ├── debezium-connector-mongodb-0.7.1.jar
   │   ├── debezium-core-0.7.1.jar
   │   ├── LICENSE.txt
   │   ├── mongodb-driver-3.4.2.jar
   │   ├── mongodb-driver-core-3.4.2.jar
   │   └── README.md
   ├── debezium-connector-mysql
   │   ├── CHANGELOG.md
   │   ├── CONTRIBUTE.md
   │   ├── COPYRIGHT.txt
   │   ├── debezium-connector-mysql-0.7.1.jar
   │   ├── debezium-core-0.7.1.jar
   │   ├── LICENSE.txt
   │   ├── mysql-binlog-connector-java-0.13.0.jar
   │   ├── mysql-connector-java-5.1.40.jar
   │   ├── README.md
   │   └── wkb-1.0.2.jar
   └── debezium-connector-postgres
       ├── CHANGELOG.md
       ├── CONTRIBUTE.md
       ├── COPYRIGHT.txt
       ├── debezium-connector-postgres-0.7.1.jar
       ├── debezium-core-0.7.1.jar
       ├── LICENSE.txt
       ├── postgresql-42.0.0.jar
       ├── protobuf-java-2.6.1.jar
       └── README.md

3. Use the oc start-build command to start a new build of the image using the prepared directory:

   oc start-build my-connect-cluster-connect --from-dir ./my-plugins/

   Note

   The name of the build is the same as the name of the deployed Kafka Connect cluster.

4. When the build has finished, the new image is used automatically by the Kafka Connect deployment.

1. Update the Kafka resource in an editor.

   oc edit kafka my-cluster

2. Copy the example configuration in kafka-metrics.yaml to your own Kafka resource definition.
3. Save the file, exit the editor and wait for the updated resource to be reconciled.

Procedure

1. Download the bundle.yaml resources file from the repository.

   On Linux, use:

   curl -s https://raw.githubusercontent.com/coreos/prometheus-operator/master/bundle.yaml | sed -e 's/namespace: .*/namespace: my-namespace/' > prometheus-operator-deployment.yaml

   On MacOS, use:

   curl -s https://raw.githubusercontent.com/coreos/prometheus-operator/master/bundle.yaml | sed -e '' 's/namespace: .*/namespace: my-namespace/' > prometheus-operator-deployment.yaml

   • Replace the example namespace with your own.

     Note

     If using OpenShift, specify a release of the OpenShift fork of the Prometheus Operator repository.

2. (Optional) If it is not required, you can manually remove the spec.template.spec.securityContext property from the prometheus-operator-deployment.yaml file.

3. Deploy the Prometheus Operator:

   oc apply -f prometheus-operator-deployment.yaml

Procedure

1. Modify the Prometheus installation file (prometheus.yaml) according to the namespace Prometheus is going to be installed into:

   On Linux, use:

   sed -i 's/namespace: .*/namespace: my-namespace/' prometheus.yaml

   On MacOS, use:

   sed -i '' 's/namespace: .*/namespace: my-namespace/' prometheus.yaml

2. Edit the ServiceMonitor resource in strimzi-service-monitor.yaml to define Prometheus jobs that will scrape the metrics data from services. ServiceMonitor is used to scrape metrics through services and is used for Apache Kafka, ZooKeeper.
3. Edit the PodMonitor resource in strimzi-pod-monitor.yaml to define Prometheus jobs that will scrape the metrics data from pods. PodMonitor is used to scrape data directly from pods and is used for Operators.

4. To use another role:

   1. Create a Secret resource:

      oc create secret generic additional-scrape-configs --from-file=prometheus-additional.yaml

   2. Edit the additionalScrapeConfigs property in the prometheus.yaml file to include the name of the Secret and the YAML file (prometheus-additional.yaml) that contains the additional configuration.

5. Deploy the Prometheus resources:

   oc apply -f strimzi-service-monitor.yaml
   oc apply -f strimzi-pod-monitor.yaml
   oc apply -f prometheus-rules.yaml
   oc apply -f prometheus.yaml

Procedure

1. Create a Secret resource from the Alertmanager configuration file (alert-manager-config.yaml):

   oc create secret generic alertmanager-alertmanager --from-file=alertmanager.yaml=alert-manager-config.yaml

2. Update the alert-manager-config.yaml file to replace the:

   • slack_api_url property with the actual value of the Slack API URL related to the application for the Slack workspace
   • channel property with the actual Slack channel on which to send notifications

3. Deploy Alertmanager:

   oc apply -f alert-manager.yaml

Procedure

1. Deploy Grafana:

   oc apply -f grafana.yaml

2. Enable the Grafana dashboards.

1. Running oc port-forward svc/grafana 3000:3000
2. Pointing a browser to http://localhost:3000

Procedure

1. Access the Grafana user interface using admin/admin credentials.

   On the initial view choose to reset the password.

   

2. Click the Add data source button.

   

3. Add Prometheus as a data source.

   • Specify a name
   • Add Prometheus as the type
   • Specify the connection string to the Prometheus server (http://prometheus-operated:9090) in the URL field

4. Click Add to test the connection to the data source.

   

5. Click Dashboards, then Import to open the Import Dashboard window and import the example dashboards (or paste the JSON).

   

Procedure

1. Take note of any configuration changes made to the existing Cluster Operator resources (in the /install/cluster-operator directory). Any changes will be overwritten by the new version of the Cluster Operator.

2. Update the Cluster Operator.

   1. Modify the installation files for the new version according to the namespace the Cluster Operator is running in.

      On Linux, use:

      sed -i 's/namespace: .*/namespace: my-cluster-operator-namespace/' install/cluster-operator/*RoleBinding*.yaml

      On MacOS, use:

      sed -i '' 's/namespace: .*/namespace: my-cluster-operator-namespace/' install/cluster-operator/*RoleBinding*.yaml

   2. If you modified one or more environment variables in your existing Cluster Operator Deployment, edit the install/cluster-operator/050-Deployment-cluster-operator.yaml file to use those environment variables.

3. When you have an updated configuration, deploy it along with the rest of the installation resources:

   oc apply -f install/cluster-operator

   Wait for the rolling updates to complete.

4. Get the image for the Kafka pod to ensure the upgrade was successful:

   oc get po my-cluster-kafka-0 -o jsonpath='{.spec.containers[0].image}'

   The image tag shows the new AMQ Streams version followed by the Kafka version. For example, <New AMQ Streams version>-kafka-<Current Kafka version>.

5. Update existing resources to handle deprecated custom resource properties.

   • AMQ Streams resource upgrades

Procedure

1. Update the Kafka cluster configuration in an editor, as required:

   oc edit kafka my-cluster

   1. If the log.message.format.version of the current Kafka version is the same as that of the new Kafka version, proceed to the next step.

      Otherwise, ensure that Kafka.spec.kafka.config has the log.message.format.version configured to the default for the current version.

      For example, if upgrading from Kafka 2.4.1:

      kind: Kafka
      spec:
        # ...
        kafka:
          version: 2.4.1
          config:
            log.message.format.version: "2.4"
            # ...

      If the log.message.format.version is unset, set it to the current version.

      Note

      The value of log.message.format.version must be a string to prevent it from being interpreted as a floating point number.

   2. Change the Kafka.spec.kafka.version to specify the new version (leaving the log.message.format.version as the current version).

      For example, if upgrading from Kafka 2.4.1 to 2.5.0:

      apiVersion: v1alpha1
      kind: Kafka
      spec:
        # ...
        kafka:
          version: 2.5.0 1
          config:
            log.message.format.version: "2.4" 2
            # ...

      1

      This is changed to the new version

      2

      This remains at the current version

   3. If the image for the Kafka version is different from the image defined in STRIMZI_KAFKA_IMAGES for the Cluster Operator, update Kafka.spec.kafka.image.

      See Section 7.1.3.1, “Kafka version and image mappings”

2. Save and exit the editor, then wait for rolling updates to complete.

   Note

   Additional rolling updates occur if the new version of Kafka has a new ZooKeeper version.

   Check the update in the logs or by watching the pod state transitions:

   oc logs -f <cluster-operator-pod-name> | grep -E "Kafka version upgrade from [0-9.]+ to [0-9.]+, phase ([0-9]+) of \1 completed"

   oc get po -w

   If the current and new versions of Kafka have different interbroker protocol versions, check the Cluster Operator logs for an INFO level message:

   Reconciliation #<num>(watch) Kafka(<namespace>/<name>): Kafka version upgrade from <from-version> to <to-version>, phase 2 of 2 completed

   Alternatively, if the current and new versions of Kafka have the same interbroker protocol version, check for:

   Reconciliation #<num>(watch) Kafka(<namespace>/<name>): Kafka version upgrade from <from-version> to <to-version>, phase 1 of 1 completed

   The rolling updates:

   • Ensure each pod is using the broker binaries for the new version of Kafka

   • Configure the brokers to send messages using the interbroker protocol of the new version of Kafka

     Note

     Clients are still using the old version, so brokers will convert messages to the old version before sending them to the clients. To minimize this additional load, update the clients as quickly as possible.

3. Depending on your chosen strategy for upgrading clients, upgrade all client applications to use the new version of the client binaries.

   See Section 7.1.3.2, “Strategies for upgrading clients”

   Warning

   You cannot downgrade after completing this step. If you need to revert the update at this point, follow the procedure Section 7.1.4.2, “Downgrading Kafka brokers and client applications”.

   If required, set the version property for Kafka Connect and MirrorMaker as the new version of Kafka:

   1. For Kafka Connect, update KafkaConnect.spec.version
   2. For MirrorMaker, update KafkaMirrorMaker.spec.version

4. If the log.message.format.version identified in step 1 is the same as the new version proceed to the next step.

   Otherwise change the log.message.format.version in Kafka.spec.kafka.config to the default version for the new version of Kafka now being used.

   For example, if upgrading to 2.5.0:

   apiVersion: v1alpha1
   kind: Kafka
   spec:
     # ...
     kafka:
       version: 2.5.0
       config:
         log.message.format.version: "2.5"
         # ...

5. Wait for the Cluster Operator to update the cluster.

   The Kafka cluster and clients are now using the new Kafka version.

1. Replace the Kafka clients .jar file with the new version.
2. In the consumer configuration, append cooperative-sticky to the partition.assignment.strategy. For example, if the range strategy is set, change the configuration to range, cooperative-sticky.
3. Restart each consumer in the group in turn, waiting for the consumer to rejoin the group after each restart.
4. Reconfigure each consumer in the group by removing the earlier partition.assignment.strategy from the consumer configuration, leaving only the cooperative-sticky strategy.
5. Restart each consumer in the group in turn, waiting for the consumer to rejoin the group after each restart.

1. Replace the Kafka Streams .jar file with the new version.
2. In the Kafka Streams configuration, set the upgrade.from configuration parameter to the Kafka version you are upgrading from (for example, 2.3).
3. Restart each of the stream processors (nodes) in turn.
4. Remove the upgrade.from configuration parameter from the Kafka Streams configuration.
5. Restart each consumer in the group in turn.

Procedure

1. Update the Kafka cluster configuration in an editor, as required:

   Use oc edit:

   oc edit kafka my-cluster

   1. Change the Kafka.spec.kafka.version to specify the previous version.

      For example, if downgrading from Kafka 2.5.0 to 2.4.1:

      apiVersion: v1alpha1
      kind: Kafka
      spec:
        # ...
        kafka:
          version: 2.4.1 1
          config:
            log.message.format.version: "2.4" 2
            # ...

      1

      This is changed to the previous version

      2

      This is unchanged

      Note

      You must format the value of log.message.format.version as a string to prevent it from being interpreted as a floating point number.

   2. If the image for the Kafka version is different from the image defined in STRIMZI_KAFKA_IMAGES for the Cluster Operator, update Kafka.spec.kafka.image.

      See Section 7.1.3.1, “Kafka version and image mappings”

2. Save and exit the editor, then wait for rolling updates to complete.

   Check the update in the logs or by watching the pod state transitions:

   oc logs -f <cluster-operator-pod-name> | grep -E "Kafka version downgrade from [0-9.]+ to [0-9.]+, phase ([0-9]+) of \1 completed"

   oc get po -w

   If the previous and current versions of Kafka have different interbroker protocol versions, check the Cluster Operator logs for an INFO level message:

   Reconciliation #<num>(watch) Kafka(<namespace>/<name>): Kafka version downgrade from <from-version> to <to-version>, phase 2 of 2 completed

   Alternatively, if the previous and current versions of Kafka have the same interbroker protocol version, check for:

   Reconciliation #<num>(watch) Kafka(<namespace>/<name>): Kafka version downgrade from <from-version> to <to-version>, phase 1 of 1 completed

3. Downgrade all client applications (consumers) to use the previous version of the client binaries.

   The Kafka cluster and clients are now using the previous Kafka version.