AMQ Streams 2.2.x is the Long Term Support (LTS) offering for AMQ Streams.

The latest patch release is AMQ Streams 2.2.2. The AMQ Streams product images have changed to version 2.2.2. The supported Kafka version remains at 3.2.3.

For information on the LTS terms and dates, see the AMQ Streams LTS Support Policy.

AMQ Streams 2.2 is supported on OpenShift Container Platform 4.8 to 4.11.

For more information about the supported platform versions, see the AMQ Streams Supported Configurations.

AMQ Streams now supports Apache Kafka version 3.2.3.

AMQ Streams uses Kafka 3.2.3. Only Kafka distributions built by Red Hat are supported.

You must upgrade the Cluster Operator to AMQ Streams version 2.2 before you can upgrade brokers and client applications to Kafka 3.2.3. For upgrade instructions, see Upgrading AMQ Streams.

Refer to the Kafka 3.1.0, Kafka 3.2.0, Kafka 3.2.1, and Kafka 3.2.3 Release Notes for additional information.

For more information on supported versions, see the AMQ Streams Component Details.

Kafka 3.2.3 uses ZooKeeper version 3.6.3, which is the same version as Kafka 3.1.0.

The v1beta2 API version for all custom resources was introduced with AMQ Streams 1.7. For AMQ Streams 1.8, v1alpha1 and v1beta1 API versions were removed from all AMQ Streams custom resources apart from KafkaTopic and KafkaUser.

Upgrade of the custom resources to v1beta2 prepares AMQ Streams for a move to Kubernetes CRD v1, which is required for Kubernetes v1.22.

If you are upgrading from an AMQ Streams version prior to version 1.7:

See Deploying and upgrading AMQ Streams.

AMQ Streams 2.2 is enabled to run on IBM Z and LinuxONE s390x architecture.

Support for IBM Z and LinuxONE applies to AMQ Streams running with Kafka on OpenShift Container Platform 4.10 and later.

AMQ Streams 2.2 is enabled to run on IBM Power ppc64le architecture.

Support for IBM Power applies to AMQ Streams running with Kafka on OpenShift Container Platform 4.9 and later.

The UseStrimziPodSets feature gate controls a resource for managing pods called StrimziPodSet. When the feature gate is enabled, this resource is used instead of the StatefulSets. AMQ Streams handles the creation and management of pods instead of OpenShift. Using StrimziPodSets instead of StatefulSets provides more control over the functionality.

The feature gate is at an alpha level of maturity, so should be treated as a technology preview.

The preview provides an opportunity to test the StrimziPodSet resource. The feature will be enabled by default in release 2.3.

To enable the feature gate, specify +UseStrimziPodSets as a value for the STRIMZI_FEATURE_GATES environment variable in the Cluster Operator configuration.

env:
  - name: STRIMZI_FEATURE_GATES
    value: +UseStrimziPodSets

See UseStrimziPodSets feature gate and Feature gate releases.

As a Kafka cluster administrator, you can toggle a subset of features on and off using feature gates in the Cluster Operator deployment configuration.

Apache Kafka is in the process of phasing out the need for ZooKeeper. With the new UseKRaft feature gate enabled, you can try deploying a Kafka cluster in KRaft (Kafka Raft metadata) mode without ZooKeeper.

This feature gate is at an alpha level of maturity, but it should be treated as a development preview.

To enable the UseKRaft feature gate, specify +UseKRaft and +USeStrimziPodSets as values for the STRIMZI_FEATURE_GATES environment variable in the Cluster Operator configuration. The UseKRaft feature gate depends on the UseStrimziPodSets feature gate.

env:
  - name: STRIMZI_FEATURE_GATES
    value: +UseKRaft, +USeStrimziPodSets

Currently, the KRaft mode in AMQ Streams has the following major limitations:

See UseKRaft feature gate and Feature gate releases.

Cruise Control moves from Technology Preview to General Availability (GA). You can deploy Cruise Control and use it to rebalance your Kafka cluster using optimization goals — defined constraints on CPU, disk, network load, and more. In a balanced Kafka cluster, the workload is more evenly distributed across the broker pods.

Cruise Control is configured and deployed as part of a Kafka resource. You can use the default optimization goals or modify them to suit your requirements. Example YAML configuration files for Cruise Control are provided in examples/cruise-control/.

When Cruise Control is deployed, you can create KafkaRebalance custom resources to:

Other Cruise Control features are not currently supported, including anomaly detection, notifications, write-your-own goals, and changing the topic replication factor.

See Cruise Control for cluster rebalancing.

You can now generate optimization proposals for rebalancing operations in one of the following modes:

Previously, the proposal was generated in full mode, where replicas might move across all brokers in a cluster. Now you use the add-brokers and remove-brokers modes to take into account scaling up and scaling down operations.

Use the add-brokers mode after scaling up. You specify new brokers and the rebalancing operation moves replicas from existing brokers to the newly added brokers. This is a faster option than rebalancing the whole cluster.

Use the remove-brokers mode before scaling down. You specify brokers you are going to remove, which means that any replicas on those brokers are moved off in the rebalance operation.

See Rebalancing modes, Generating optimization proposals, and Approving optimization proposals.

Red Hat Debezium is a distributed change data capture platform. It captures row-level changes in databases, creates change event records, and streams the records to Kafka topics. Debezium is built on Apache Kafka. You can deploy and integrate Debezium with AMQ Streams. Following a deployment of AMQ Streams, you deploy Debezium as a connector configuration through Kafka Connect. Debezium passes change event records to AMQ Streams on OpenShift. Applications can read these change event streams and access the change events in the order in which they occurred.

Debezium has multiple uses, including:

Debezium provides connectors (based on Kafka Connect) for the following common databases:

For more information on deploying Debezium with AMQ Streams, refer to the product documentation.

You can use Service Registry as a centralized store of service schemas for data streaming. For Kafka, you can use Service Registry to store Apache Avro or JSON schema.

Service Registry provides a REST API and a Java REST client to register and query the schemas from client applications through server-side endpoints.

Using Service Registry decouples the process of managing schemas from the configuration of client applications. You enable an application to use a schema from the registry by specifying its URL in the client code.

For example, the schemas to serialize and deserialize messages can be stored in the registry, which are then referenced from the applications that use them to ensure that the messages that they send and receive are compatible with those schemas.

Kafka client applications can push or pull their schemas from Service Registry at runtime.

For more information on using Service Registry with AMQ Streams, refer to the Service Registry documentation.

For an overview of the enhancements introduced with Kafka 3.2.0, 3.2.1, and 3.2.3, refer to the Kafka 3.2.0 Release Notes, Kafka 3.2.1 Release Notes, and Kafka 3.2.3 Release Notes.

It is now easier to check the status of optimization proposals. Instead of inspecting the resource configuration YAML, you can check the status on the command line.

When you run a proposal, run the following command and wait for the status of the optimization proposal to change to ProposalReady:

oc get kafkarebalance -o wide -w -n <namespace>

When the status changes to ProposalReady, the optimization proposal is ready to approve.

See Generating optimization proposals and Approving optimization proposals.

Maintenance time windows allow you to schedule certain rolling updates of your Kafka and ZooKeeper clusters to start at a convenient time. Maintenance windows are now supported in the User Operator. If you have deployed the User Operator using the Cluster Operator, and not as a standalone operator, automatic renewal of user certificates is included in the schedule.

If you are deploying the standalone User Operator, you can configure the maintenance time window during which expiring user certificates are renewed. You specify the time window as a Cron expression for the STRIMZI_MAINTENANCE_TIME_WINDOWS environment variable.

See Deploying the standalone User Operator.

It’s now possible to rename the topics related to metrics that are created automatically by Cruise Control. You can use the following Cruise Control configuration properties to make the name change.

apiVersion: kafka.strimzi.io/v1beta2
kind: Kafka
metadata:
  name: my-cluster
spec:
  # ...
  cruiseControl:
      # ...
    config: 1
      # ...
      metric.reporter.topic: cruise-control-metrics-reporter-topic-name
      partition.metric.sample.store.topic: cruise-control-partitions-metrics-name
      broker.metric.sample.store.topic: cruise-control-broker-metrics
      # ...
# ...

If you change the names of the topics in an existing deployment, you need to remove the topics with the old names manually.

See Configuring and deploying Cruise Control with Kafka.

Cruise Control now runs without ZooKeeper, using Kafka APIs instead. The TLS sidecar that was used for secure communication with ZooKeeper has been removed.

The TLS sidecar configuration for Cruise Control in the Kafka resource is no longer required. For this reason, the .spec.cruiseControl.tlsSidecar and .spec.cruiseControl.template.tlsSidecar properties are now deprecated.

See Configuring and deploying Cruise Control with Kafka.

You can now enable rack awareness in your MirrorMaker 2.0 resource configuration. This is a specialized option intended for a deployment within the same location, not across regions. You can use this option if you want connectors to consume from the closest replica rather than the leader replica.

A topologyKey in the rack configuration must match a node label containing the rack ID. In the following example, the standard topology.kubernetes.io/zone label is specified.

apiVersion: kafka.strimzi.io/v1beta2
kind: KafkaMirrorMaker2
metadata:
  name: my-mirror-maker2
spec:
  version: 3.2.3
  # ...
  rack:
    topologyKey: topology.kubernetes.io/zone

To consume from the closest replica, you must also enable the RackAwareReplicaSelector in the Kafka broker configuration.

apiVersion: kafka.strimzi.io/v1beta2
kind: Kafka
metadata:
  name: my-cluster
spec:
  kafka:
    # ...
    rack:
      topologyKey: topology.kubernetes.io/zone
    config:
      # ...
      replica.selector.class: org.apache.kafka.common.replica.RackAwareReplicaSelector
    # ...

See Configuring Kafka MirrorMaker 2.0 and Rack schema reference.

If you don’t specify heap sizes in your configuration, the Cluster Operator imposes default heap sizes automatically. The Cluster Operator sets default maximum and minimum heap values based on a percentage of the memory resource configuration. The available memory allocated by default is now set at the levels shown in the following table.

See jvmOptions

Previews of the UseKRaft and UseStrimziPodSets feature gates are now available.

See Chapter 1, Features.

Use the Kafka Static Quota plugin to set throughput and storage limits on brokers in your Kafka cluster. You enable the plugin and set limits by configuring the Kafka resource. You can set a byte-rate threshold and storage quotas to put limits on the clients interacting with your brokers.

apiVersion: kafka.strimzi.io/v1beta2
kind: Kafka
metadata:
  name: my-cluster
spec:
  kafka:
    # ...
    config:
      client.quota.callback.class: io.strimzi.kafka.quotas.StaticQuotaCallback
      client.quota.callback.static.produce: 1000000
      client.quota.callback.static.fetch: 1000000
      client.quota.callback.static.storage.soft: 400000000000
      client.quota.callback.static.storage.hard: 500000000000
      client.quota.callback.static.storage.check-interval: 5

See Setting limits on brokers using the Kafka Static Quota plugin.

Kafka no longer includes the example file connectors FileStreamSourceConnector and FileStreamSinkConnector in its CLASSPATH and plugin.path by default. AMQ Streams has been updated so that you can still use these example connectors. The examples now have to be added to the plugin path like any connector.

Two example connector configuration files are provided:

See Deploying example KafkaConnector resources and Extending Kafka Connect with connector plugins.

Support for OpenTracing is deprecated.

The Jaeger clients are now retired and the OpenTracing project archived. As such, we cannot guarantee their support for future Kafka versions. We are introducing a new tracing implementation based on the OpenTelemetry project.

Support for Java 8 was deprecated in Kafka 3.0.0 and AMQ Streams 2.0. Java 8 will be unsupported for all AMQ Streams components, including clients, in the future.

AMQ Streams supports Java 11. Use Java 11 when developing new applications. Plan to migrate any applications that currently use Java 8 to Java 11.

Kafka MirrorMaker replicates data between two or more active Kafka clusters, within or across data centers. Kafka MirrorMaker 1 is deprecated for Kafka 3.0.0 and will be removed in Kafka 4.0.0. MirrorMaker 2.0 will be the only version available. MirrorMaker 2.0 is based on the Kafka Connect framework, connectors managing the transfer of data between clusters.

As a consequence, the AMQ Streams KafkaMirrorMaker custom resource which is used to deploy Kafka MirrorMaker 1 has been deprecated. The KafkaMirrorMaker resource will be removed from AMQ Streams when Kafka 4.0.0 is adopted.

If you are using MirrorMaker 1 (referred to as just MirrorMaker in the AMQ Streams documentation), use the KafkaMirrorMaker2 custom resource with the IdentityReplicationPolicy. MirrorMaker 2.0 renames topics replicated to a target cluster. IdentityReplicationPolicy configuration overrides the automatic renaming. Use it to produce the same active/passive unidirectional replication as MirrorMaker 1.

See Kafka MirrorMaker 2.0 cluster configuration.

For this release, Cruise Control TLS sidecar has been removed. As a result, the .spec.cruiseControl.tlsSidecar and .spec.cruiseControl.template.tlsSidecar properties are now deprecated. The properties are ignored and will be removed in the future.

Identity replication policy is used with MirrorMaker 2.0 to override the automatic renaming of remote topics. Instead of prepending the name with the name of the source cluster, the topic retains its original name. This optional setting is useful for active/passive backups and data migration.

The AMQ Streams Identity Replication Policy class (io.strimzi.kafka.connect.mirror.IdentityReplicationPolicy) is now deprecated and will be removed in the future. You can update to use Kafka’s own Identity Replication Policy (class org.apache.kafka.connect.mirror.IdentityReplicationPolicy).

See Kafka MirrorMaker 2.0 cluster configuration.

The type property of ListenerStatus has been deprecated and will be removed in the future. ListenerStatus is used to specify the addresses of internal and external listeners. Instead of using the type, the addresses are now specified by name.

See ListenerStatus schema reference.

The disk and cpuUtilization capacity configuration properties have been deprecated, are ignored, and will be removed in the future. The properties were used in setting capacity limits in optimization proposals to determine if resource-based optimization goals are being broken. Disk and CPU capacity limits are now automatically generated by AMQ Streams.

See Cruise Control configuration.

The AMQ Streams 2.2.2 patch release (Long Term Support) is now available.

For additional details about the issues resolved in AMQ Streams 2.2.2, see AMQ Streams 2.2.x Resolved Issues.

For additional details about the issues resolved in AMQ Streams 2.2.1, see AMQ Streams 2.2.x Resolved Issues.

The AMQ Streams Cluster Operator does not start on Internet Protocol version 6 (IPv6) clusters.

Cruise Control for AMQ Streams has a known issue that relates to the calculation of CPU utilization estimation. CPU utilization is calculated as a percentage of the defined capacity of a broker pod. The issue occurs when running Kafka brokers across nodes with varying CPU cores. For example, node1 might have 2 CPU cores and node2 might have 4 CPU cores. In this situation, Cruise Control can underestimate and overestimate CPU load of brokers The issue can prevent cluster rebalances when the pod is under heavy load.

apiVersion: kafka.strimzi.io/v1beta2
kind: Kafka
metadata:
  name: my-cluster
spec:
  kafka:
    # ...
  zookeeper:
    # ...
  entityOperator:
    topicOperator: {}
    userOperator: {}
  cruiseControl:
    brokerCapacity:
      inboundNetwork: 10000KB/s
      outboundNetwork: 10000KB/s
    config:
      hard.goals: >
        com.linkedin.kafka.cruisecontrol.analyzer.goals.RackAwareGoal,
        com.linkedin.kafka.cruisecontrol.analyzer.goals.MinTopicLeadersPerBrokerGoal,
        com.linkedin.kafka.cruisecontrol.analyzer.goals.ReplicaCapacityGoal,
        com.linkedin.kafka.cruisecontrol.analyzer.goals.DiskCapacityGoal,
        com.linkedin.kafka.cruisecontrol.analyzer.goals.NetworkInboundCapacityGoal,
        com.linkedin.kafka.cruisecontrol.analyzer.goals.NetworkOutboundCapacityGoal
      default.goals: >
        com.linkedin.kafka.cruisecontrol.analyzer.goals.RackAwareGoal,
        com.linkedin.kafka.cruisecontrol.analyzer.goals.MinTopicLeadersPerBrokerGoal,
        com.linkedin.kafka.cruisecontrol.analyzer.goals.ReplicaCapacityGoal,
        com.linkedin.kafka.cruisecontrol.analyzer.goals.DiskCapacityGoal,
        com.linkedin.kafka.cruisecontrol.analyzer.goals.NetworkInboundCapacityGoal,
        com.linkedin.kafka.cruisecontrol.analyzer.goals.NetworkOutboundCapacityGoal,
        com.linkedin.kafka.cruisecontrol.analyzer.goals.ReplicaDistributionGoal,
        com.linkedin.kafka.cruisecontrol.analyzer.goals.PotentialNwOutGoal,
        com.linkedin.kafka.cruisecontrol.analyzer.goals.DiskUsageDistributionGoal,
        com.linkedin.kafka.cruisecontrol.analyzer.goals.NetworkInboundUsageDistributionGoal,
        com.linkedin.kafka.cruisecontrol.analyzer.goals.NetworkOutboundUsageDistributionGoal,
        com.linkedin.kafka.cruisecontrol.analyzer.goals.TopicReplicaDistributionGoal,
        com.linkedin.kafka.cruisecontrol.analyzer.goals.LeaderReplicaDistributionGoal,
        com.linkedin.kafka.cruisecontrol.analyzer.goals.LeaderBytesInDistributionGoal

For more information, see Insufficient CPU capacity.

The User Operator can timeout when creating multiple users at the same time. Reconciliation can take too long.