Chapter 4. Technology Previews
Technology Preview features are not supported with Red Hat production service-level agreements (SLAs) and might not be functionally complete; therefore, Red Hat does not recommend implementing any Technology Preview features in production environments. This Technology Preview feature provides early access to upcoming product innovations, enabling you to test functionality and provide feedback during the development process. For more information about support scope, see Technology Preview Features Support Scope.
4.1. Kafka Static Quota plugin configuration
Use the Kafka Static Quota plugin to set throughput and storage limits on brokers in your Kafka cluster. You enable the plugin and set limits by configuring the
Kafka resource. You can set a byte-rate threshold and storage quotas to put limits on the clients interacting with your brokers.
Example Kafka Static Quota plugin configuration
apiVersion: kafka.strimzi.io/v1beta2 kind: Kafka metadata: name: my-cluster spec: kafka: # ... config: client.quota.callback.class: io.strimzi.kafka.quotas.StaticQuotaCallback client.quota.callback.static.produce: 1000000 client.quota.callback.static.fetch: 1000000 client.quota.callback.static.storage.soft: 400000000000 client.quota.callback.static.storage.hard: 500000000000 client.quota.callback.static.storage.check-interval: 5
4.2. Cruise Control for cluster rebalancing
Cruise Control remains in Technology Preview, with some new enhancements.
You can deploy Cruise Control and use it to rebalance your Kafka cluster using optimization goals — defined constraints on CPU, disk, network load, and more. In a balanced Kafka cluster, the workload is more evenly distributed across the broker pods.
Cruise Control is configured and deployed as part of a
Kafka resource. You can use the default optimization goals or modify them to suit your requirements. Example YAML configuration files for Cruise Control are provided in
When Cruise Control is deployed, you can create
KafkaRebalance custom resources to:
- Generate optimization proposals from multiple optimization goals
- Rebalance a Kafka cluster based on an optimization proposal
Other Cruise Control features are not currently supported, including anomaly detection, notifications, write-your-own goals, and changing the topic replication factor.
4.2.1. Enhancements to the Technology Preview
Refresh optimization proposals
You can now reuse existing
KafkaRebalance resources with a status of
Ready, which indicates that cluster rebalancing completed successfully. You can reuse the optimization goals defined in the
KafkaRebalance resource, or change the goals.
To refresh an optimization proposal:
Check the status of the
oc describe kafkarebalance REBALANCE-NAME
oc annotate kafkarebalance REBALANCE-NAME strimzi.io/rebalance=refresh
Cruise Control refreshes the optimization proposal to reflect the latest state of your Kafka cluster.
View the broker load in optimization proposals
Optimization proposals now consist of the broker load in addition to the summary status. The broker load is returned inside a ConfigMap and shows metrics for the load on each Kafka broker, including the CPU utilization, disk usage, network output rate, and more. The metrics are presented in three categories:
- Current value before the optimization proposal is applied
- Expected value after the optimization proposal is applied
- Difference between the after value and the before value
The broker load ConfigMap has the same name as the
KafkaRebalance resource. The metrics are encoded as a JSON string. To view them in a human readable format, use
jq or a similiar JSON parser. For example:
oc get configmap MY-REBALANCE -o json | jq '.["data"]["brokerLoad.json"]|fromjson|.'