Jump To Close Expand all Collapse all Table of contents Using AMQ Streams on OpenShift 1. Overview of AMQ Streams Expand section "1. Overview of AMQ Streams" Collapse section "1. Overview of AMQ Streams" 1.1. Kafka capabilities 1.2. Kafka use cases 1.3. How AMQ Streams supports Kafka 1.4. AMQ Streams Operators Expand section "1.4. AMQ Streams Operators" Collapse section "1.4. AMQ Streams Operators" 1.4.1. Cluster Operator 1.4.2. Topic Operator 1.4.3. User Operator 1.5. AMQ Streams custom resources Expand section "1.5. AMQ Streams custom resources" Collapse section "1.5. AMQ Streams custom resources" 1.5.1. AMQ Streams custom resource example 1.6. Listener configuration 1.7. Document Conventions 2. Deployment configuration Expand section "2. Deployment configuration" Collapse section "2. Deployment configuration" 2.1. Kafka cluster configuration Expand section "2.1. Kafka cluster configuration" Collapse section "2.1. Kafka cluster configuration" 2.1.1. Sample Kafka YAML configuration 2.1.2. Data storage considerations Expand section "2.1.2. Data storage considerations" Collapse section "2.1.2. Data storage considerations" 2.1.2.1. File systems 2.1.2.2. Apache Kafka and ZooKeeper storage 2.1.3. Kafka and ZooKeeper storage types Expand section "2.1.3. Kafka and ZooKeeper storage types" Collapse section "2.1.3. Kafka and ZooKeeper storage types" 2.1.3.1. Ephemeral storage Expand section "2.1.3.1. Ephemeral storage" Collapse section "2.1.3.1. Ephemeral storage" 2.1.3.1.1. Log directories 2.1.3.2. Persistent storage Expand section "2.1.3.2. Persistent storage" Collapse section "2.1.3.2. Persistent storage" 2.1.3.2.1. Storage class overrides 2.1.3.2.2. Persistent Volume Claim naming 2.1.3.2.3. Log directories 2.1.3.3. Resizing persistent volumes 2.1.3.4. JBOD storage overview Expand section "2.1.3.4. JBOD storage overview" Collapse section "2.1.3.4. JBOD storage overview" 2.1.3.4.1. JBOD configuration 2.1.3.4.2. JBOD and Persistent Volume Claims 2.1.3.4.3. Log directories 2.1.3.5. Adding volumes to JBOD storage 2.1.3.6. Removing volumes from JBOD storage 2.1.4. Kafka broker replicas Expand section "2.1.4. Kafka broker replicas" Collapse section "2.1.4. Kafka broker replicas" 2.1.4.1. Configuring the number of broker nodes 2.1.5. Kafka broker configuration Expand section "2.1.5. Kafka broker configuration" Collapse section "2.1.5. Kafka broker configuration" 2.1.5.1. Configuring Kafka brokers 2.1.6. Listener configuration 2.1.7. ZooKeeper replicas Expand section "2.1.7. ZooKeeper replicas" Collapse section "2.1.7. ZooKeeper replicas" 2.1.7.1. Number of ZooKeeper nodes 2.1.7.2. Changing the number of ZooKeeper replicas 2.1.8. ZooKeeper configuration Expand section "2.1.8. ZooKeeper configuration" Collapse section "2.1.8. ZooKeeper configuration" 2.1.8.1. ZooKeeper configuration 2.1.8.2. Configuring ZooKeeper 2.1.9. ZooKeeper connection Expand section "2.1.9. ZooKeeper connection" Collapse section "2.1.9. ZooKeeper connection" 2.1.9.1. Connecting to ZooKeeper from a terminal 2.1.10. Entity Operator Expand section "2.1.10. Entity Operator" Collapse section "2.1.10. Entity Operator" 2.1.10.1. Entity Operator configuration properties 2.1.10.2. Topic Operator configuration properties 2.1.10.3. User Operator configuration properties 2.1.10.4. Operator loggers 2.1.10.5. Configuring the Entity Operator 2.1.11. CPU and memory resources Expand section "2.1.11. CPU and memory resources" Collapse section "2.1.11. CPU and memory resources" 2.1.11.1. Resource limits and requests Expand section "2.1.11.1. Resource limits and requests" Collapse section "2.1.11.1. Resource limits and requests" 2.1.11.1.1. Resource requests 2.1.11.1.2. Resource limits 2.1.11.1.3. Supported CPU formats 2.1.11.1.4. Supported memory formats 2.1.11.2. Configuring resource requests and limits 2.1.12. Kafka loggers 2.1.13. Kafka rack awareness Expand section "2.1.13. Kafka rack awareness" Collapse section "2.1.13. Kafka rack awareness" 2.1.13.1. Configuring rack awareness in Kafka brokers 2.1.14. Healthchecks Expand section "2.1.14. Healthchecks" Collapse section "2.1.14. Healthchecks" 2.1.14.1. Healthcheck configurations 2.1.14.2. Configuring healthchecks 2.1.15. Prometheus metrics Expand section "2.1.15. Prometheus metrics" Collapse section "2.1.15. Prometheus metrics" 2.1.15.1. Metrics configuration 2.1.15.2. Configuring Prometheus metrics 2.1.16. JMX Options Expand section "2.1.16. JMX Options" Collapse section "2.1.16. JMX Options" 2.1.16.1. Configuring JMX options 2.1.17. JVM Options Expand section "2.1.17. JVM Options" Collapse section "2.1.17. JVM Options" 2.1.17.1. JVM configuration 2.1.17.2. Configuring JVM options 2.1.18. Container images Expand section "2.1.18. Container images" Collapse section "2.1.18. Container images" 2.1.18.1. Container image configurations 2.1.18.2. Configuring container images 2.1.19. TLS sidecar Expand section "2.1.19. TLS sidecar" Collapse section "2.1.19. TLS sidecar" 2.1.19.1. TLS sidecar configuration 2.1.19.2. Configuring TLS sidecar 2.1.20. Configuring pod scheduling Expand section "2.1.20. Configuring pod scheduling" Collapse section "2.1.20. Configuring pod scheduling" 2.1.20.1. Scheduling pods based on other applications Expand section "2.1.20.1. Scheduling pods based on other applications" Collapse section "2.1.20.1. Scheduling pods based on other applications" 2.1.20.1.1. Avoid critical applications to share the node 2.1.20.1.2. Affinity 2.1.20.1.3. Configuring pod anti-affinity in Kafka components 2.1.20.2. Scheduling pods to specific nodes Expand section "2.1.20.2. Scheduling pods to specific nodes" Collapse section "2.1.20.2. Scheduling pods to specific nodes" 2.1.20.2.1. Node scheduling 2.1.20.2.2. Affinity 2.1.20.2.3. Configuring node affinity in Kafka components 2.1.20.3. Using dedicated nodes Expand section "2.1.20.3. Using dedicated nodes" Collapse section "2.1.20.3. Using dedicated nodes" 2.1.20.3.1. Dedicated nodes 2.1.20.3.2. Affinity 2.1.20.3.3. Tolerations 2.1.20.3.4. Setting up dedicated nodes and scheduling pods on them 2.1.21. Kafka Exporter 2.1.22. Performing a rolling update of a Kafka cluster 2.1.23. Performing a rolling update of a ZooKeeper cluster 2.1.24. Scaling clusters Expand section "2.1.24. Scaling clusters" Collapse section "2.1.24. Scaling clusters" 2.1.24.1. Scaling Kafka clusters Expand section "2.1.24.1. Scaling Kafka clusters" Collapse section "2.1.24.1. Scaling Kafka clusters" 2.1.24.1.1. Adding brokers to a cluster 2.1.24.1.2. Removing brokers from a cluster 2.1.24.2. Partition reassignment Expand section "2.1.24.2. Partition reassignment" Collapse section "2.1.24.2. Partition reassignment" 2.1.24.2.1. Reassignment JSON file 2.1.24.2.2. Reassigning partitions between JBOD volumes 2.1.24.3. Generating reassignment JSON files 2.1.24.4. Creating reassignment JSON files manually 2.1.24.5. Reassignment throttles 2.1.24.6. Scaling up a Kafka cluster 2.1.24.7. Scaling down a Kafka cluster 2.1.25. Deleting Kafka nodes manually 2.1.26. Deleting ZooKeeper nodes manually 2.1.27. Maintenance time windows for rolling updates Expand section "2.1.27. Maintenance time windows for rolling updates" Collapse section "2.1.27. Maintenance time windows for rolling updates" 2.1.27.1. Maintenance time windows overview 2.1.27.2. Maintenance time window definition 2.1.27.3. Configuring a maintenance time window 2.1.28. Renewing CA certificates manually 2.1.29. Replacing private keys 2.1.30. List of resources created as part of Kafka cluster 2.2. Kafka Connect/S2I cluster configuration Expand section "2.2. Kafka Connect/S2I cluster configuration" Collapse section "2.2. Kafka Connect/S2I cluster configuration" 2.2.1. Configuring Kafka Connect 2.2.2. Kafka Connect configuration for multiple instances 2.2.3. Configuring Kafka Connect user authorization 2.2.4. List of Kafka Connect cluster resources 2.2.5. List of Kafka Connect (S2I) cluster resources 2.2.6. Integrating with Debezium for change data capture 2.3. Kafka MirrorMaker cluster configuration Expand section "2.3. Kafka MirrorMaker cluster configuration" Collapse section "2.3. Kafka MirrorMaker cluster configuration" 2.3.1. Configuring Kafka MirrorMaker 2.3.2. List of Kafka MirrorMaker cluster resources 2.4. Kafka MirrorMaker 2.0 cluster configuration Expand section "2.4. Kafka MirrorMaker 2.0 cluster configuration" Collapse section "2.4. Kafka MirrorMaker 2.0 cluster configuration" 2.4.1. MirrorMaker 2.0 data replication 2.4.2. Cluster configuration Expand section "2.4.2. Cluster configuration" Collapse section "2.4.2. Cluster configuration" 2.4.2.1. Bidirectional replication (active/active) 2.4.2.2. Unidirectional replication (active/passive) 2.4.2.3. Topic configuration synchronization 2.4.2.4. Data integrity 2.4.2.5. Offset tracking 2.4.2.6. Connectivity checks 2.4.3. ACL rules synchronization 2.4.4. Synchronizing data between Kafka clusters using MirrorMaker 2.0 2.5. Kafka Bridge cluster configuration Expand section "2.5. Kafka Bridge cluster configuration" Collapse section "2.5. Kafka Bridge cluster configuration" 2.5.1. Configuring the Kafka Bridge 2.5.2. List of Kafka Bridge cluster resources 2.6. Customizing OpenShift resources Expand section "2.6. Customizing OpenShift resources" Collapse section "2.6. Customizing OpenShift resources" 2.6.1. Customizing the image pull policy 2.7. External logging Expand section "2.7. External logging" Collapse section "2.7. External logging" 2.7.1. Creating a ConfigMap for logging 3. Configuring external listeners Expand section "3. Configuring external listeners" Collapse section "3. Configuring external listeners" 3.1. Accessing Kafka using node ports 3.2. Accessing Kafka using loadbalancers 3.3. Accessing Kafka using ingress 3.4. Accessing Kafka using OpenShift routes 4. Managing secure access to Kafka Expand section "4. Managing secure access to Kafka" Collapse section "4. Managing secure access to Kafka" 4.1. Security options for Kafka Expand section "4.1. Security options for Kafka" Collapse section "4.1. Security options for Kafka" 4.1.1. Listener authentication Expand section "4.1.1. Listener authentication" Collapse section "4.1.1. Listener authentication" 4.1.1.1. Mutual TLS authentication 4.1.1.2. SCRAM-SHA-512 authentication 4.1.1.3. Network policies 4.1.1.4. Additional listener configuration options 4.1.2. Kafka authorization Expand section "4.1.2. Kafka authorization" Collapse section "4.1.2. Kafka authorization" 4.1.2.1. Super users 4.2. Security options for Kafka clients Expand section "4.2. Security options for Kafka clients" Collapse section "4.2. Security options for Kafka clients" 4.2.1. Identifying a Kafka cluster for user handling 4.2.2. User authentication Expand section "4.2.2. User authentication" Collapse section "4.2.2. User authentication" 4.2.2.1. TLS Client Authentication 4.2.2.2. SCRAM-SHA-512 Authentication 4.2.3. User authorization Expand section "4.2.3. User authorization" Collapse section "4.2.3. User authorization" 4.2.3.1. ACL rules 4.2.3.2. Super user access to Kafka brokers 4.2.3.3. User quotas 4.3. Securing access to Kafka brokers Expand section "4.3. Securing access to Kafka brokers" Collapse section "4.3. Securing access to Kafka brokers" 4.3.1. Securing Kafka brokers 4.3.2. Securing user access to Kafka 4.3.3. Restricting access to Kafka listeners using network policies 4.4. Using OAuth 2.0 token-based authentication Expand section "4.4. Using OAuth 2.0 token-based authentication" Collapse section "4.4. Using OAuth 2.0 token-based authentication" 4.4.1. OAuth 2.0 authentication mechanism 4.4.2. OAuth 2.0 Kafka broker configuration Expand section "4.4.2. OAuth 2.0 Kafka broker configuration" Collapse section "4.4.2. OAuth 2.0 Kafka broker configuration" 4.4.2.1. OAuth 2.0 client configuration on an authorization server 4.4.2.2. OAuth 2.0 authentication configuration in the Kafka cluster 4.4.2.3. Fast local JWT token validation configuration 4.4.2.4. OAuth 2.0 introspection endpoint configuration 4.4.3. Session re-authentication for Kafka brokers 4.4.4. OAuth 2.0 Kafka client configuration 4.4.5. OAuth 2.0 client authentication flow Expand section "4.4.5. OAuth 2.0 client authentication flow" Collapse section "4.4.5. OAuth 2.0 client authentication flow" 4.4.5.1. Example client authentication flows 4.4.6. Configuring OAuth 2.0 authentication Expand section "4.4.6. Configuring OAuth 2.0 authentication" Collapse section "4.4.6. Configuring OAuth 2.0 authentication" 4.4.6.1. Configuring Red Hat Single Sign-On as an OAuth 2.0 authorization server 4.4.6.2. Configuring OAuth 2.0 support for Kafka brokers 4.4.6.3. Configuring Kafka Java clients to use OAuth 2.0 4.4.6.4. Configuring OAuth 2.0 for Kafka components 4.5. Using OAuth 2.0 token-based authorization Expand section "4.5. Using OAuth 2.0 token-based authorization" Collapse section "4.5. Using OAuth 2.0 token-based authorization" 4.5.1. OAuth 2.0 authorization mechanism Expand section "4.5.1. OAuth 2.0 authorization mechanism" Collapse section "4.5.1. OAuth 2.0 authorization mechanism" 4.5.1.1. Kafka broker custom authorizer 4.5.2. Configuring OAuth 2.0 authorization support 5. Using AMQ Streams Operators Expand section "5. Using AMQ Streams Operators" Collapse section "5. Using AMQ Streams Operators" 5.1. Using the Cluster Operator Expand section "5.1. Using the Cluster Operator" Collapse section "5.1. Using the Cluster Operator" 5.1.1. Cluster Operator configuration Expand section "5.1.1. Cluster Operator configuration" Collapse section "5.1.1. Cluster Operator configuration" 5.1.1.1. Periodic reconciliation 5.1.2. Provisioning Role-Based Access Control (RBAC) Expand section "5.1.2. Provisioning Role-Based Access Control (RBAC)" Collapse section "5.1.2. Provisioning Role-Based Access Control (RBAC)" 5.1.2.1. Delegated privileges 5.1.2.2. ServiceAccount 5.1.2.3. ClusterRoles 5.1.2.4. ClusterRoleBindings 5.2. Using the Topic Operator Expand section "5.2. Using the Topic Operator" Collapse section "5.2. Using the Topic Operator" 5.2.1. Kafka topic resource Expand section "5.2.1. Kafka topic resource" Collapse section "5.2.1. Kafka topic resource" 5.2.1.1. Identifying a Kafka cluster for topic handling 5.2.1.2. Handling changes to topics 5.2.1.3. Kafka topic usage recommendations 5.2.1.4. Kafka topic naming conventions 5.2.2. Configuring a Kafka topic 5.2.3. Configuring the Topic Operator with resource requests and limits 5.3. Using the User Operator Expand section "5.3. Using the User Operator" Collapse section "5.3. Using the User Operator" 5.3.1. Configuring the User Operator with resource requests and limits 5.4. Monitoring operators using Prometheus metrics 6. Kafka Bridge Expand section "6. Kafka Bridge" Collapse section "6. Kafka Bridge" 6.1. Kafka Bridge overview Expand section "6.1. Kafka Bridge overview" Collapse section "6.1. Kafka Bridge overview" 6.1.1. Kafka Bridge interface Expand section "6.1.1. Kafka Bridge interface" Collapse section "6.1.1. Kafka Bridge interface" 6.1.1.1. HTTP requests 6.1.2. Supported clients for the Kafka Bridge 6.1.3. Securing the Kafka Bridge 6.1.4. Accessing the Kafka Bridge outside of OpenShift 6.1.5. Requests to the Kafka Bridge Expand section "6.1.5. Requests to the Kafka Bridge" Collapse section "6.1.5. Requests to the Kafka Bridge" 6.1.5.1. Content Type headers 6.1.5.2. Embedded data format 6.1.5.3. Message format 6.1.5.4. Accept headers 6.1.6. CORS Expand section "6.1.6. CORS" Collapse section "6.1.6. CORS" 6.1.6.1. Simple request 6.1.6.2. Preflighted request 6.1.7. Kafka Bridge API resources 6.1.8. Kafka Bridge deployment 6.2. Kafka Bridge quickstart Expand section "6.2. Kafka Bridge quickstart" Collapse section "6.2. Kafka Bridge quickstart" 6.2.1. Deploying the Kafka Bridge to your OpenShift cluster 6.2.2. Exposing the Kafka Bridge service to your local machine 6.2.3. Producing messages to topics and partitions 6.2.4. Creating a Kafka Bridge consumer 6.2.5. Subscribing a Kafka Bridge consumer to topics 6.2.6. Retrieving the latest messages from a Kafka Bridge consumer 6.2.7. Commiting offsets to the log 6.2.8. Seeking to offsets for a partition 6.2.9. Deleting a Kafka Bridge consumer 7. Using the Kafka Bridge with 3scale Expand section "7. Using the Kafka Bridge with 3scale" Collapse section "7. Using the Kafka Bridge with 3scale" 7.1. Using the Kafka Bridge with 3scale Expand section "7.1. Using the Kafka Bridge with 3scale" Collapse section "7.1. Using the Kafka Bridge with 3scale" 7.1.1. Kafka Bridge service discovery 7.1.2. 3scale APIcast gateway policies 7.1.3. TLS validation 7.1.4. 3scale documentation 7.2. Deploying 3scale for the Kafka Bridge 8. Cruise Control for cluster rebalancing Expand section "8. Cruise Control for cluster rebalancing" Collapse section "8. Cruise Control for cluster rebalancing" 8.1. Why use Cruise Control? 8.2. Optimization goals overview 8.3. Optimization proposals overview 8.4. Rebalance performance tuning overview 8.5. Cruise Control configuration 8.6. Deploying Cruise Control 8.7. Generating optimization proposals 8.8. Approving an optimization proposal 8.9. Stopping a cluster rebalance 8.10. Fixing problems with a KafkaRebalance resource 9. Managing schemas with Service Registry Expand section "9. Managing schemas with Service Registry" Collapse section "9. Managing schemas with Service Registry" 9.1. Why use Service Registry? 9.2. Producer schema configuration 9.3. Consumer schema configuration 9.4. Strategies to lookup a schema 9.5. Service Registry constants 9.6. Installing Service Registry 9.7. Registering a schema to Service Registry 9.8. Using a Service Registry schema from a producer client 9.9. Using a Service Registry schema from a consumer client 10. Distributed tracing Expand section "10. Distributed tracing" Collapse section "10. Distributed tracing" 10.1. Overview of OpenTracing and Jaeger 10.2. Setting up tracing for Kafka clients Expand section "10.2. Setting up tracing for Kafka clients" Collapse section "10.2. Setting up tracing for Kafka clients" 10.2.1. Initializing a Jaeger tracer for Kafka clients 10.2.2. Environment variables for tracing 10.3. Instrumenting Kafka clients with tracers Expand section "10.3. Instrumenting Kafka clients with tracers" Collapse section "10.3. Instrumenting Kafka clients with tracers" 10.3.1. Instrumenting producers and consumers for tracing Expand section "10.3.1. Instrumenting producers and consumers for tracing" Collapse section "10.3.1. Instrumenting producers and consumers for tracing" 10.3.1.1. Custom span names in a Decorator pattern 10.3.1.2. Built-in span names 10.3.2. Instrumenting Kafka Streams applications for tracing 10.4. Setting up tracing for MirrorMaker, Kafka Connect, and the Kafka Bridge Expand section "10.4. Setting up tracing for MirrorMaker, Kafka Connect, and the Kafka Bridge" Collapse section "10.4. Setting up tracing for MirrorMaker, Kafka Connect, and the Kafka Bridge" 10.4.1. Enabling tracing in MirrorMaker, Kafka Connect, and Kafka Bridge resources 11. Managing TLS certificates Expand section "11. Managing TLS certificates" Collapse section "11. Managing TLS certificates" 11.1. Certificate Authorities Expand section "11.1. Certificate Authorities" Collapse section "11.1. Certificate Authorities" 11.1.1. CA certificates 11.1.2. Installing your own CA certificates 11.2. Secrets Expand section "11.2. Secrets" Collapse section "11.2. Secrets" 11.2.1. PKCS #12 storage 11.2.2. Cluster CA Secrets 11.2.3. Client CA Secrets 11.2.4. User Secrets 11.3. Certificate renewal and validity periods Expand section "11.3. Certificate renewal and validity periods" Collapse section "11.3. Certificate renewal and validity periods" 11.3.1. Renewal process with generated CAs 11.3.2. Client applications Expand section "11.3.2. Client applications" Collapse section "11.3.2. Client applications" 11.3.2.1. Client certificate renewal 11.3.3. Renewing CA certificates manually 11.3.4. Renewing your own CA certificates 11.4. Replacing private keys 11.5. TLS connections Expand section "11.5. TLS connections" Collapse section "11.5. TLS connections" 11.5.1. ZooKeeper communication 11.5.2. Kafka interbroker communication 11.5.3. Topic and User Operators 11.5.4. Cruise Control 11.5.5. Kafka Client connections 11.6. Configuring internal clients to trust the cluster CA 11.7. Configuring external clients to trust the cluster CA 11.8. Kafka listener certificates Expand section "11.8. Kafka listener certificates" Collapse section "11.8. Kafka listener certificates" 11.8.1. Providing your own Kafka listener certificates 11.8.2. Alternative subjects in server certificates for Kafka listeners Expand section "11.8.2. Alternative subjects in server certificates for Kafka listeners" Collapse section "11.8.2. Alternative subjects in server certificates for Kafka listeners" 11.8.2.1. TLS listener SAN examples 11.8.2.2. External listener SAN examples 12. Managing AMQ Streams Expand section "12. Managing AMQ Streams" Collapse section "12. Managing AMQ Streams" 12.1. Working with custom resources Expand section "12.1. Working with custom resources" Collapse section "12.1. Working with custom resources" 12.1.1. Performing oc operations on custom resources Expand section "12.1.1. Performing oc operations on custom resources" Collapse section "12.1.1. Performing oc operations on custom resources" 12.1.1.1. Resource categories 12.1.1.2. Querying the status of sub-resources 12.1.2. AMQ Streams custom resource status information 12.1.3. Finding the status of a custom resource 12.2. Discovering services using labels and annotations Expand section "12.2. Discovering services using labels and annotations" Collapse section "12.2. Discovering services using labels and annotations" 12.2.1. Returning connection details on services 12.3. Recovering a cluster from persistent volumes Expand section "12.3. Recovering a cluster from persistent volumes" Collapse section "12.3. Recovering a cluster from persistent volumes" 12.3.1. Recovery from namespace deletion 12.3.2. Recovery from loss of an OpenShift cluster 12.3.3. Recovering a deleted cluster from persistent volumes 12.4. Tuning client configuration Expand section "12.4. Tuning client configuration" Collapse section "12.4. Tuning client configuration" 12.4.1. Kafka producer configuration tuning Expand section "12.4.1. Kafka producer configuration tuning" Collapse section "12.4.1. Kafka producer configuration tuning" 12.4.1.1. Basic producer configuration 12.4.1.2. Data durability 12.4.1.3. Ordered delivery 12.4.1.4. Reliability guarantees 12.4.1.5. Optimizing throughput and latency 12.4.2. Kafka consumer configuration tuning Expand section "12.4.2. Kafka consumer configuration tuning" Collapse section "12.4.2. Kafka consumer configuration tuning" 12.4.2.1. Basic consumer configuration 12.4.2.2. Scaling data consumption using consumer groups 12.4.2.3. Message ordering guarantees 12.4.2.4. Optimizing throughput and latency 12.4.2.5. Avoiding data loss or duplication when committing offsets Expand section "12.4.2.5. Avoiding data loss or duplication when committing offsets" Collapse section "12.4.2.5. Avoiding data loss or duplication when committing offsets" 12.4.2.5.1. Controlling transactional messages 12.4.2.6. Recovering from failure to avoid data loss 12.4.2.7. Managing offset policy 12.4.2.8. Minimizing the impact of rebalances 12.5. Uninstalling AMQ Streams A. Frequently asked questions Expand section "A. Frequently asked questions" Collapse section "A. Frequently asked questions" A.1. Questions related to the Cluster Operator Expand section "A.1. Questions related to the Cluster Operator" Collapse section "A.1. Questions related to the Cluster Operator" A.1.1. Why do I need cluster administrator privileges to install AMQ Streams? A.1.2. Why does the Cluster Operator need to create ClusterRoleBindings? A.1.3. Can standard OpenShift users create Kafka custom resources? A.1.4. What do the failed to acquire lock warnings in the log mean? A.1.5. Why is hostname verification failing when connecting to NodePorts using TLS? B. Custom Resource API Reference Expand section "B. Custom Resource API Reference" Collapse section "B. Custom Resource API Reference" B.1. Common configuration properties Expand section "B.1. Common configuration properties" Collapse section "B.1. Common configuration properties" B.1.1. replicas B.1.2. bootstrapServers B.1.3. ssl B.1.4. trustedCertificates B.1.5. resources B.1.6. image B.1.7. livenessProbe and readinessProbe healthchecks B.1.8. metrics B.1.9. jvmOptions B.1.10. Garbage collector logging B.2. Kafka schema reference B.3. KafkaSpec schema reference B.4. KafkaClusterSpec schema reference Expand section "B.4. KafkaClusterSpec schema reference" Collapse section "B.4. KafkaClusterSpec schema reference" B.4.1. listeners B.4.2. config B.5. EphemeralStorage schema reference B.6. PersistentClaimStorage schema reference B.7. PersistentClaimStorageOverride schema reference B.8. JbodStorage schema reference B.9. GenericKafkaListener schema reference Expand section "B.9. GenericKafkaListener schema reference" Collapse section "B.9. GenericKafkaListener schema reference" B.9.1. listeners B.9.2. type B.9.3. port B.9.4. tls B.9.5. authentication B.9.6. networkPolicyPeers B.10. KafkaListenerAuthenticationTls schema reference B.11. KafkaListenerAuthenticationScramSha512 schema reference B.12. KafkaListenerAuthenticationOAuth schema reference B.13. GenericSecretSource schema reference B.14. CertSecretSource schema reference B.15. GenericKafkaListenerConfiguration schema reference Expand section "B.15. GenericKafkaListenerConfiguration schema reference" Collapse section "B.15. GenericKafkaListenerConfiguration schema reference" B.15.1. brokerCertChainAndKey B.15.2. externalTrafficPolicy B.15.3. loadBalancerSourceRanges B.15.4. class B.15.5. preferredNodePortAddressType B.15.6. useServiceDnsDomain B.16. CertAndKeySecretSource schema reference B.17. GenericKafkaListenerConfigurationBootstrap schema reference Expand section "B.17. GenericKafkaListenerConfigurationBootstrap schema reference" Collapse section "B.17. GenericKafkaListenerConfigurationBootstrap schema reference" B.17.1. alternativeNames B.17.2. host B.17.3. nodePort B.17.4. loadBalancerIP B.17.5. annotations B.18. GenericKafkaListenerConfigurationBroker schema reference B.19. KafkaListeners schema reference B.20. KafkaListenerPlain schema reference B.21. KafkaListenerTls schema reference B.22. TlsListenerConfiguration schema reference B.23. KafkaListenerExternalRoute schema reference B.24. RouteListenerOverride schema reference B.25. RouteListenerBootstrapOverride schema reference B.26. RouteListenerBrokerOverride schema reference B.27. KafkaListenerExternalConfiguration schema reference B.28. KafkaListenerExternalLoadBalancer schema reference B.29. LoadBalancerListenerOverride schema reference B.30. LoadBalancerListenerBootstrapOverride schema reference B.31. LoadBalancerListenerBrokerOverride schema reference B.32. KafkaListenerExternalNodePort schema reference B.33. NodePortListenerOverride schema reference B.34. NodePortListenerBootstrapOverride schema reference B.35. NodePortListenerBrokerOverride schema reference B.36. NodePortListenerConfiguration schema reference B.37. KafkaListenerExternalIngress schema reference B.38. IngressListenerConfiguration schema reference B.39. IngressListenerBootstrapConfiguration schema reference B.40. IngressListenerBrokerConfiguration schema reference B.41. KafkaAuthorizationSimple schema reference Expand section "B.41. KafkaAuthorizationSimple schema reference" Collapse section "B.41. KafkaAuthorizationSimple schema reference" B.41.1. superUsers B.42. KafkaAuthorizationOpa schema reference Expand section "B.42. KafkaAuthorizationOpa schema reference" Collapse section "B.42. KafkaAuthorizationOpa schema reference" B.42.1. url B.42.2. allowOnError B.42.3. initialCacheCapacity B.42.4. maximumCacheSize B.42.5. expireAfterMs B.42.6. superUsers B.43. KafkaAuthorizationKeycloak schema reference B.44. Rack schema reference B.45. Probe schema reference B.46. JvmOptions schema reference B.47. SystemProperty schema reference B.48. KafkaJmxOptions schema reference B.49. KafkaJmxAuthenticationPassword schema reference B.50. InlineLogging schema reference B.51. ExternalLogging schema reference B.52. TlsSidecar schema reference B.53. KafkaClusterTemplate schema reference B.54. StatefulSetTemplate schema reference B.55. MetadataTemplate schema reference B.56. PodTemplate schema reference Expand section "B.56. PodTemplate schema reference" Collapse section "B.56. PodTemplate schema reference" B.56.1. hostAliases B.57. ResourceTemplate schema reference B.58. ExternalServiceTemplate schema reference B.59. PodDisruptionBudgetTemplate schema reference B.60. ContainerTemplate schema reference B.61. ContainerEnvVar schema reference B.62. ZookeeperClusterSpec schema reference B.63. ZookeeperClusterTemplate schema reference B.64. TopicOperatorSpec schema reference B.65. EntityOperatorSpec schema reference B.66. EntityTopicOperatorSpec schema reference B.67. EntityUserOperatorSpec schema reference B.68. EntityOperatorTemplate schema reference B.69. CertificateAuthority schema reference B.70. CruiseControlSpec schema reference B.71. CruiseControlTemplate schema reference B.72. BrokerCapacity schema reference B.73. KafkaExporterSpec schema reference B.74. KafkaExporterTemplate schema reference B.75. KafkaStatus schema reference B.76. Condition schema reference B.77. ListenerStatus schema reference B.78. ListenerAddress schema reference B.79. KafkaConnect schema reference B.80. KafkaConnectSpec schema reference Expand section "B.80. KafkaConnectSpec schema reference" Collapse section "B.80. KafkaConnectSpec schema reference" B.80.1. config B.80.2. logging B.81. KafkaConnectTls schema reference Expand section "B.81. KafkaConnectTls schema reference" Collapse section "B.81. KafkaConnectTls schema reference" B.81.1. trustedCertificates B.82. KafkaClientAuthenticationTls schema reference Expand section "B.82. KafkaClientAuthenticationTls schema reference" Collapse section "B.82. KafkaClientAuthenticationTls schema reference" B.82.1. certificateAndKey B.83. KafkaClientAuthenticationScramSha512 schema reference Expand section "B.83. KafkaClientAuthenticationScramSha512 schema reference" Collapse section "B.83. KafkaClientAuthenticationScramSha512 schema reference" B.83.1. username B.83.2. passwordSecret B.84. PasswordSecretSource schema reference B.85. KafkaClientAuthenticationPlain schema reference Expand section "B.85. KafkaClientAuthenticationPlain schema reference" Collapse section "B.85. KafkaClientAuthenticationPlain schema reference" B.85.1. username B.85.2. passwordSecret B.86. KafkaClientAuthenticationOAuth schema reference B.87. JaegerTracing schema reference B.88. KafkaConnectTemplate schema reference B.89. ExternalConfiguration schema reference Expand section "B.89. ExternalConfiguration schema reference" Collapse section "B.89. ExternalConfiguration schema reference" B.89.1. env B.89.2. volumes B.90. ExternalConfigurationEnv schema reference B.91. ExternalConfigurationEnvVarSource schema reference B.92. ExternalConfigurationVolumeSource schema reference B.93. KafkaConnectStatus schema reference B.94. ConnectorPlugin schema reference B.95. KafkaConnectS2I schema reference B.96. KafkaConnectS2ISpec schema reference B.97. KafkaConnectS2IStatus schema reference B.98. KafkaTopic schema reference B.99. KafkaTopicSpec schema reference B.100. KafkaTopicStatus schema reference B.101. KafkaUser schema reference B.102. KafkaUserSpec schema reference B.103. KafkaUserTlsClientAuthentication schema reference B.104. KafkaUserScramSha512ClientAuthentication schema reference B.105. KafkaUserAuthorizationSimple schema reference B.106. AclRule schema reference Expand section "B.106. AclRule schema reference" Collapse section "B.106. AclRule schema reference" B.106.1. resource B.106.2. type B.106.3. operation B.106.4. host B.107. AclRuleTopicResource schema reference B.108. AclRuleGroupResource schema reference B.109. AclRuleClusterResource schema reference B.110. AclRuleTransactionalIdResource schema reference B.111. KafkaUserQuotas schema reference Expand section "B.111. KafkaUserQuotas schema reference" Collapse section "B.111. KafkaUserQuotas schema reference" B.111.1. quotas B.112. KafkaUserTemplate schema reference B.113. KafkaUserStatus schema reference B.114. KafkaMirrorMaker schema reference B.115. KafkaMirrorMakerSpec schema reference Expand section "B.115. KafkaMirrorMakerSpec schema reference" Collapse section "B.115. KafkaMirrorMakerSpec schema reference" B.115.1. whitelist B.115.2. KafkaMirrorMakerConsumerSpec and KafkaMirrorMakerProducerSpec B.115.3. logging B.116. KafkaMirrorMakerConsumerSpec schema reference Expand section "B.116. KafkaMirrorMakerConsumerSpec schema reference" Collapse section "B.116. KafkaMirrorMakerConsumerSpec schema reference" B.116.1. numStreams B.116.2. offsetCommitInterval B.116.3. config B.116.4. groupId B.117. KafkaMirrorMakerTls schema reference Expand section "B.117. KafkaMirrorMakerTls schema reference" Collapse section "B.117. KafkaMirrorMakerTls schema reference" B.117.1. trustedCertificates B.118. KafkaMirrorMakerProducerSpec schema reference Expand section "B.118. KafkaMirrorMakerProducerSpec schema reference" Collapse section "B.118. KafkaMirrorMakerProducerSpec schema reference" B.118.1. abortOnSendFailure B.118.2. config B.119. KafkaMirrorMakerTemplate schema reference B.120. KafkaMirrorMakerStatus schema reference B.121. KafkaBridge schema reference B.122. KafkaBridgeSpec schema reference Expand section "B.122. KafkaBridgeSpec schema reference" Collapse section "B.122. KafkaBridgeSpec schema reference" B.122.1. logging B.123. KafkaBridgeTls schema reference B.124. KafkaBridgeHttpConfig schema reference Expand section "B.124. KafkaBridgeHttpConfig schema reference" Collapse section "B.124. KafkaBridgeHttpConfig schema reference" B.124.1. cors B.125. KafkaBridgeHttpCors schema reference B.126. KafkaBridgeConsumerSpec schema reference B.127. KafkaBridgeProducerSpec schema reference B.128. KafkaBridgeTemplate schema reference B.129. KafkaBridgeStatus schema reference B.130. KafkaConnector schema reference B.131. KafkaConnectorSpec schema reference B.132. KafkaConnectorStatus schema reference B.133. KafkaMirrorMaker2 schema reference B.134. KafkaMirrorMaker2Spec schema reference B.135. KafkaMirrorMaker2ClusterSpec schema reference Expand section "B.135. KafkaMirrorMaker2ClusterSpec schema reference" Collapse section "B.135. KafkaMirrorMaker2ClusterSpec schema reference" B.135.1. config B.136. KafkaMirrorMaker2Tls schema reference B.137. KafkaMirrorMaker2MirrorSpec schema reference B.138. KafkaMirrorMaker2ConnectorSpec schema reference B.139. KafkaMirrorMaker2Status schema reference B.140. KafkaRebalance schema reference B.141. KafkaRebalanceSpec schema reference B.142. KafkaRebalanceStatus schema reference C. Using Your Subscription Legal Notice Settings Close Language: 简体中文 日本語 English Language: 简体中文 日本語 English Format: Multi-page Single-page PDF Format: Multi-page Single-page PDF Language and Page Formatting Options Language: 简体中文 日本語 English Language: 简体中文 日本語 English Format: Multi-page Single-page PDF Format: Multi-page Single-page PDF Using AMQ Streams on OpenShift Red Hat AMQ 2020.Q4For use with AMQ Streams 1.6 on OpenShift Container PlatformLegal NoticeAbstract This guide describes how to install, configure, and manage Red Hat AMQ Streams to build a large-scale messaging network. Next