Chapter 4. Preparing for your AMQ Streams deployment

This section shows how you prepare for an AMQ Streams deployment, describing:

Note

To run the commands in this guide, your cluster user must have the rights to manage role-based access control (RBAC) and CRDs.

4.1. Deployment prerequisites

To deploy AMQ Streams, you will need the following:

  • An OpenShift 4.10 to 4.13 cluster.

    AMQ Streams is based on Strimzi 0.34.x.

  • The oc command-line tool is installed and configured to connect to the running cluster.

4.2. Downloading AMQ Streams release artifacts

To use deployment files to install AMQ Streams, download and extract the files from the AMQ Streams software downloads page.

AMQ Streams release artifacts include sample YAML files to help you deploy the components of AMQ Streams to OpenShift, perform common operations, and configure your Kafka cluster.

Use oc to deploy the Cluster Operator from the install/cluster-operator folder of the downloaded ZIP file. For more information about deploying and configuring the Cluster Operator, see Section 6.2, “Deploying the Cluster Operator”.

In addition, if you want to use standalone installations of the Topic and User Operators with a Kafka cluster that is not managed by the AMQ Streams Cluster Operator, you can deploy them from the install/topic-operator and install/user-operator folders.

Note

Additionally, AMQ Streams container images are available through the Red Hat Ecosystem Catalog. However, we recommend that you use the YAML files provided to deploy AMQ Streams.

4.3. Example configuration and deployment files

Use the example configuration and deployment files provided with AMQ Streams to deploy Kafka components with different configurations and monitor your deployment. Example configuration files for custom resources contain important properties and values, which you can extend with additional supported configuration properties for your own deployment.

4.3.1. Example files location

The example files are provided with the downloadable release artifacts from the AMQ Streams software downloads page.

You can download and apply the examples using the oc command-line tool. The examples can serve as a starting point when building your own Kafka component configuration for deployment.

Note

If you installed AMQ Streams using the Operator, you can still download the example files and use them to upload configuration.

4.3.2. Example files provided with AMQ Streams

The release artifacts include an examples directory that contains the configuration examples.

Examples directory

examples
├── user 1
├── topic 2
├── security 3
│   ├── tls-auth
│   ├── scram-sha-512-auth
│   └── keycloak-authorization
├── mirror-maker 4
├── metrics 5
├── kafka 6
├── cruise-control 7
├── connect 8
└── bridge 9

1
KafkaUser custom resource configuration, which is managed by the User Operator.
2
KafkaTopic custom resource configuration, which is managed by Topic Operator.
3
Authentication and authorization configuration for Kafka components. Includes example configuration for TLS and SCRAM-SHA-512 authentication. The Red Hat Single Sign-On example includes Kafka custom resource configuration and a Red Hat Single Sign-On realm specification. You can use the example to try Red Hat Single Sign-On authorization services. There is also an example with enabled oauth authentication and keycloak authorization metrics.
4
Kafka custom resource configuration for a deployment of Mirror Maker. Includes example configuration for replication policy and synchronization frequency.
5
Metrics configuration, including Prometheus installation and Grafana dashboard files.
6
Kafka custom resource configuration for a deployment of Kafka. Includes example configuration for an ephemeral or persistent single or multi-node deployment.
7
Kafka custom resource with a deployment configuration for Cruise Control. Includes KafkaRebalance custom resources to generate optimizations proposals from Cruise Control, with example configurations to use the default or user optimization goals.
8
KafkaConnect and KafkaConnector custom resource configuration for a deployment of Kafka Connect. Includes example configuration for a single or multi-node deployment.
9
KafkaBridge custom resource configuration for a deployment of Kafka Bridge.

4.4. Pushing container images to your own registry

Container images for AMQ Streams are available in the Red Hat Ecosystem Catalog. The installation YAML files provided by AMQ Streams will pull the images directly from the Red Hat Ecosystem Catalog.

If you do not have access to the Red Hat Ecosystem Catalog or want to use your own container repository, do the following:

  1. Pull all container images listed here
  2. Push them into your own registry
  3. Update the image names in the installation YAML files
Note

Each Kafka version supported for the release has a separate image.

Container imageNamespace/RepositoryDescription

Kafka

  • registry.redhat.io/amq-streams/kafka-34-rhel8:2.4.0
  • registry.redhat.io/amq-streams/kafka-33-rhel8:2.4.0

AMQ Streams image for running Kafka, including:

  • Kafka Broker
  • Kafka Connect
  • Kafka MirrorMaker
  • ZooKeeper
  • TLS Sidecars

Operator

  • registry.redhat.io/amq-streams/strimzi-rhel8-operator:2.4.0

AMQ Streams image for running the operators:

  • Cluster Operator
  • Topic Operator
  • User Operator
  • Kafka Initializer

Kafka Bridge

  • registry.redhat.io/amq-streams/bridge-rhel8:2.4.0

AMQ Streams image for running the AMQ Streams Kafka Bridge

AMQ Streams Drain Cleaner

  • registry.redhat.io/amq-streams/drain-cleaner-rhel8:2.4.0

AMQ Streams image for running the AMQ Streams Drain Cleaner

4.5. Creating a pull secret for authentication to the container image registry

The installation YAML files provided by AMQ Streams pull container images directly from the Red Hat Ecosystem Catalog. If an AMQ Streams deployment requires authentication, configure authentication credentials in a secret and add it to the installation YAML.

Note

Authentication is not usually required, but might be requested on certain platforms.

Prerequisites

  • You need your Red Hat username and password or the login details from your Red Hat registry service account.
Note

You can use your Red Hat subscription to create a registry service account from the Red Hat Customer Portal.

Procedure

  1. Create a pull secret containing your login details and the container registry where the AMQ Streams image is pulled from:

    oc create secret docker-registry <pull_secret_name> \
        --docker-server=registry.redhat.io \
        --docker-username=<user_name> \
        --docker-password=<password> \
        --docker-email=<email>

    Add your user name and password. The email address is optional.

  2. Edit the install/cluster-operator/060-Deployment-strimzi-cluster-operator.yaml deployment file to specify the pull secret using the STRIMZI_IMAGE_PULL_SECRET environment variable:

    apiVersion: apps/v1
    kind: Deployment
    metadata:
      name: strimzi-cluster-operator
    spec:
      # ...
      template:
        spec:
          serviceAccountName: strimzi-cluster-operator
          containers:
            # ...
            env:
              - name: STRIMZI_IMAGE_PULL_SECRETS
                value: "<pull_secret_name>"
    # ...

    The secret applies to all pods created by the Cluster Operator.

4.6. Designating AMQ Streams administrators

AMQ Streams provides custom resources for configuration of your deployment. By default, permission to view, create, edit, and delete these resources is limited to OpenShift cluster administrators. AMQ Streams provides two cluster roles that you can use to assign these rights to other users:

  • strimzi-view allows users to view and list AMQ Streams resources.
  • strimzi-admin allows users to also create, edit or delete AMQ Streams resources.

When you install these roles, they will automatically aggregate (add) these rights to the default OpenShift cluster roles. strimzi-view aggregates to the view role, and strimzi-admin aggregates to the edit and admin roles. Because of the aggregation, you might not need to assign these roles to users who already have similar rights.

The following procedure shows how to assign a strimzi-admin role that allows non-cluster administrators to manage AMQ Streams resources.

A system administrator can designate AMQ Streams administrators after the Cluster Operator is deployed.

Prerequisites

Procedure

  1. Create the strimzi-view and strimzi-admin cluster roles in OpenShift.

    oc create -f install/strimzi-admin
  2. If needed, assign the roles that provide access rights to users that require them.

    oc create clusterrolebinding strimzi-admin --clusterrole=strimzi-admin --user=user1 --user=user2