Getting started with the rhoas CLI for Red Hat OpenShift Streams for Apache Kafka

Guide
  • Red Hat OpenShift Streams for Apache Kafka 1
  • Updated 09 November 2021
  • Published 19 October 2021

Getting started with the rhoas CLI for Red Hat OpenShift Streams for Apache Kafka

Guide
Red Hat OpenShift Streams for Apache Kafka 1
  • Updated 09 November 2021
  • Published 19 October 2021

With the rhoas command-line interface (CLI), you can manage OpenShift Streams for Apache Kafka from a terminal.

As a developer of applications and services, you can use rhoas to create Kafka instances and connect your applications and services to these instances. You can also create Kafka topics to store data for producers and consumers.

This guide describes how to get started quickly by doing the following:

Prerequisites

Creating a Kafka instance

A Kafka instance includes a Kafka cluster, bootstrap server, and other required configurations for connecting to Kafka producer and consumer services.

Prerequisites
  • You are logged in to rhoas.

Procedure
  1. Enter the following command to create a Kafka instance with default values.

    This example creates a Kafka instance called my-kafka.

    Creating a Kafka instance
    $ rhoas kafka create --name my-kafka
    {
      "cloud_provider": "aws",
      "created_at": "2021-03-17T18:33:50.000799005Z",
      "href": "/api/managed-services-api/v1/kafkas/1ptcsACdx7HVzacKZBA6HRRa1oW",
      "id": "1ptcsACdx7HVzacKZBA6HRRa1oW",
      "kind": "Kafka",
      "multi_az": true,
      "name": "my-kafka",
      "owner": "developer",
      "region": "us-east-1",
      "status": "accepted",
      "updated_at": "2021-03-17T18:33:50.000799005Z"
    }
    Kafka instance "my-kafka" is being created. To monitor its status run "rhoas status".

    If you do not want to use the default values, enter the following command: rhoas kafka create. You will be prompted to enter the Name, Cloud Provider, and Cloud Region for the Kafka instance.

  2. Enter the following command to verify that the Kafka instance is ready to use.

    This command shows that the Kafka instance is ready to use, because the Status field is ready.

    Reviewing details of a Kafka instance
    $ rhoas status kafka
    
      Kafka
      -----------------------------------------------------------------------------------
      ID:                     1ptdfZRHmLKwqW6A3YKM2MawgDh
      Name:                   my-kafka
      Status:                 ready
      Bootstrap URL:          my-kafka--ptdfzrhmlkwqw-a-ykm-mawgdh.kafka.devshift.org:443

    When you created the Kafka instance, it was set as the current instance automatically. If you have multiple Kafka instances, you can switch to a different instance by using the rhoas kafka use command.

Creating a service account

To connect your applications or services to a Kafka instance, you must first create a service account with credentials. The credentials are exported to a file on your computer, which you can use to authenticate your application with your Kafka instance.

Prerequisites
  • You have created a Kafka instance, and it has a ready status.

Procedure
  1. Enter the following command to create a service account.

    This example creates a service account and saves the credentials in a JSON file.

    Creating a service account
    $ rhoas service-account create --file-format json --short-description="rhoas-service-account"
    
    ✔️ Service account created successfully with ID "04ba07d6-b08c-45fb-af53-3522c1b4c32f".
    ✔️ Credentials saved to /home/developer/my-project/credentials.json

    Your service account credentials are created and saved to a JSON file.

    When creating a service account, you can choose the file format and location to save the credentials. For more information, see the rhoas service-account create command help.

  2. To verify your service account credentials, view the credentials.json file that you created.

    Verifying service account credentials
    $ cat credentials.json
    {
    	"clientID":"srvc-acct-8c95ca5e1225-94a-41f1-ab97-aacf3df1",
    	"clientSecret":"facf3df1-ab97-2253-aa87-ab97",
            "oauthTokenUrl": "https://identity.api.openshift.com/auth/realms/rhoas/protocol/openid-connect/token"
    }

    You’ll use these credentials and the bootstrap server URL to connect your applications and services to your Kafka instance.

  3. Set the permissions for your service account to access the Kafka instance resources.

    This example allows applications to produce and consume messages from topics starting with a specified prefix that belong to any consumer group.

    Assigning access permissions to a service account
    $ rhoas kafka acl grant-access --consumer --producer \
        --service-account srvc-acct-8c95ca5e1225-94a-41f1-ab97-aacf3df1 --topic-prefix myapp  --group all
    
    The following ACL rules are to be created:
    
      PRINCIPAL (7)                                   PERMISSION   OPERATION   DESCRIPTION
     ----------------------------------------------- ------------ ----------- ---------------------------
      srvc-acct-8c95ca5e1225-94a-41f1-ab97-aacf3df1   allow        describe    topic starts with "myapp"
      srvc-acct-8c95ca5e1225-94a-41f1-ab97-aacf3df1   allow        read        topic starts with "myapp"
      srvc-acct-8c95ca5e1225-94a-41f1-ab97-aacf3df1   allow        read        group is "*"
      srvc-acct-8c95ca5e1225-94a-41f1-ab97-aacf3df1   allow        write       topic starts with "myapp"
      srvc-acct-8c95ca5e1225-94a-41f1-ab97-aacf3df1   allow        create      topic starts with "myapp"
      srvc-acct-8c95ca5e1225-94a-41f1-ab97-aacf3df1   allow        write       transactional-id is "*"
      srvc-acct-8c95ca5e1225-94a-41f1-ab97-aacf3df1   allow        describe    transactional-id is "*"
    
    ? Are you sure you want to create the listed ACL rules (y/N)

    The permissions you assign depend on whether your application produces or consumes messages.

Creating a Kafka topic

After creating a Kafka instance, you can create Kafka topics to start producing and consuming messages in your services.

Prerequisites
  • You have created a Kafka instance, and it has a ready status.

You can use rhoas kafka use to switch to a specific Kafka instance.

Selecting a Kafka instance to use
$ rhoas kafka use --name my-kafka
Kafka instance "my-kafka" has been set as the current instance.
Procedure
  1. Create a Kafka topic with default values.

    This example creates the my-topic Kafka topic.

    Creating a Kafka topic with default values
    $ rhoas kafka topic create --name my-topic
    Topic "my-topic" created in Kafka instance "my-kafka":
    {
        "config": [
            {
                "key": "compression.type",
                "value": "producer"
            },
            {
                "key": "leader.replication.throttled.replicas",
                "value": ""
            },
            {
                "key": "min.insync.replicas",
                "value": "2"
            },
            {
                "key": "message.downconversion.enable",
                "value": "true"
            },
            {
                "key": "segment.jitter.ms",
                "value": "0"
            },
            {
                "key": "cleanup.policy",
                "value": "delete"
            },
            {
                "key": "flush.ms",
                "value": "9223372036854775807"
            },
            {
                "key": "follower.replication.throttled.replicas",
                "value": ""
            },
            {
                "key": "segment.bytes",
                "value": "1073741824"
            },
            {
                "key": "retention.ms",
                "value": "604800000"
            },
            {
                "key": "flush.messages",
                "value": "9223372036854775807"
            },
            {
                "key": "message.format.version",
                "value": "2.7-IV2"
            },
            {
                "key": "max.compaction.lag.ms",
                "value": "9223372036854775807"
            },
            {
                "key": "file.delete.delay.ms",
                "value": "60000"
            },
            {
                "key": "max.message.bytes",
                "value": "1048588"
            },
            {
                "key": "min.compaction.lag.ms",
                "value": "0"
            },
            {
                "key": "message.timestamp.type",
                "value": "CreateTime"
            },
            {
                "key": "preallocate",
                "value": "false"
            },
            {
                "key": "index.interval.bytes",
                "value": "4096"
            },
            {
                "key": "min.cleanable.dirty.ratio",
                "value": "0.5"
            },
            {
                "key": "unclean.leader.election.enable",
                "value": "false"
            },
            {
                "key": "retention.bytes",
                "value": "-1"
            },
            {
                "key": "delete.retention.ms",
                "value": "86400000"
            },
            {
                "key": "segment.ms",
                "value": "604800000"
            },
            {
                "key": "message.timestamp.difference.max.ms",
                "value": "9223372036854775807"
            },
            {
                "key": "segment.index.bytes",
                "value": "10485760"
            }
        ],
        "name": "my-topic",
        "partitions": [
            {
                "id": 0,
                "isr": [
                    {
                        "id": 1
                    },
                    {
                        "id": 2
                    },
                    {
                        "id": 0
                    }
                ],
                "leader": {
                    "id": 1
                },
                "replicas": [
                    {
                        "id": 1
                    },
                    {
                        "id": 2
                    },
                    {
                        "id": 0
                    }
                ]
            }
        ]
    }

    If you do not want to use the default values, you can specify the number of partitions (--partitions) and message retention time (--retention-ms). For more information, use the command help rhoas kafka topic create -h.

  2. If necessary, you can edit or delete the topic by using the rhoas kafka topic update and rhoas kafka topic delete commands.

Commands for managing Kafka

For more information about the rhoas commands you can use to manage your Kafka instance, use the following command help:

  • rhoas kafka -h for Kafka instances

  • rhoas service-account -h for service accounts

  • rhoas kafka acl -h for access management

  • rhoas kafka topic -h for Kafka topics

Additional resources