Chapter 4. AWS Kinesis Sink

Send data to AWS Kinesis.

The Kamelet expects the following header:

  • partition / ce-partition: to set the Kinesis partition key

If the header won’t be set the exchange ID will be used.

The Kamelet is also able to recognize the following header:

  • sequence-number / ce-sequencenumber: to set the Sequence number

This header is optional.

4.1. Configuration Options

The following table summarizes the configuration options available for the aws-kinesis-sink Kamelet:

PropertyNameDescriptionTypeDefaultExample

accessKey *

Access Key

The access key obtained from AWS

string

  

region *

AWS Region

The AWS region to connect to

string

 

"eu-west-1"

secretKey *

Secret Key

The secret key obtained from AWS

string

  

stream *

Stream Name

The Kinesis stream that you want to access (needs to be created in advance)

string

  
Note

Fields marked with an asterisk (*) are mandatory.

4.2. Dependencies

At runtime, the aws-kinesis-sink Kamelet relies upon the presence of the following dependencies:

  • camel:aws2-kinesis
  • camel:kamelet

4.3. Usage

This section describes how you can use the aws-kinesis-sink.

4.3.1. Knative Sink

You can use the aws-kinesis-sink Kamelet as a Knative sink by binding it to a Knative object.

aws-kinesis-sink-binding.yaml

apiVersion: camel.apache.org/v1alpha1
kind: KameletBinding
metadata:
  name: aws-kinesis-sink-binding
spec:
  source:
    ref:
      kind: Channel
      apiVersion: messaging.knative.dev/v1
      name: mychannel
  sink:
    ref:
      kind: Kamelet
      apiVersion: camel.apache.org/v1alpha1
      name: aws-kinesis-sink
    properties:
      accessKey: "The Access Key"
      region: "eu-west-1"
      secretKey: "The Secret Key"
      stream: "The Stream Name"

4.3.1.1. Prerequisite

Make sure you have "Red Hat Integration - Camel K" installed into the OpenShift cluster you’re connected to.

4.3.1.2. Procedure for using the cluster CLI

  1. Save the aws-kinesis-sink-binding.yaml file to your local drive, and then edit it as needed for your configuration.
  2. Run the sink by using the following command:

    oc apply -f aws-kinesis-sink-binding.yaml

4.3.1.3. Procedure for using the Kamel CLI

Configure and run the sink by using the following command:

kamel bind channel:mychannel aws-kinesis-sink -p "sink.accessKey=The Access Key" -p "sink.region=eu-west-1" -p "sink.secretKey=The Secret Key" -p "sink.stream=The Stream Name"

This command creates the KameletBinding in the current namespace on the cluster.

4.3.2. Kafka Sink

You can use the aws-kinesis-sink Kamelet as a Kafka sink by binding it to a Kafka topic.

aws-kinesis-sink-binding.yaml

apiVersion: camel.apache.org/v1alpha1
kind: KameletBinding
metadata:
  name: aws-kinesis-sink-binding
spec:
  source:
    ref:
      kind: KafkaTopic
      apiVersion: kafka.strimzi.io/v1beta1
      name: my-topic
  sink:
    ref:
      kind: Kamelet
      apiVersion: camel.apache.org/v1alpha1
      name: aws-kinesis-sink
    properties:
      accessKey: "The Access Key"
      region: "eu-west-1"
      secretKey: "The Secret Key"
      stream: "The Stream Name"

4.3.2.1. Prerequisites

Ensure that you’ve installed the AMQ Streams operator in your OpenShift cluster and created a topic named my-topic in the current namespace. Make also sure you have "Red Hat Integration - Camel K" installed into the OpenShift cluster you’re connected to.

4.3.2.2. Procedure for using the cluster CLI

  1. Save the aws-kinesis-sink-binding.yaml file to your local drive, and then edit it as needed for your configuration.
  2. Run the sink by using the following command:

    oc apply -f aws-kinesis-sink-binding.yaml

4.3.2.3. Procedure for using the Kamel CLI

Configure and run the sink by using the following command:

kamel bind kafka.strimzi.io/v1beta1:KafkaTopic:my-topic aws-kinesis-sink -p "sink.accessKey=The Access Key" -p "sink.region=eu-west-1" -p "sink.secretKey=The Secret Key" -p "sink.stream=The Stream Name"

This command creates the KameletBinding in the current namespace on the cluster.

4.4. Kamelet source file

https://github.com/openshift-integration/kamelet-catalog/tree/kamelet-catalog-1.8//aws-kinesis-sink.kamelet.yaml