Chapter 2. Connecting to Amazon Web Services

Procedure

1. In Fuse Online, in the left panel, click Connections to display any available connections.
2. Click Create Connection to display Fuse Online connectors.
3. Click the Amazon DynamoDB connector.
4. In the Access Key field, enter an Amazon access key ID that is part of a user access key in the AWS account that manages the DynamoDB table that you want the connection to access.
5. In the Secret Key field, enter the Amazon secret access key for the Amazon access key ID that you specified.
6. In the Region field, select the AWS region in which the DynamoDB table resides.
7. In the Table Name field, enter the name of the DynamoDB table that you want this connection to access.
8. Click Validate. Fuse Online immediately tries to validate the connection and displays a message that indicates whether or not validation is successful. If validation fails, revise the configuration details as needed and try again.
9. When validation is successful, click Next.
10. In the Name field, enter your choice of a name that helps you distinguish this connection from any other connections. For example, enter DynamoDB West.
11. In the Description field, optionally enter any information that is helpful to know about this connection. For example, enter Sample DynamoDB connection that accesses sales data in the western region.
12. Click Save to see that the connection you created is now available. If you entered the example name, you would see that DynamoDB West appears as a connection that you can choose to add to an integration.

Procedure

1. In the flow visualization, click the plus sign where you want to add an Amazon DynamoDB connection.
2. On the Choose a connection page, click the Amazon DynamoDB connection that is configured to access that table that you want to obtain data from.
3. On the Choose an action page, select the Query action.

4. In the Filter field, specify JSON notation that identifies the key attribute(s) for the item that the connection should return. You can specify input parameters with :#. For example, to obtain an item that has two key attributes, the specification might be something like this:

   {"key1":":#PARAM1", "key2":":#PARAM2"}

5. Optional. In the Attributes to query field, enter a comma-separated list of the attribute names that contain the data that you want the connection to return. For example:

   key1,key2,attribute3,attribute4,attribute5

   If you do not specify attributes, the query returns the key attribute(s).

6. Click Next.

Procedure

1. On the Choose a connection page, click the Amazon DynamoDB connection that is configured to access the table that you want to add to.
2. On the Choose an action page, select Put Item.

3. In the JSON definition of the element field, enter JSON notation that defines the table item to be added. You must specify the item’s key attribute(s). Input parameter placeholders prefixed with :# are allowed. Be sure to specify all attributes/values that you want to store. In the following example, userID is the key:

   {"userID":"aslan","attr1":"some-value","attr2":"another-value"}

4. Click Next.

Procedure

1. On the Choose a connection page, click the Amazon DynamoDB connection that is configured to access the table that contains the item that you want to remove.
2. On the Choose an action page, select Remove Item.

3. In the JSON filter of the element field, enter JSON notation that identifies the item that you want to remove. You must specify the item’s key attribute(s). Input parameters prefixed with :# are allowed. In the following example input, userID is a key attribute:

   {"userID":"aslan"}

4. Click Next.

Procedure

1. In Fuse Online, in the left panel, click Connections to display any available connections.
2. Click Create Connection to display Fuse Online connectors.
3. Click the Amazon SNS connector.

4. In the Access Key field, enter an Amazon access key ID that is part of a user access key in the AWS account that manages the topic that you want the connection to send a message to.

   If the topic that you want the connection to access does not already exist then when Fuse Online tries to start running the integration, it uses the AWS account associated with this access key to create the topic. However, if the topic already exists in some other AWS account, then the connection cannot create the topic and the integration cannot start.

5. In the Secret Key field, enter the Amazon secret access key for the Amazon access key ID that you specified.
6. In the Region field, select the AWS region in which the topic resides. If the connection creates the topic, then it creates it in the selected region.
7. Click Validate. Fuse Online immediately tries to validate the connection and displays a message that indicates whether or not validation is successful. If validation fails, revise the configuration details as needed and try again.
8. When validation is successful, click Next.
9. In the Name field, enter your choice of a name that helps you distinguish this connection from any other connections. For example, enter SNS North.
10. In the Description field, optionally enter any information that is helpful to know about this connection. For example, enter Sample SNS connection that sends messages to our north region topic.
11. Click Save to see that the connection you created is now available. If you entered the example name, you would see that SNS North appears as a connection that you can choose to add to an integration.

Procedure

1. On the Add to Integration page, click the plus sign where you want to add the connection. Skip this step if Fuse Online is prompting you to choose the finish connection.
2. Click the Amazon SNS connection that you want to use to send a message.
3. Click the Send Object action.

4. In the Topic Name or Amazon Resource Name field, enter the name of the SNS topic to send a message to or enter the topics’s Amazon Resource Name (ARN).

   If the SNS topic does not already exist in either the AWS account that the connection is authorized to access, or in any other AWS account, the connection creates the topic in the AWS account that the connection is configured to access.

5. Click Next to add the connection to the integration.

Procedure

1. In Fuse Online, in the left panel, click Connections to display any available connections.
2. Click Create Connection to display Fuse Online connectors.
3. Click the Amazon SQS connector.

4. In the Access Key field, enter an Amazon access key ID that is part of a user access key in the AWS account that manages the queue that you want the connection to access.

   If the queue that you want the connection to access does not already exist then when Fuse Online tries to start running the integration, it uses the AWS account associated with this access key to create the queue. However, if the queue already exists in some other AWS account, then the connection cannot create the queue and the integration cannot start.

5. In the Secret Key field, enter the Amazon secret access key for the Amazon access key ID that you specified.
6. In the Region field, select the AWS region in which the queue resides. If the connection creates the queue, then it creates it in the selected region.
7. Click Validate. Fuse Online immediately tries to validate the connection and displays a message that indicates whether or not validation is successful. If validation fails, revise the configuration details as needed and try again.
8. When validation is successful, click Next.
9. In the Name field, enter your choice of a name that helps you distinguish this connection from any other connections. For example, enter SQS South.
10. In the Description field, optionally enter any information that is helpful to know about this connection. For example, enter Sample SQS connection that sends messages to our southern region queue.
11. Click Save to see that the connection you created is now available. If you entered the example name, you would see that SQS South appears as a connection that you can choose to add to an integration.

Procedure

1. In the Fuse Online panel on the left, click Integrations.
2. Click Create Integration.
3. On the Choose a connection page, click the Amazon SQS connection that you want to use to start the integration.
4. On the Choose an action page, select Poll an Amazon SQS Queue to periodically retrieve messages from an SQS queue.

5. Configure this action:

   1. In the Delay field, accept the default of 500 milliseconds as the time that elapses between polls. Or, to specify a different polling interval, enter a number and select its time unit.

   2. In the Maximum Objects to Retrieve field, enter the largest number of messages that one poll operation can obtain. The default is 10.

      To have no limit on the number of messages that can be obtained, specify 0 or a negative integer. When Maximum Objects to Retrieve is unlimited, the poll action obtains all messages in the SQS queue.

   3. In the Queue Name or Amazon Resource Name field, specify the name of the SQS queue to retrieve messages from, or the queue’s ARN. If a queue with this name does not exist, the connection creates it. This is the only required field.
   4. Indicate whether you want to Obtain messages and then delete them from the queue. Obtaining messages and then deleting them from the queue is the default behavior. Unselect this option only if you want the connection to retrieve messages and also leave them on the queue, which means that the messages will be retrieved again.
   5. Indicate whether you want to Obtain messages and delete the message if it does make it through a Camel filter. This option is selected by default. Leave it selected when you want to pass a retrieved message through a filter to determine whether to delete it from the queue. You do not need to unselect Obtain messages and then delete from the queue. Fuse Online ensures the correct behavior when you select Obtain messages and delete the message if it does make it through a Camel filter.
6. Click Next.

Procedure

1. On the Add to Integration page, click the plus sign where you want to add the connection. Skip this step if Fuse Online is prompting you to choose the finish connection.
2. Click the Amazon SQS connection that you want to use to send messages.

3. Select the action that you want the connection to perform:

   • Send Object sends one message to the queue.
   • Send Batch Object sends a batch of messages to the queue.

4. In the Message group ID strategy field, for a first-in, first-out (FIFO) queue, accept ConstantMessageGroupIdStrategy or click in the field and select a different strategy.

   The setting of this parameter has no effect on simple queues. The message group ID identifies the group that a message belongs to. Messages that belong to the same message group are always processed one by one, in a strict order relative to the message group. The message group ID strategy determines how the connection assigns a group ID to a message. The options are:

   • ConstantMessageGroupIdStrategy — The connection uses a constant to group messages.
   • ExchangeIdMessageGroupIdStrategy — The connection uses the exchange ID, which is in each message, to group messages.
   • PropertyValueMessageGroupIdStrategy — The connection uses the value of an internal property to group messages.

   FIFO queues are designed to ensure that the order in which messages are sent and received is strictly preserved and that each message is processed exactly once. A FIFO queue name has .fifo as a suffix.

5. In the Message deduplication ID strategy field, for a FIFO queue, accept NullMessageDeduplicationIdStrategy or click in the field to select ExchangeIdMessageDeduplicationIdStrategy.

   The setting of this parameter has no effect on simple queues. For a FIFO queue, the connection uses the selected strategy to ensure that it does not send duplicate messages to the queue.

   • NullMessageDeduplicationIdStrategy uses the message body as the deduplication element. In other words, the connection compares message bodies to identify duplicate messages.
   • ExchangeIdMessageDeduplicationIdStrategy uses the message’s exchange ID as the deduplication element. In other words, the connection compares exchange IDs to identify duplicate messages.
6. In the Queue Name or Amazon Resource Name field, enter the name of the queue to send messages to. If the queue does not exist, the connection creates it.
7. In the Delay field, accept the default, which is 0 for no delay. Or, to add a delay, specify a number. The integration waits this number of seconds before sending messages. This is useful when the message consumers might not be immediately ready for the messages that the connection is sending.
8. Click Next.

Procedure

1. In Fuse Online, in the left panel, click Connections to display any available connections.
2. Click Create Connection to display Fuse Online connectors.
3. Click the Amazon S3 connector.

4. In the Access Key field, enter an Amazon access key ID that is part of a user access key in the AWS account that manages the bucket that you want the connection to access.

   If the bucket that you want the connection to access does not already exist then when Fuse Online tries to start running the integration, it uses the AWS account associated with this access key to create the bucket. However, if the bucket already exists in some other AWS account, then the connection cannot create the bucket and the integration cannot start.

5. In the Secret Key field, enter the Amazon secret access key for the Amazon access key ID that you specified.
6. In the Region field, select the AWS region in which the bucket resides. If the connection creates the bucket, then it creates it in the selected region.

7. In the Bucket Name or Amazon Resource Name field, enter the name of the bucket that you want this connection to access or enter the bucket’s ARN.

   If the bucket you specify does not yet exist then the connection tries to create a bucket with the name that you specify. Because S3 allows a bucket to be used as a URL that can be accessed publicly, the bucket name that you specify must be globally unique. Also, it must meet S3 bucket naming requirements.

8. Click Validate. Fuse Online immediately tries to validate the connection and displays a message that indicates whether or not validation is successful. If validation fails, revise the configuration details as needed and try again.
9. When validation is successful, click Next.
10. In the Name field, enter your choice of a name that helps you distinguish this connection from any other connections. For example, enter Obtain S3 Data.
11. In the Description field, optionally enter any information that is helpful to know about this connection. For example, enter Sample S3 connection that obtains data from a northeast bucket.
12. Click Save to see that the connection you created is now available. If you entered the example name, you would see that Obtain S3 Data appears as a connection that you can choose to add to an integration.

Procedure

1. In the Fuse Online panel on the left, click Integrations.
2. Click Create Integration.
3. On the Choose a connection page, click the Amazon S3 connection that you want to use to start the integration.

4. On the Choose an action page, select the action that you want the connection to perform:

   • Get Object obtains a file from the bucket that the connection accesses. In the File Name field, enter the name of the file that you want to obtain. If the specified file is not in the bucket, it is a runtime error.

   • Poll an Amazon S3 Bucket periodically obtains files from the bucket that the connection accesses. To configure this action:

     1. In the Delay field, accept the default of 500 milliseconds as the time that elapses between polls. Or, to specify a different polling interval, enter a number and select its time unit.

     2. In the Maximum Objects to Retrieve field, enter the largest number of files that one poll operation can obtain. The default is 10.

        To have no limit on the number of files that can be obtained, specify 0 or a negative integer. When Maximum Objects to Retrieve is unlimited, the poll action obtains all files in the bucket.

        If the bucket contains more than the specified maximum number of files then the action obtains the files that were most recently modified or created.

     3. In the Prefix field, optionally specify a regular expression that evaluates to a string. If you specify a prefix then this action retrieves a file only when its name starts with that string.
     4. Indicate whether you want to Obtain files and then delete them from the bucket.
5. After you configure the action, click Next to specify the action’s output type.

6. In the Select Type field, if the data type does not need to be known, accept Type specification not required and click Next. You do not need to follow the rest of these instructions.

   However, a structured data type is recommended. For example, if you want to map the connection output in a data mapper step then you must specify the data type. The data mapper cannot display fields for unstructured data.

   To specify the data type, click in the Select Type field and select one of the following as the schema type:

   • JSON schema is a document that describes the structure of JSON data. The document’s media type is application/schema+json.
   • JSON instance is a document that contains JSON data. The document’s media type is application/json.
   • XML schema is a document that describes the structure of XML data. The document’s file extension is .xsd.
   • XML instance is a document that contains XML data. The document’s file extension is .xml.
7. In the Definition input box, paste a definition that conforms to the schema type you selected. For example, if you select JSON schema then you would paste the content of a JSON schema file, which has a media type of application/schema+json.

8. In the Data Type Name field, enter a name that you choose for the data type. For example, if you are specifying a JSON schema for vendors then you might specify Vendor as the data type name.

   You will see this data type name when you are creating or editing an integration that uses the connection for which you are specifying this type. Fuse Online displays the type name in the integration visualization and in the data mapper.

9. In the Data Type Description field, provide information that helps you distinguish this type. This description appears in the data mapper when you hover over the step that processes this type.
10. Click Next.

Procedure

1. On the Choose a connection page, click the Amazon S3 connection that you want to use to finish the integration.

2. Select the action that you want the connection to perform:

   • Copy Object adds one or more objects to the bucket.

     To add one file to the bucket, you can enter its name in the File Name field.

     To add multiple files to the bucket, do not specify a file name. In this case, the action adds all objects that it obtains from the previous integration step(s).

     If you used the poll action to obtain multiple files and you specify a file name then the Copy Object action adds only the last file that was received from the poll action.

   • Delete Object deletes an object from the bucket. In the File Name field, specify the name of the object that you want to delete. If the specified file is not in the bucket, the integration continues with no error.
3. After you configure the chosen action, click Next to specify the action’s input type.

4. In the Select Type field, if the data type does not need to be known, accept Type specification not required and click Next. You do not need to follow the rest of these instructions.

   However, a structured data type is recommended. For example, if you want to map the connection input in a data mapper step then you must specify the data type. The data mapper cannot display fields for unstructured data.

   To specify the data type, click in the Select Type field and select one of the following as the schema type:

   • JSON schema is a document that describes the structure of JSON data. The document’s media type is application/schema+json.
   • JSON instance is a document that contains JSON data. The document’s media type is application/json.
   • XML schema is a document that describes the structure of XML data. The document’s file extension is .xsd.
   • XML instance is a document that contains XML data. The document’s file extension is .xml.
5. In the Definition input box, paste a definition that conforms to the schema type you selected. For example, if you select JSON schema then you would paste the content of a JSON schema file, which has a media type of application/schema+json.

6. In the Data Type Name field, enter a name that you choose for the data type. For example, if you are specifying a JSON schema for vendors then you might specify Vendor as the data type name.

   You will see this data type name when you are creating or editing an integration that uses the connection for which you are specifying this type. Fuse Online displays the type name in the integration visualization and in the data mapper.

7. In the Data Type Description field, provide information that helps you distinguish this type. This description appears in the data mapper when you hover over the step that processes this type.
8. Click Next.

Procedure

1. In the flow visualization, click the plus sign where you want to add an Amazon S3 connection.
2. Click the Amazon S3 connection that you want to use in the middle of a flow.

3. Select the action that you want the connection to perform:

   • Copy Object adds one or more objects to the bucket.

     To add one file to the bucket, you can enter its name in the File Name field.

     To add multiple files to the bucket, do not specify a file name. In this case, the action adds all objects that it obtains from the previous integration step(s).

     If you used the poll action to obtain multiple files and you specify a file name then the Copy Object action adds only the last file that was received from the poll action.

   • Delete Object deletes an object from the bucket. In the File Name field, specify the name of the object that you want to delete. If the specified file is not in the bucket, the integration continues with no error.
4. After you configure the chosen action, click Next to specify the action’s input type.

5. In the Select Type field, if the data type does not need to be known, accept Type specification not required and click Next. You do not need to follow the rest of these instructions.

   However, a structured data type is recommended. For example, if you want to map the connection input in a data mapper step then you must specify the data type. The data mapper cannot display fields for unstructured data.

   To specify the data type, click in the Select Type field and select one of the following as the schema type:

   • JSON schema is a document that describes the structure of JSON data. The document’s media type is application/schema+json.
   • JSON instance is a document that contains JSON data. The document’s media type is application/json.
   • XML schema is a document that describes the structure of XML data. The document’s file extension is .xsd.
   • XML instance is a document that contains XML data. The document’s file extension is .xml.
6. In the Definition input box, paste a definition that conforms to the schema type you selected. For example, if you select JSON schema then you would paste the content of a JSON schema file, which has a media type of application/schema+json.

7. In the Data Type Name field, enter a name that you choose for the data type. For example, if you are specifying a JSON schema for vendors then you might specify Vendor as the data type name.

   You will see this data type name when you are creating or editing an integration that uses the connection for which you are specifying this type. Fuse Online displays the type name in the integration visualization and in the data mapper.

8. In the Data Type Description field, provide information that helps you distinguish this type. This description appears in the data mapper when you hover over the step that processes this type.
9. Click Next.