Chapter 2. Connect to Amazon S3

1. Go to https://aws.amazon.com/s3/.
2. Hover over My Account, select AWS Management Console and sign in to the console with the AWS account that created the bucket that you want to access or with the account that you want the connection to use to create the bucket.
3. In the console, in the upper right, click the down arrow next to the user name and click My Security Credentials.
4. Expand Access Keys and click Create New Access Keys.
5. Follow the prompts to obtain the keys.

Procedure

1. In Fuse Online, in the left panel, click Connections to display any available connections.
2. In the upper right, click Create Connection to display Fuse Online connectors.
3. Click the Amazon S3 connector.
4. In the Access Key field, enter the Amazon S3 access key ID, provided by AWS, for the AWS account that created the bucket that you want this connection to access. If the bucket you want the connection to access does not already exist then when Fuse Online tries to start running the integration, it uses the AWS account associated with this access key to create the bucket. However, if the bucket already exists in some other AWS account, then the connection cannot create the bucket and the integration cannot start.
5. In the Bucket Name or Amazon Resource Name field, enter the name of the bucket that you want this connection to access or enter the bucket’s ARN. If the bucket does not already exist in either the AWS account being used or in any other AWS account, then the connection creates it.
6. In the Region field, select the AWS region in which the bucket resides. If the connection creates the bucket, then it creates it in the selected region.
7. In the Secret Key field, enter the Amazon S3 secret access key, provided by AWS, for the account that created, or will create, the bucket that you want this connection to access.
8. Click Validate. Fuse Online immediately tries to validate the connection and displays a message that indicates whether or not validation is successful. If validation fails, revise the configuration details as needed and try again.
9. When validation is successful, click Next.
10. In the Connection Name field, enter your choice of a name that helps you distinguish this connection from any other connections. For example, enter Obtain S3 Data.
11. In the Description field, optionally enter any information that is helpful to know about this connection. For example, enter Sample S3 connection that obtains data from the northeast bucket.
12. In the upper right, click Create to see that the connection you created is now available. If you entered the example name, you would see that Obtain S3 Data appears as a connection that you can choose to add to an integration.

Procedure

1. In the Fuse Online panel on the left, click Integrations.
2. Click Create Integration.
3. On the Choose a Start Connection page, click the Amazon S3 connection that you want to use to start the integration.

4. On the Choose an Action page, click the action that you want the connection to perform:

   • Get Object obtains a file from the bucket that the connection accesses. In the File Name field, enter the name of the file that you want to obtain. If the specified file is not in the bucket, it is a runtime error.

   • Poll an Amazon S3 Bucket periodically obtains files from the bucket that the connection accesses. To configure this action:

     1. In the Delay field, accept the default of 500 milliseconds as the time that elapses between polls. Or, to specify a different polling interval, enter a number and select its time unit.

     2. In the Maximum Objects to Retrieve field, enter the largest number of files that one poll operation can obtain. The default is 10.

        To have no limit on the number of files that can be obtained, specify 0 or a negative integer. When Maximum Objects to Retrieve is unlimited, the poll action obtains all files in the bucket.

        If the bucket contains more than the specified maximum number of files then the action obtains the files that were most recently modified or created.

     3. In the Prefix field, optionally specify a regular expression that evaluates to a string. If you specify a prefix then this action retrieves a file only when its name starts with that string.
     4. Indicate whether you want to Obtain files and then delete them from the bucket.
5. After you configure the action, click Done to specify the action’s output type.

6. In the Select Type field, if the data type does not need to be known, accept Type specification not required and then, at the bottom, click Done. You do not need to follow the rest of these instructions.

   Otherwise, select one of the following as the schema type:

   • JSON schema is a document that describes the structure of JSON data. The document’s media type is application/schema+json.
   • JSON instance is a document that contains JSON data. The document’s media type is application/json.
   • XML schema is a document that describes the structure of XML data. The document’s file extension is .xsd.
   • XML instance is a document that contains XML data. The document’s file extension is .xml.
7. In the Definition input box, paste a definition that conforms to the schema type you selected. For example, if you select JSON schema then you would paste the content of a JSON schema file, which has a media type of application/schema+json.

8. In the Data Type Name field, enter a name that you choose for the data type. For example, suppose you are specifying a JSON schema for vendors. You can specify Vendor as the data type name.

   You will see this data type name when you are creating or editing an integration that uses the connection for which you are specifying this type. Fuse Online displays the type name in the integration visualization panel and in the data mapper.

9. In the Data Type Description field, provide information that helps you distinguish this type. This description appears in the data mapper when you hover over the step that processes this type.
10. Click Done.

Procedure

1. Start creating the integration.
2. Add and configure the start connection.
3. On the Choose a Finish Connection page, click the Amazon S3 connection that you want to use to finish the integration.

4. Click the action that you want the connection to perform:

   1. Copy Object adds one or more objects to the bucket.

      To add one file to the bucket, you can enter its name in the File Name field.

      To add multiple files to the bucket, do not specify a file name. In this case, the action adds all objects that it obtains from the previous integration step(s).

      If you used the poll action to obtain multiple files and you specify a file name then the Copy Object action adds only the last file that was received from the poll action.

   2. Delete Object deletes an object from the bucket. In the File Name field, specify the name of the object that you want to delete. If the specified file is not in the bucket, the integration continues with no error.
5. After you configure the chosen action, click Next to specify the action’s input type.

6. In the Select Type field, if the data type does not need to be known, accept Type specification not required and then, at the bottom, click Done. You do not need to follow the rest of these instructions.

   Otherwise, select one of the following as the schema type:

   • JSON schema is a document that describes the structure of JSON data. The document’s media type is application/schema+json.
   • JSON instance is a document that contains JSON data. The document’s media type is application/json.
   • XML schema is a document that describes the structure of XML data. The document’s file extension is .xsd.
   • XML instance is a document that contains XML data. The document’s file extension is .xml.
7. In the Definition input box, paste a definition that conforms to the schema type you selected. For example, if you select JSON schema then you would paste the content of a JSON schema file, which has a media type of application/schema+json.

8. In the Data Type Name field, enter a name that you choose for the data type. For example, suppose you are specifying a JSON schema for vendors. You can specify Vendor as the data type name.

   You will see this data type name when you are creating or editing an integration that uses the connection for which you are specifying this type. Fuse Online displays the type name in the integration visualization panel and in the data mapper.

9. In the Data Type Description field, provide information that helps you distinguish this type. This description appears in the data mapper when you hover over the step that processes this type.
10. Click Done.

Procedure

1. Add the start and finish connections.
2. In the left panel, hover over the plus sign that is in the location where you want to add the Amazon S3 connection.
3. In the pop-up, click Add a Connection.
4. Click the Amazon S3 connection that you want to use as a middle connection in the integration.

5. Click the action that you want the connection to perform:

   1. Copy Object adds one or more objects to the bucket.

      To add one file to the bucket, you can enter its name in the File Name field.

      To add multiple files to the bucket, do not specify a file name. In this case, the action adds all objects that it obtains from the previous integration step(s).

      If you used the poll action to obtain multiple files and you specify a file name then the Copy Object action adds only the last file that was received from the poll action.

   2. Delete Object deletes an object from the bucket. In the File Name field, specify the name of the object that you want to delete. If the specified file is not in the bucket, the integration continues with no error.
6. After you configure the chosen action, click Next to specify the action’s input type.

7. In the Select Type field, if the data type does not need to be known, accept Type specification not required and then, at the bottom, click Done. You do not need to follow the rest of these instructions.

   Otherwise, select one of the following as the schema type:

   • JSON schema is a document that describes the structure of JSON data. The document’s media type is application/schema+json.
   • JSON instance is a document that contains JSON data. The document’s media type is application/json.
   • XML schema is a document that describes the structure of XML data. The document’s file extension is .xsd.
   • XML instance is a document that contains XML data. The document’s file extension is .xml.
8. In the Definition input box, paste a definition that conforms to the schema type you selected. For example, if you select JSON schema then you would paste the content of a JSON schema file, which has a media type of application/schema+json.

9. In the Data Type Name field, enter a name that you choose for the data type. For example, suppose you are specifying a JSON schema for vendors. You can specify Vendor as the data type name.

   You will see this data type name when you are creating or editing an integration that uses the connection for which you are specifying this type. Fuse Online displays the type name in the integration visualization panel and in the data mapper.

10. In the Data Type Description field, provide information that helps you distinguish this type. This description appears in the data mapper when you hover over the step that processes this type.
11. Click Done.