Chapter 6. Connect to an FTP or SFTP server

In an integration, you can connect to an FTP or SFTP server to download or upload files. To do this, create an FTP or SFTP connection. You can then add this connection to any number of integrations. The following topics provide details:

6.1. Create an FTP or SFTP connection

In an integration, to download or upload files from/to an FTP or SFTP server, create a connection to that FTP or SFTP server. You can add the same connection to any number of integrations.

Prerequisite

You must know the host name of the server you want to connect to.

Procedure

  1. In Fuse Online, in the left panel, click Connections to display any available connections.
  2. In the upper right, click Create Connection to display Fuse Online connectors.
  3. To create a connection that uses File Transfer Protocol, click the FTP connector.

    Or, to create a connection that uses Secure File Transfer Protocol, click the SFTP connector.

  4. Configure the connection.

    • For an FTP connection:

      • Host is the only parameter that you must specify. Enter the host name of the server that you want to connect to. For example, if the name of your FTP host is FTP.WEST, then you would enter exactly that, FTP.WEST. Do not specify the protocol, for example, you should not specify something like this: ftp://FTP.WEST.
      • Port is required and has a default value of 21. This is the port that the FTP server is listening on.
      • All other parameters are either not required or have default values. The defaults are suitable for most integrations. Descriptions of these parameters are after this procedure.
    • For an SFTP connection, there must be values for these parameters:

      • Host is the host name of the SFTP server that you want to connect to. For example, if the name of your SFTP host is SFTP.EAST, then you would enter exactly that, SFTP.EAST. Do not specify the protocol, for example, you should not specify something like this: sftp://SFTP.EAST.
      • Port has a default of 22. This is the port that the SFTP server is listening on.
      • User name of the account that you want to use to access the SFTP server.
      • Password that is associated with that user name.
      • All other parameters have default values. The defaults are suitable for most integrations. Descriptions of these parameters are after this procedure.
  5. Click Validate. Fuse Online immediately tries to validate the connection and displays a message that indicates whether or not validation is successful. If validation fails, revise the configuration details as needed and try again.
  6. When validation is successful, click Next.
  7. In the Connection Name field, enter your choice of a name that helps you distinguish this connection from any other connections. For example, enter XLight FTP Server.
  8. In the Description field, optionally enter any information that is helpful to know about this connection.
  9. In the upper right, click Create to see that the connection you created is now available. If you entered the example name, you would see that XLight FTP Server appears as a connection that you can choose to add to an integration.

Descriptions of other parameters

  • Connect timeout defaults to 10000 milliseconds and indicates a maximum wait of 10 seconds to establish the connection. If 10 seconds elapse without a connection then Fuse Online waits for the number of milliseconds defined by Reconnect delay and then tries to reconnect.
  • Reconnect delay defaults to 1000 milliseconds and indicates the wait time before trying to reconnect again.
  • Maximum reconnect attempts defaults to 3. Fuse Online tries as many as 3 times to establish a connection.
  • Binary file transfer mode is used by default. Select No for ASCII transfer mode.
  • Passive connection mode defaults to Yes, which is usually the preferred mode. In passive mode, the client opens communication channels with the server as a way to avoid firewall issues. If you select No then active mode is used.
  • Disconnect from the server after use defaults to No. The connection remains established after it performs the action. Select Yes if you want to disconnect from the server after the connection performs the upload or download.
  • Data timeout defaults to 30000 milliseconds and indicates the maximum length of time that Fuse Online waits for a reply.

6.2. Obtain files from an FTP or SFTP server

To trigger integration execution when an FTP or SFTP connection finds the file(s) you are interested in, add an FTP or SFTP connection as an integration’s start connection.

Prerequisite

You created an FTP or SFTP connection.

Procedure

  1. In the Fuse Online panel on the left, click Integrations.
  2. Click Create Integration.
  3. On the Choose a Start Connection page, click the FTP or SFTP connection that you want to use to poll an FTP or SFTP server.
  4. On the Choose an Action page, click Download.
  5. In the File name expression field, if you are interested in a single file, then enter an Apache Camel Simple language expression that resolves to a file name. You cannot specify a regular expression. The connection polls (periodically checks) the server for this file and downloads it when it is found. Leave this field blank if you want to download more than one file.
  6. In the FTP directory field, enter the absolute or relative path of the server directory to poll. The connection watches this directory for any content and downloads all files when it finds any content.
  7. In the Milliseconds before polling starts field, accept the default of 1000 milliseconds or change the number of milliseconds.
  8. In the Milliseconds before the next poll field, accept the default of 500 milliseconds or change the number of milliseconds. This is the interval between polls.
  9. In the Delete after download field, accept the default of No or select Yes to download the file(s) and then delete it(them) from the server.
  10. Click Next to specify the action’s output type.
  11. In the Select Type field, if the data type does not need to be known, accept Type specification not required and then, at the bottom, click Done. You do not need to follow the rest of these instructions.

    Otherwise, select one of the following as the schema type:

    • JSON schema is a document that describes the structure of JSON data. The document’s media type is application/schema+json.
    • JSON instance is a document that contains JSON data. The document’s media type is application/json.
    • XML schema is a document that describes the structure of XML data. The document’s file extension is .xsd.
    • XML instance is a document that contains XML data. The document’s file extension is .xml.
  12. In the Definition input box, paste a definition that conforms to the schema type you selected. For example, if you select JSON schema then you would paste the content of a JSON schema file, which has a media type of application/schema+json.
  13. In the Data Type Name field, enter a name that you choose for the data type. For example, suppose you are specifying a JSON schema for vendors. You can specify Vendor as the data type name.

    You will see this data type name when you are creating or editing an integration that uses the connection for which you are specifying this type. Fuse Online displays the type name in the integration visualization panel and in the data mapper.

  14. In the Data Type Description field, provide information that helps you distinguish this type. This description appears in the data mapper when you hover over the step that processes this type.
  15. Click Done.

Result

The connection appears in the integration flow in the location where you added it.

6.3. Upload files to an FTP or SFTP server

To finish an integration by uploading files to an FTP or SFTP server, add an FTP or SFTP connection as the integration’s finish connection. You can also upload files to an FTP or SFTP server in the middle of an integration. To do this, add an FTP or SFTP connection as a middle connection.

Prerequisite

You created an FTP or SFTP connection.

Procedure

  1. Start creating the integration.
  2. Add and configure the start connection.
  3. On the Choose a Finish Connection page, do one of the following:

    • To finish an integration by uploading files, click the FTP or SFTP connection that you want to use.
    • To upload files in the middle of an integration, click the connection you want to use to finish the integration. Configure that connection. When the finish connection is part of the integration, in the left panel, hover over the plus sign where you want to add an FTP or SFTP connection and click Add a connection. Click the FTP or SFTP connection that you want to use to upload files in the middle of an integration.
  4. On the Choose an Action page, click Upload.
  5. In the File name expression field, if you want to upload only one particular file, then enter an Apache Camel Simple language expression that resolves to a file name. This is the name of the file that the action uploads to the server. You cannot specify a regular expression. To upload more than one file, leave this field blank.
  6. In the FTP directory field, enter the absolute or relative name of a server directory. If the File name expression field contains an expression, then the connection stores the specified file in this directory. If the File name expression field is blank, then the connection uploads to this directory all files that were received from the previous integration step.
  7. In the If file exists field, indicate the behavior when you are uploading a file that has the same path and name as a file that is on the server. Accept the default, Override, to overwrite the file that is on the server with the file that you are uploading. Or, select one of the following:

    • Append adds the content in the file being uploaded to the file that is on the server.
    • Fail throws GenericFileOperationException. The integration does not enter an error state.
    • Ignore does not upload the file. The integration continues running under the assumption that everything is okay.
    • Move renames one of the files.
    • TryRename uploads the file with a temporary name and renames the file to the desired name. This operation does not check for the existence of a file with the desired name, which makes the operation faster on most servers than when existence checks are done.
  8. In the Temporary file prefix while copying field, specify a string. The connection prepends this string to the name of a file while it is being uploaded. This enables the connection to write to a temporary file on the server and then rename that temporary file to have the correct name. This is useful for reducing locks when uploading very large files.
  9. In the Temporary file name while copying field, specify a string. The connection renames a file being uploaded to have this name while it is being uploaded. This enables the connection to write to a temporary file on the server and then rename that temporary file to have the correct name. This is useful for reducing locks when uploading very large files.
  10. Click Next to specify the action’s input type.
  11. In the Select Type field, if the data type does not need to be known, accept Type specification not required and then, at the bottom, click Done. You do not need to follow the rest of these instructions.

    Otherwise, select one of the following as the schema type:

    • JSON schema is a document that describes the structure of JSON data. The document’s media type is application/schema+json.
    • JSON instance is a document that contains JSON data. The document’s media type is application/json.
    • XML schema is a document that describes the structure of XML data. The document’s file extension is .xsd.
    • XML instance is a document that contains XML data. The document’s file extension is .xml.
  12. In the Definition input box, paste a definition that conforms to the schema type you selected. For example, if you select JSON schema then you would paste the content of a JSON schema file, which has a media type of application/schema+json.
  13. In the Data Type Name field, enter a name that you choose for the data type. For example, suppose you are specifying a JSON schema for vendors. You can specify Vendor as the data type name.

    You will see this data type name when you are creating or editing an integration that uses the connection for which you are specifying this type. Fuse Online displays the type name in the integration visualization panel and in the data mapper.

  14. In the Data Type Description field, provide information that helps you distinguish this type. This description appears in the data mapper when you hover over the step that processes this type.
  15. Click Done.

Result

The connection appears in the integration flow in the location where you added it.