Chapter 39. Azure Storage Blob Service Component

Available as of Camel version 2.19

The Azure Blob component supports storing and retrieving the blobs to/from Azure Storage Blob service.

Prerequisites

You must have a valid Windows Azure Storage account. More information is available at Azure Documentation Portal.

39.1. URI Format

azure-blob://accountName/containerName[/blobName][?options]

In most cases a blobName is required and the blob will be created if it does not already exist.
You can append query options to the URI in the following format: ?options=value&option2=value&…​

For example, to download a blob content from the public block blob blockBlob located on the container1 in the camelazure storage account, use the following snippet:

from("azure-blob:camelazure/container1/blockBlob").
to("file://blobdirectory");

39.2. URI Options

The Azure Storage Blob Service component has no options.

The Azure Storage Blob Service endpoint is configured using URI syntax:

azure-blob:containerOrBlobUri

with the following path and query parameters:

39.2.1. Path Parameters (1 parameter)

NameDescriptionDefaultType

containerOrBlobUri

Required: Container or Blob compact URI.

 

String

39.2.2. Query Parameters (36 parameters)

NameDescriptionDefaultType

azureBlobClient (common)

The blob service client.

 

CloudBlob

blobOffset (common)

Set the blob offset for the upload or download operations, default is 0.

0

Long

blobType (common)

Set a blob type, 'blockblob' is default.

blockblob

BlobType

closeStreamAfterRead (common)

Close the stream after read or keep it open, default is true.

true

boolean

credentials (common)

Set the storage credentials, required in most cases.

 

StorageCredentials

credentialsAccountKey (common)

Set the storage account key used during authentication phase.

 

String

credentialsAccountName (common)

Set the storage account name used during authentication phase.

 

String

dataLength (common)

Set the data length for the download or page blob upload operations.

 

Long

fileDir (common)

Set the file directory where the downloaded blobs will be saved.

 

String

publicForRead (common)

Storage resources can be public for reading their content, if this property is enabled then the credentials do not have to be set.

false

boolean

streamReadSize (common)

Set the minimum read size in bytes when reading the blob content.

 

int

bridgeErrorHandler (consumer)

Allows for bridging the consumer to the Camel routing Error Handler. This means any exceptions that occurred (for example, while the consumer was trying to pick up incoming messages), will now be processed as a message and handled by the routing Error Handler. By default, the consumer will use the org.apache.camel.spi.ExceptionHandler to deal with exceptions, which will be logged at WARN or ERROR level and ignored.

false

boolean

sendEmptyMessageWhenIdle (consumer)

If the polling consumer did not poll any files, you can enable this option to send an empty message (no body) instead.

false

boolean

exceptionHandler (consumer)

To let the consumer use a custom ExceptionHandler. Note that if the option bridgeErrorHandler is enabled, this option is not in use. By default, the consumer will deal with exceptions, which will be logged at WARN or ERROR level and ignored.

 

ExceptionHandler

exchangePattern (consumer)

Sets the exchange pattern when the consumer creates an exchange.

 

ExchangePattern

pollStrategy (consumer)

A pluggable org.apache.camel.PollingConsumerPollingStrategy allowing you to provide your custom implementation to control error handling usually occurring during the poll operation before an Exchange has been created which will be routed in Camel.

 

PollingConsumerPoll Strategy

blobMetadata (producer)

Set the blob meta-data.

 

Map

blobPrefix (producer)

Set a prefix which can be used for listing the blobs.

 

String

closeStreamAfterWrite (producer)

Close the stream after write or keep it open, default is true.

true

boolean

operation (producer)

Blob service operation hint to the producer.

listBlobs

BlobServiceOperations

streamWriteSize (producer)

Set the size of the buffer for writing block and page blocks.

 

int

useFlatListing (producer)

Specify if the flat or hierarchical blob listing should be used.

true

boolean

synchronous (advanced)

Sets whether synchronous processing should be strictly used, or Camel is allowed to use asynchronous processing (if supported).

false

boolean

backoffErrorThreshold (scheduler)

The number of subsequent error polls (failed due to some error) that should happen before the backoffMultipler should kick-in.

 

int

backoffIdleThreshold (scheduler)

The number of subsequent idle polls that should happen before the backoffMultipler should kick-in.

 

int

backoffMultiplier (scheduler)

To let the scheduled polling consumer backoff if there has been a number of subsequent idles/errors in a row. The multiplier is then the number of polls that will be skipped before the next actual attempt happens again. When this option is in use then backoffIdleThreshold and/or backoffErrorThreshold must also be configured.

 

int

delay (scheduler)

Milliseconds before the next poll. You can also specify time values using units, such as 60s (60 seconds), 5m30s (5 minutes and 30 seconds), and 1h (1 hour).

500

long

greedy (scheduler)

If greedy is enabled, then the ScheduledPollConsumer will run immediately again, if the previous run polled 1 or more messages.

false

boolean

initialDelay (scheduler)

Milliseconds before the first poll starts. You can also specify time values using units, such as 60s (60 seconds), 5m30s (5 minutes and 30 seconds), and 1h (1 hour).

1000

long

runLoggingLevel (scheduler)

The consumer logs a start/complete log line when it polls. This option allows you to configure the logging level for that.

TRACE

LoggingLevel

scheduledExecutorService (scheduler)

Allows for configuring a custom/shared thread pool to use for the consumer. By default each consumer has its own single threaded thread pool.

 

ScheduledExecutor Service

scheduler (scheduler)

To use a cron scheduler from either camel-spring or camel-quartz2 component

none

ScheduledPollConsumer Scheduler

schedulerProperties (scheduler)

To configure additional properties when using a custom scheduler or any of the Quartz2, Spring based scheduler.

 

Map

startScheduler (scheduler)

Whether the scheduler should be auto started.

true

boolean

timeUnit (scheduler)

Time unit for initialDelay and delay options.

MILLISECONDS

TimeUnit

useFixedDelay (scheduler)

Controls if fixed delay or fixed rate is used. See ScheduledExecutorService in JDK for details.

true

boolean

Required Azure Storage Blob Service component options

You must provide the containerOrBlob name and the credentials if the private blob needs to be accessed.

39.3. Usage

39.3.1. Message headers set by the Azure Storage Blob Service producer

HeaderTypeDescription

CamelFileName

String

The file name for the downloaded blob content.

39.3.2. Message headers set by the Azure Storage Blob Service producer consumer

HeaderTypeDescription

CamelFileName

String

The file name for the downloaded blob content.

39.3.3. Azure Blob Service operations

Operations common to all block types

OperationDescription

getBlob

Get the content of the blob. You can restrict the output of this operation to a blob range.

deleteBlob

Delete the blob.

listBlobs

List the blobs.

Block blob operations

OperationDescription

updateBlockBlob

Put block blob content that either creates a new block blob or overwrites the existing block blob content.

uploadBlobBlocks

Upload block blob content, by first generating a sequence of blob blocks and then committing them to a blob. If you enable the message CommitBlockListLater property, you can execute the commit later with the commitBlobBlockList operation. You can later update individual block blobs.

commitBlobBlockList

Commit a sequence of blob blocks to the block list that you previously uploaded to the blob service (by using the updateBlockBlob operation with the message CommitBlockListLater property enabled).

getBlobBlockList

Get the block blob list.

Append blob operations

OperationDescription

createAppendBlob

Create an append block. By default, if the block already exists then it is not reset. Note that you can alternately create an append blob by enabling the message AppendBlobCreated property and using the updateAppendBlob operation.

updateAppendBlob

Append the new content to the blob. This operation also creates the blob if it does not already exist and if you enabled a message AppendBlobCreated property.

Page Block operations

OperationDescription

createPageBlob

Create a page block. By default, if the block already exists then it is not reset. Note that you can also create a page blob (and set its contents) by enabling a message PageBlobCreated property and by using the updatePageBlob operation.

updatePageBlob

Create a page block (unless you enable a message PageBlobCreated property and the identically named block already exists) and set the content of this blob.

resizePageBlob

Resize the page blob.

clearPageBlob

Clear the page blob.

getPageBlobRanges

Get the page blob page ranges.

39.3.4. Azure Blob Client configuration

If your Camel application is running behind a firewall or if you need more control over the Azure Blob Client configuration, you can create your own instance:

StorageCredentials credentials = new StorageCredentialsAccountAndKey(accountName, accessKey);
CloudBlob client = new CloudBlockBlob(URI.create("https://"
                    + accountName + ".blob.core.windows.net/" + containerName
                    + "/" + fileName), credentials);
registry.bind("azureBlobClient", client);

Refer to this instance in your Camel azure-blob component configuration:

from("azure-blob://" + accountName + "/" + containerName + "/" + fileName + "?azureBlobClient=#client")
.to("mock:result");

39.4. Dependencies

Maven users must add the following dependency to their pom.xml:

pom.xml

<dependency>
    <groupId>org.apache.camel</groupId>
    <artifactId>camel-azure</artifactId>
    <version>${camel-version}</version>
</dependency>

where ${camel-version} must be replaced by the actual version of Camel (2.19.0 or higher).

39.5. See Also

  • Configuring Camel
  • Component
  • Endpoint
  • Getting Started
  • Azure Component