Chapter 34. Use Red Hat JBoss Data Grid with Amazon Web Services

34.1. The S3_PING JGroups Discovery Protocol

S3_PING is a discovery protocol that is ideal for use with Amazon’s Elastic Compute Cloud (EC2) because EC2 does not allow multicast and therefore MPING is not allowed.

Each EC2 instance adds a small file to an S3 data container, known as a bucket. Each instance then reads the files in the bucket to discover the other members of the cluster.

34.2. S3_PING Configuration Options

34.2.1. S3_PING Configuration Options

Red Hat JBoss Data Grid works with Amazon Web Services in two ways:

  • In Library mode, use JGroups' default-configs/default-jgroups-ec2.xml file (see default-jgroups-ec2.xml for details) or use the S3_PING protocol.
  • In Remote Client-Server mode, use JGroups' S3_PING protocol.

In Library and Remote Client-Server mode, there are three ways to configure the S3_PING protocol for clustering to work in Amazon AWS:

  • Use Private S3 Buckets. These buckets use Amazon AWS credentials.
  • Use Pre-Signed URLs. These pre-assigned URLs are assigned to buckets with private write and public read rights.
  • Use Public S3 Buckets. These buckets do not have any credentials.

34.2.2. Using Private S3 Buckets

This configuration requires access to a private bucket that can only be accessed with the appropriate AWS credentials. To confirm that the appropriate permissions are available, confirm that the user has the following permissions for the bucket:

  • List
  • Upload/Delete
  • View Permissions
  • Edit Permissions

Ensure that the S3_PING configuration includes the following properties:

  • the location where the bucket is found.
  • the access_key and secret_access_key properties for the AWS user.
Note

If a 403 error displays when using this configuration, verify that the properties have the correct values. If the problem persists, confirm that the system time in the EC2 node is correct. Amazon S3 rejects requests with a time stamp that is more than 15 minutes old compared to their server’s times for security purposes.

Start the Red Hat JBoss Data Grid Server with a Private Bucket

Run the following command from the top level of the server directory to start the Red Hat JBoss Data Grid server using a private S3 bucket:

bin/standalone.sh
  -c cloud.xml
  -Djboss.node.name={node_name}
  -Djboss.socket.binding.port-offset={port_offset}
  -Djboss.default.jgroups.stack=s3-private
  -Djgroups.s3.bucket={s3_bucket_name}
  -Djgroups.s3.access_key={access_key}
  -Djgroups.s3.secret_access_key={secret_access_key}
  1. Replace {node_name} with the server’s desired node name.
  2. Replace {port_offset} with the port offset. To use the default ports specify this as 0.
  3. Replace {s3_bucket_name} with the appropriate bucket name.
  4. Replace {access_key} with the user’s access key.
  5. Replace {secret_access_key} with the user’s secret access key.

34.2.3. Using Pre-Signed URLs

34.2.3.1. Using Pre-Signed URLs

For this configuration, create a publically readable bucket in S3 by setting the List permissions to Everyone to allow public read access. Each node in the cluster may share a pre-signed URL that points to a single file, allowing a single file to be shared across every node in the cluster. This URL points to a unique file and can include a folder path within the bucket.

Note

Longer paths will cause errors in S3_PING . For example, a path such as my_bucket/DemoCluster/jgroups.list works while a longer path such as my_bucket/Demo/Cluster/jgroups.list will not.

34.2.3.2. Generating Pre-Signed URLs

JGroup’s S3_PING class includes a utility method to generate pre-signed URLs. The last argument for this method is the time when the URL expires expressed in the number of seconds since the Unix epoch (January 1, 1970).

The syntax to generate a pre-signed URL is as follows:

String Url = S3_PING.generatePreSignedUrl("{access_key}", "{secret_access_key}", "{operation}", "{bucket_name}", "{path}", {seconds});
  1. Replace {operation} with either PUT or DELETE.
  2. Replace {access_key} with the user’s access key.
  3. Replace {secret_access_key} with the user’s secret access key.
  4. Replace {bucket_name} with the name of the bucket.
  5. Replace {path} with the desired path to the file within the bucket.
  6. Replace {seconds} with the number of seconds since the Unix epoch (January 1, 1970) that the path remains valid.

Generate a Pre-Signed URL

String putUrl = S3_PING.generatePreSignedUrl("access_key", "secret_access_key", "put", "my_bucket", "DemoCluster/jgroups.list", 1234567890);

Ensure that the S3_PING configuration includes the pre_signed_put_url and pre_signed_delete_url properties generated by the call to S3_PING.generatePreSignedUrl(). This configuration is more secure than one using private S3 buckets, because the AWS credentials are not stored on each node in the cluster

Note

If a pre-signed URL is entered into an XML file, then the & characters in the URL must be replaced with its XML entity (&).

34.2.3.3. Set Pre-Signed URLs Using the Command Line

To set the pre-signed URLs using the command line, use the following guidelines:

  • Enclose the URL in double quotation marks (" ").
  • In the URL, each occurrence of the ampersand (&) character must be escaped with a backslash (\)

Start a JBoss Data Grid Server with a Pre-Signed URL

bin/standalone.sh
  -c cloud.xml
  -Djboss.node.name={node_name}
  -Djboss.socket.binding.port-offset={port_offset}
  -Djboss.default.jgroups.stack=s3-presigned
  -Djgroups.s3.pre_signed_delete_url="http://{s3_bucket_name}.s3.amazonaws.com/jgroups.list?AWSAccessKeyId={access_key}\&Expires={expiration_time}\&Signature={signature}"
  -Djgroups.s3.pre_signed_put_url="http://{s3_bucket_name}.s3.amazonaws.com/jgroups.list?AWSAccessKeyId={access_key}\&Expires={expiration_time}\&Signature={signature}"

  1. Replace {node_name} with the server’s desired node name.
  2. Replace {port_offset} with the port offset. To use the default ports specify this as 0.
  3. Replace {s3_bucket_name} with the appropriate bucket name.
  4. Replace {access_key} with the user’s access key.
  5. Replace {expiration_time} with the values for the URL that are passed into the S3_PING.generatePreSignedUrl() method.
  6. Replace {signature} with the values generated by the S3_PING.generatePreSignedUrl() method.

34.2.4. Using Public S3 Buckets

This configuration involves an S3 bucket that has public read and write permissions, which means that Everyone has permissions to List , Upload/Delete , View Permissions , and Edit Permissions for the bucket.

The location property must be specified with the bucket name for this configuration. This configuration method is the least secure because any user who knows the name of the bucket can upload and store data in the bucket and the bucket creator’s account is charged for this data.

To start the Red Hat JBoss Data Grid server, use the following command:

bin/standalone.sh
  -c cloud.xml
  -Djboss.node.name={node_name}
  -Djboss.socket.binding.port-offset={port_offset}
  -Djboss.default.jgroups.stack=s3-public
  -Djgroups.s3.bucket={s3_bucket_name}
  1. Replace {node_name} with the server’s desired node name.
  2. Replace {port_offset} with the port offset. To use the default ports specify this as 0.
  3. Replace {s3_bucket_name} with the appropriate bucket name.

34.3. Utilizing an Elastic IP Address

While each node in the cluster is able to discover other nodes in the cluster using the S3_PING protocol, all network traffic is over the internal private network. It is recommended to configure an Elastic IP, or static IP, for a single node, so that a consistent address is available for configuring the cluster, such as through the Administration Console, across restarts. If no Elastic IP is configured each instance will contain a randomized IP address on its public network whenever it is started.

Full instructions for configuring an Elastic IP address may be found in Amazon’s Getting Started Guide.