Red Hat Training

A Red Hat training course is available for Red Hat AMQ

Chapter 9. Broker Clusters

You can connect brokers together to form a cluster. Broker clusters enable you to distribute message processing load and balance client connections. They also provide fault tolerance by increasing the number of brokers to which clients can connect.

9.1. Broker Clustering Changes

In AMQ Broker 7, broker networks are called broker clusters. The brokers in the cluster are connected by cluster connections (which reference connector elements). Members of a cluster can be configured to discover each other dynamically (using UDP or JGroups), or statically (by manually specifying a list of cluster members).

A cluster configuration is a required prerequisite for high-availability (HA). You must configure the cluster before you can configure HA, even if the cluster consists of only a single live broker.

You can configure broker clusters in many different topologies, though symmetric and chain clusters are the most common. Regardless of the topology, you can scale clusters up and down without message loss (as long as you have configured the broker to send its messages to another broker in the cluster).

Broker clusters distribute (and redistribute) messages differently than broker networks in AMQ 6. In AMQ 6, messages always arrived on a specific queue and were then pulled from one broker to another based on consumer interest. In AMQ Broker 7, queue definitions and consumers are shared across the cluster, and messages are routed across the cluster as they are received at the broker.

Important

Do not attempt to combine AMQ 6 brokers and AMQ Broker 7 brokers in the same cluster.

9.2. How Broker Clusters are Configured

You configure a broker cluster by creating a broker instance for each member of the cluster, and then adding the cluster settings to each broker instance.

Cluster settings consist of the following:

Discovery groups
For use with dynamic discovery, a discovery group defines how the broker instance discovers other members in the cluster. Discovery can use either UDP or JGroups.
Broadcast groups
For use with dynamic discovery, a broadcast group defines how the broker instance transmits cluster-related information to other members in the cluster. Broadcast can use either UDP or JGroups, but it must match its discovery groups counterpart.
Cluster connections
How the broker instance should connect to other members of the cluster. You can specify a discovery group or a static list of cluster members. You can also specify message redistribution and max hop properties.

9.2.1. Creating a Broker Cluster

This procedure demonstrates how to create a basic, two-broker cluster with static discovery.

Procedure

  1. Create the first broker instance by using the artemis create command.

    This example creates a new broker instance called broker1. 

    $ sudo INSTALL_DIR/bin/artemis create broker1 --user user --password pass --role amq

  2. Create a second broker instance for the second member of the cluster.

    For each additional broker instance, you should use the --port-offset parameter to avoid port collisions with the previous broker instances.

    This example creates a second broker instance called broker2. 

    $ sudo INSTALL_DIR/bin/artemis create broker2 --port-offset 100 --user user --password pass --role amq

  3. For the first broker instance, open the BROKER_INSTANCE_DIR/etc/broker.xml configuration file and add the cluster settings.

    For static discovery, you must add a connector and a static cluster connection. This example configures broker1 to connect to broker2. 

    <!-- Connectors -->
<connectors>
    <connector name="netty-connector">tcp://localhost:61616</connector>
    <!-- connector to broker2 -->
    <connector name="broker2-connector">tcp://localhost:61617</connector>
</connectors>

<!-- Clustering configuration -->
<cluster-connections>
    <cluster-connection name="my-cluster">
        <connector-ref>netty-connector</connector-ref>
        <retry-interval>500</retry-interval>
        <use-duplicate-detection>true</use-duplicate-detection>
        <message-load-balancing>STRICT</message-load-balancing>
        <max-hops>1</max-hops>
        <static-connectors>
            <connector-ref>broker2-connector</connector-ref>
        </static-connectors>
    </cluster-connection>
</cluster-connections>

  4. For the second broker instance, open the BROKER_INSTANCE_DIR/etc/broker.xml configuration file and add the cluster settings.

    This example configures broker2 to connect to broker1. 

    <!-- Connectors -->
<connectors>
    <connector name="netty-connector">tcp://localhost:61617</connector>
    <!-- connector to broker1 -->
    <connector name="broker1-connector">tcp://localhost:61616</connector>
</connectors>

<!-- Clustering configuration -->
<cluster-connections>
    <cluster-connection name="my-cluster">
        <connector-ref>netty-connector</connector-ref>
        <retry-interval>500</retry-interval>
        <use-duplicate-detection>true</use-duplicate-detection>
        <message-load-balancing>STRICT</message-load-balancing>
        <max-hops>1</max-hops>
        <static-connectors>
            <connector-ref>broker1-connector</connector-ref>
        </static-connectors>
    </cluster-connection>
</cluster-connections>

Related Information

  • For full details about creating broker clusters and configuring message redistribution and client load balancing, see Clustering in Using AMQ Broker.

9.2.2. Additional Broker Cluster Topologies

Broker clusters can be connected in many different topologies. In AMQ Broker 7, symmetric and chain clusters are the most common.

Example: Symmetric Cluster

In a full mesh topology, each broker is connected to every other broker in the cluster. This means that every broker in the cluster is no more than one hop away from every other broker.

This example uses dynamic discovery to enable the brokers in the cluster to discover each other. By setting max-hops to 1, each broker will connect to every other broker: 

<!-- Clustering configuration -->
<broadcast-groups>
    <broadcast-group name="my-broadcast-group">
        <group-address>${udp-address:231.7.7.7}</group-address>
        <group-port>9876</group-port>
        <broadcast-period>100</broadcast-period>
        <connector-ref>netty-connector</connector-ref>
    </broadcast-group>
</broadcast-groups>

<discovery-groups>
    <discovery-group name="my-discovery-group">
        <group-address>${udp-address:231.7.7.7}</group-address>
        <group-port>9876</group-port>
        <refresh-timeout>10000</refresh-timeout>
    </discovery-group>
</discovery-groups>

<cluster-connections>
    <cluster-connection name="my-cluster">
        <connector-ref>netty-connector</connector-ref>
        <retry-interval>500</retry-interval>
        <use-duplicate-detection>true</use-duplicate-detection>
        <message-load-balancing>ON_DEMAND</message-load-balancing>
        <max-hops>1</max-hops>
        <discovery-group-ref discovery-group-name="my-discovery-group"/>
    </cluster-connection>
</cluster-connections>

Example: Chain Cluster

In a chain cluster, the brokers form a linear "chain" with a broker on each end and all other brokers connecting to the previous and next brokers in the chain (for example, A→B→C).

This example uses static discovery to connect three brokers into a chain cluster. Each broker connects to the next broker in the chain, and max-hops is set to 2 to enable messages to flow through the full chain.

The first broker is configured like this: 

<connectors>
   <connector name="netty-connector">tcp://localhost:61616</connector>
   <!-- connector to broker2 -->
   <connector name="broker2-connector">tcp://localhost:61716</connector>
</connectors>


<cluster-connections>
   <cluster-connection name="my-cluster">
      <address>jms</address>
      <connector-ref>netty-connector</connector-ref>
      <retry-interval>500</retry-interval>
      <use-duplicate-detection>true</use-duplicate-detection>
      <message-load-balancing>STRICT</message-load-balancing>
      <max-hops>2</max-hops>
      <static-connectors allow-direct-connections-only="true">
         <connector-ref>broker2-connector</connector-ref>
      </static-connectors>
   </cluster-connection>
</cluster-connections>

The second broker is configured like this: 

<connectors>
   <connector name="netty-connector">tcp://localhost:61716</connector>
   <!-- connector to broker3 -->
   <connector name="broker3-connector">tcp://localhost:61816</connector>
</connectors>


<cluster-connections>
   <cluster-connection name="my-cluster">
      <address>jms</address>
      <connector-ref>netty-connector</connector-ref>
      <retry-interval>500</retry-interval>
      <use-duplicate-detection>true</use-duplicate-detection>
      <message-load-balancing>STRICT</message-load-balancing>
      <max-hops>1</max-hops>
      <static-connectors allow-direct-connections-only="true">
         <connector-ref>broker3-connector</connector-ref>
      </static-connectors>
   </cluster-connection>
</cluster-connections>

Finally, the third broker is configured like this: 

<connectors>
   <connector name="netty-connector">tcp://localhost:61816</connector>
</connectors>

<cluster-connections>
   <cluster-connection name="my-cluster">
      <address>jms</address>
      <connector-ref>netty-connector</connector-ref>
      <retry-interval>500</retry-interval>
      <use-duplicate-detection>true</use-duplicate-detection>
      <message-load-balancing>STRICT</message-load-balancing>
      <max-hops>0</max-hops>
   </cluster-connection>
</cluster-connections>

9.3. Broker Cluster Configuration Properties

The following table compares the broker network configuration properties in AMQ 6 to the equivalent cluster-connection properties in AMQ Broker 7:

To set…​In AMQ 6In AMQ Broker 7

Excluded destinations

excludedDestinations

No equivalent.

The number of hops that a message can make through the cluster

networkTTL

The default is 1, which means that a message can make just one hop to a neighboring broker.

<max-hops>

Sets this broker instance to load balance messages to brokers which might be connected to it indirectly with other brokers are intermediaries in a chain. The default is 1, which means that messages are distributed only to other brokers directly connected to this broker instance.

Replay messages when there are no consumers

replayWhenNoConsumers

No equivalent. However, you can set <redistribution-delay> to define the amount of time with no consumers (in milliseconds) after which messages should be redelivered as though arriving for the first time.

Whether to broadcast advisory messages for temporary destinations in the cluster

bridgeTempDestinations

The default is true. This property was typically used for temporary destinations created for request-reply messages. This would enable consumers of these messages to be connected to another broker in the network and still be able to send the reply to the temporary destination specified in the JMSReplyTo header.

No equivalent. In AMQ Broker 7, temporary destinations are never clustered.

The credentials to use to authenticate this broker with a remote broker

userNamepassword

<cluster-user><cluster-password>

Set the route priority for a connector

decreaseNetworkConsumerPriority

The default is false. If set to true, local consumers have a priority of 0, and network subscriptions have a priority of -5. In addition, the priority of a network subscription is reduced by 1 for every network hop that it traverses.

No equivalent.

Whether and how messages should be distributed between other brokers in the cluster

No equivalent.

<message-load-balancing>

This can be set to OFF (no load balancing), STRICT (forward messages to all brokers in the cluster that have a matching queue), or ON_DEMAND (forward messages only to brokers in the cluster that have active consumers or a matching selector). The default is ON_DEMAND.

Enable a cluster network connection to both produce and consume messages

duplex

By default, network connectors are unidirectional. However, you could set them to duplex to enable messages to flow in both directions. This was typically used for hub-and-spoke networks in which the hub was behind a firewall.

No equivalent. Cluster connections are unidirectional only. However, you can configure a pair of cluster connections between each broker, one from each end. For more information, see About Cluster Connections in Using AMQ Broker.