36.3. Server-Side Message Load Balancing

If cluster connections are defined between nodes of a cluster, then HornetQ will load balance messages arriving at a particular node from a client.
Let us take a simple example of a cluster of four nodes A, B, C, and D arranged in a symmetric cluster (described in Section 36.7.1, “Symmetric cluster”). A queue called OrderQueue is deployed on each node of the cluster.
A client Ca is connected to node A, sending orders to the server. Also, order processor clients Pa, Pb, Pc, and Pd are connected to each of the nodes A, B, C, D. If no cluster connection was defined on node A, as order messages arrive on node A they will all end up in the OrderQueue on node A, so they will get consumed by the order processor client attached to node A, Pa.
If a cluster connection on node A is defined, as ordered messages arrive on node A, they are distributed in a round-robin fashion between all the nodes of the cluster, instead of all of them going into the local OrderQueue instance. The messages are forwarded from the receiving node to other nodes of the cluster. This is all done on the server side, the client maintains a single connection to node A.
For example, messages arriving on node A might be distributed in the following order between the nodes: B, D, C, A, B, D, C, A, B, D. The exact order depends on the order the nodes started up, but the algorithm used is round robin.

36.3.1. Configuring Cluster Connections

Cluster connections group servers into clusters so that messages can be load balanced between the nodes of the cluster. Typical cluster connections are defined in <JBOSS_HOME>/jboss-as/server/<PROFILE>/deploy/hornetq/hornetq-configuration.xml inside a cluster-connection element. There can be zero or more cluster connections defined per HornetQ server. 
<cluster-connections>
   <cluster-connection name="my-cluster">
      <address>jms</address>
      <retry-interval>500</retry-interval>
      <use-duplicate-detection>true</use-duplicate-detection>
      <forward-when-no-consumers>false</forward-when-no-consumers>
      <max-hops>1</max-hops>
      <discovery-group-ref discovery-group-name="my-discovery-group"/>
   </cluster-connection>
</cluster-connections>
In the above cluster connection all parameters have been explicitly specified. In practice you might use the defaults for some.
  • address. Each cluster connection only applies to messages sent to an address that starts with this value.
    In this case, this cluster connection will load balance messages sent to address that start with jms. This cluster connection, applies to all JMS queue and topic subscriptions since they map to core queues that start with the substring "jms".
    The address can be any value and you can have many cluster connections with different values of address, simultaneously balancing messages for those addresses, potentially to different clusters of servers. By having multiple cluster connections on different addresses a single HornetQ Server can effectively take part in multiple clusters simultaneously.
    Be careful not to have multiple cluster connections with overlapping values of address, (for example, "europe" and "europe.news") since this could result in the same messages being distributed between more than one cluster connection, possibly resulting in duplicate deliveries.
    This parameter is mandatory.
  • discovery-group-ref. This parameter determines which discovery group is used to obtain the list of other servers in the cluster to which this cluster connection will make connections.
  • forward-when-no-consumers. This parameter determines whether messages will be distributed round robin between other nodes of the cluster irrespective of whether there are matching or indeed any consumers on other nodes.
    If this is set to true then each incoming message will be processed in a round robin style even though the same queues on the other nodes of the cluster may have no consumers at all, or they may have consumers that have non matching message filters (selectors). Note that HornetQ will not forward messages to other nodes if there are no queues of the same name on the other nodes, even if this parameter is set to true.
    If this is set to false then HornetQ will only forward messages to other nodes of the cluster if the address to which they are being forwarded has queues which have consumers, and if those consumers have message filters (selectors) at least one of those selectors must match the message.
    This parameter is optional and the default value is false.
  • max-hops. When a cluster connection decides the set of nodes to which it might load balance a message, those nodes do not have to be directly connected to it via a cluster connection. HornetQ can be configured to also load balance messages to nodes which might be connected to it only indirectly with other HornetQ servers as intermediates in a chain.
    This allows HornetQ to be configured in more complex topologies and still provide message load balancing. This is covered later in this chapter.
    The default value for this parameter is 1, which means messages are only load balanced to other HornetQ serves which are directly connected to this server. This parameter is optional.
  • min-large-message-size. This parameter determines the size threshold above which a message will be split into multiple packages when sent over the cluster. This parameter is optional and the default is 100 kB.
  • reconnect-attempts. This parameter determines the number of times the system will try to connect a node on the cluster. If the max-retry is achieved this node will be considered permanently down and the system will stop routing messages to it. This parameter is optional and the default is -1 (infinite retries).
  • retry-interval. Internally, cluster connections cause bridges to be created between the nodes of the cluster. If the cluster connection is created and the target node has not been started, or say, is being rebooted, then the cluster connections from other nodes will retry connecting to the target until it comes back up, in the same way as a bridge does.
    This parameter determines the interval in milliseconds between retry attempts. It has the same meaning as the retry-interval on a bridge (as described in Chapter 34, Core Bridges).
    This parameter is optional and its default value is 500 milliseconds.
  • use-duplicate-detection. Internally cluster connections use bridges to link the nodes, and bridges can be configured to add a duplicate id property in each message that is forwarded. If the target node of the bridge crashes and then recovers, messages might be resent from the source node. By enabling duplicate detection any duplicate messages will be filtered out and ignored on receipt at the target node.
    This parameter has the same meaning as use-duplicate-detection on a bridge. For more information on duplicate detection, refer to Chapter 35, Duplicate Message Detection.
    This parameter is optional and has a default value of true.