Red Hat Training

A Red Hat training course is available for Red Hat AMQ

Chapter 16. Message Grouping

Message groups are sets of messages that have the following characteristics:

  • Messages in a message group share the same group ID, that is, they have same group identifier property. For JMS messages, the property is JMSXGroupID.
  • Messages in a message group are always consumed by the same consumer, even if there are many consumers on a queue. Another consumer is chosen to receive a message group if the original consumer closes.

Message groups are useful when you want all messages for a certain value of the property to be processed serially by the same consumer. For example, you may want orders for any particular stock purchase to be processed serially by the same consumer. To do this you could create a pool of consumers, then set the stock name as the value of the message property. This ensures that all messages for a particular stock are always processed by the same consumer.

Note

Grouped messages might impact the concurrent processing of non-grouped messages due to the underlying FIFO semantics of a queue. For example, if there is a chunk of 100 grouped messages at the head of a queue followed by 1,000 non-grouped messages, all the grouped messages are sent to the appropriate client before any of the non-grouped messages are consumed. The functional impact in this scenario is a temporary suspension of concurrent message processing while all the grouped messages are processed. Keep this potential performance bottleneck in mind when determining the size of your message groups. Consider whether to isolate your grouped messages from your non-grouped messages.

16.1. Client-Side Message Grouping

The examples below show how to use message grouping using Core JMS clients.

Procedure

  • Set the group ID.

    • If you are using JNDI to establish a JMS connection factory for your JMS client, add the groupID parameter and supply a value. All messages sent using this connection factory have the property JMSXGroupID set to the specified value. 

      java.naming.factory.initial=org.apache.activemq.artemis.jndi.ActiveMQInitialContextFactory
connectionFactory.myConnectionFactory=tcp://localhost:61616?groupID=MyGroup

    • If you are not using JNDI, set the JMSXGroupID property using the setStringProperty() method. 

       Message message = new TextMessage();
 message.setStringProperty("JMSXGroupID", "MyGroup");
 producer.send(message);

Related Information

See mesagge-group and message-group2 under INSTALL_DIR/examples/features/standard for working examples of how message groups are configured and used.

16.2. Automatic Message Grouping

Instead of supplying a group ID yourself, you can have the ID automatically generated for you. Messages grouped in this way are still processed serially by a single consumer.

Procedure

The examples below show how to enable message grouping using Core JMS clients.

  • Enable automatic generation of the group ID.

    • If you are using a JNDI context environment to instantiate your JMS connection factory, add the autogroup=true name-value pair to the query string of the connection URL. 

      java.naming.factory.initial=org.apache.activemq.artemis.jndi.ActiveMQInitialContextFactory
connectionFactory.myConnectionFactory=tcp://localhost:61616?autoGroup=true

    • If you are not using JNDI, set autogroup to true on the ActiveMQConnectonFactory. 

      ActiveMQConnectionFactory cf = ActiveMQJMSClient.createConnectionFactoryWithoutHA(...);
cf.setAutoGroup(true);

16.3. Clustered Message Grouping

Using message groups in a cluster is a complex undertaking. Messages with a particular group ID can arrive on any broker, so each broker needs to know which group IDs are bound to which consumer on which broker. Each clustered broker therefore uses a grouping handler to manage the complexity of routing of grouped messages. There are two types of grouping handlers: Local and Remote. Each cluster should choose one broker to have a local grouping handler. All the other brokers use remote handlers.

Specifically, a route is initially proposed by the broker that received the message. A suitable route follows the normal clustered routing conditions, round robin for example. If the proposal is accepted by the grouping handlers, the broker routes messages to this queue from that point on. If the route is rejected, an alternative is offered, and the broker again routes to that queue indefinitely. Other brokers also route to the queue chosen at proposal time. Once the message arrives at the queue, the message is pinned to a consumer on that queue.

Configuration for a grouping handler includes three important elements, as shown in the example below. 

<grouping-handler name="my-grouping-handler">
   <type>LOCAL</type> 1
   <address>jms</address> 2
   <timeout>100000</timeout> 3
</grouping-handler>
1
The type element determines whether the broker uses a LOCAL or REMOTE handler.
2
The address element references a pre-existing cluster connection. See Configuring a Cluster Connection in the chapter on clustering for more information.
3
The timeout element sets how long to wait for a decision to be made, an exception is thrown during the send if this timeout is reached, ensuring strict message ordering. The default is 5000 milliseconds, or 5 seconds.

Procedure

You configure clustered brokers to use grouping handlers by adding configuration to the BROKER_INSTACE_DIR/etc/broker.xml file.

  1. Choose a broker from the cluster to be the one to use a local handler and add the following configuration. 

    <grouping-handler name="my-grouping-handler">
   <type>LOCAL</type>
   <address>jms</address>
   <timeout>10000</timeout>
</grouping-handler>

  2. Configure the other brokers to use a remote handler. 

    <grouping-handler name="my-grouping-handler">
   <type>REMOTE</type>
   <address>jms</address>
   <timeout>5000</timeout>
</grouping-handler>

Related Information

See clustered-grouping under INSTALL_DIR/examples/features/clustered for a working example of how message groups are used and configured within a cluster of brokers.

Clustered Message Grouping Best Practices

Some best practices should be followed when using clustered grouping:

  • There is a single point of failure when using only one local handler. To avoid this, create a slave broker as backup, using the same configuration for the local handler as the master.
  • Distribute consumers evenly across brokers. This is only an issue if you are creating and closing consumers regularly. Since messages are always routed to the same queue, removing a consumer from a queue may leave it with no consumers.
  • Use durable queues when possible. If you remove a queue after a message group is bound to it, other brokers may still try to route messages to it. Avoid this situation by making sure that the queue is deleted by the session that sent the messages. Deleting the queue in this way forces a new routing proposal. Alternatively, you could start using a different group ID.
  • When using a timeout, set the value for the remote handlers to at least half of the value of the local handler.