Chapter 6. Persistent Messaging

Abstract

This chapter outlines the tuning techniques that can be used to optimize the performance of a persistent broker. Hence, the tuning techniques in this chapter are focused mainly on the interaction between the broker and its message store.

6.1. Serializing to Disk

KahaDB message store

KahaDB is the recommended message store to use with Red Hat JBoss A-MQ in order to achieve maximum performance. The KahaDB supports several options that you can customize to obtain optimum performance.

Synchronous dispatch through a persistent broker

Figure 6.1, “Synchronous Dispatch through a Persistent Broker” gives an overview of the sequence of steps for a message dispatched synchronously through a persistent broker.

Figure 6.1. Synchronous Dispatch through a Persistent Broker

Synchronous Dispatch through a Persistent Broker
After receiving a message from a producer, the broker dispatches the messages to the consumers, as follows:
  1. The broker pushes the message into the message store. Assuming that the enableJournalDiskSyncs option is true, the message store also writes the message to disk, before the broker proceeds.
  2. The broker now sends the message to all of the interested consumers (but does not wait for consumer acknowledgments). For topics, the broker dispatches the message immediately, while for queues, the broker adds the message to a destination cursor.
  3. The broker then sends a receipt back to the producer. The receipt can thus be sent back before the consumers have finished acknowledging messages (in the case of topic messages, consumer acknowledgments are usually not required anyway).

Concurrent store and dispatch

To speed up the performance of the broker, you can enable the concurrent store and dispatch optimization, which allows storing the message and sending to the consumer to proceed concurrently.
Note
Concurrent store and dispatch is enabled, by default, for queues.
Figure 6.1, “Synchronous Dispatch through a Persistent Broker” gives an overview of message dispatch when the concurrent store and dispatch optimization is enabled.

Figure 6.2. Concurrent Store and Dispatch

Concurrent Store and Dispatch
After receiving a message from a producer, the broker dispatches the messages to the consumers, as follows:
  1. The broker pushes the message onto the message store and, concurrently, sends the message to all of the interested consumers. After sending the message to the consumers, the broker then sends a receipt back to the producer, without waiting for consumer acknowledgments or for the message store to synchronize to disk.
  2. As soon as the broker receives acknowledgments from all the consumers, the broker removes the message from the message store. Because consumers typically acknowledge messages faster than a message store can write them to disk, this often means that write to disk is optimized away entirely. That is, the message is removed from the message store before it is ever physically written to disk.
One drawback of concurrent store and dispatch is that it does reduce reliability.

Configuring concurrent store and dispatch

The concurrent store and dispatch feature can be enabled separately for queues and topics, by setting the concurrentStoreAndDispatchQueues flag and the concurrentStoreAndDispatchTopics flag. By default, it is enabled for queues, but disabled for topics. To enable concurrent store and dispatch for both queues and topics, configure the kahaDB element in the broker configuration as follows: 
<broker brokerName="broker" persistent="true" useShutdownHook="false">
  ...
  <persistenceAdapter>
    <kahaDB directory="activemq-data"
            journalMaxFileLength="32mb"
            concurrentStoreAndDispatchQueues="true"
            concurrentStoreAndDispatchTopics="true"
            />
  </persistenceAdapter>
</broker>

Reducing memory footprint of pending messages

After a queue message is written to persistent storage, a copy of the message remains in memory, pending dispatch to a consumer. If the relevant consumer is very slow, however, this can lead to a build-up of messages in the broker and, in some cases, can lead to an out-of-memory error. If you observe this problem in your broker, you can enable an option to reduce the memory footprint of the pending messages; but you should note that this option is not compatible with concurrent store and dispatch.
To reduce the memory footprint of pending queue messages, define a destination policy for the relevant queues, enabling the reduceMemoryFootprint option, as follows: 
<broker ... >
  ...
  <destinationPolicy>
    <policyMap>
      <policyEntries>
        <policyEntry queue=">" reduceMemoryFootprint="true" />
      </policyEntries>
    </policyMap>
  </destinationPolicy>
  ...
</broker>
When the reduceMemoryFootprint option is enabled, a message's marshalled content is cleared immediately after the message is written to persistent storage. This results in approximately a 50% reduction in the amount of memory occupied by the pending messages.