LibraryToggle FramesPrintFeedback

Prefetch Limit

Overview

Figure 4 illustrates the behavior of a broker, as it waits to receive acknowledgments for the messages it has already sent to a consumer.

Figure 4. Consumer Prefetch Limit

Consumer Prefetch Limit

If a consumer is slow to acknowledge messages, it can happen that the broker sends it another message before the previous message is acknowledged. If the consumer continues to be slow, moreover, the number of unacknowledged messages can grow continuously larger. The broker does not continue to send messages indefinitely in these circumstances. When the number of unacknowledged messages reaches a set limit—the prefetch limit—the server ceases sending new messages to the consumer. No more messages will be sent until the consumer starts sending back some acknowledgments.

Default prefetch limits

Different prefetch limits can be set for each consumer type. The default prefetch limits are as follows:

Queue consumer

Default prefetch limit is 1000.

If you are using a collection of consumers to distibute the workload (many consumers processing messages from the same queue), you typically want this limit to be small. If one consumer is allowed to accumulate a large number of unacknowledged messages, it could starve the other consumers of messages. Also, if the consumer fails, there would be a large number of messages unavailable for processing until the failed consumer is restored.

Queue browser

Default prefetch limit is 500.

Topic consumer

Default prefetch limit is 32766.

The default limit of 32766 is the largest value of a short and is the maximum possible value of the prefetch limit.

Durable topic subscriber

Default prefetch limit is 100.

You can typically improve the efficientcy of a consumer by increasing this prefetch limit.

Optimizing prefetch limits

Typically, it is a good idea to optimize queue consumers and durable topic subscribers as follows:

  • Queue consumers—if you have just a single consumer attached to a queue, you can leave the prefetch limit at a fairly large value. But if you are using a group of consumers to distribute the workload, it is usually better to restrict the prefetch limit to a very small number—for example, 0 or 1.

  • Durable topic subscribers—the efficiency of topic subscribers is generally improved by increasing the prefetch limit. Try increasing the limit to 1000.

How to set prefetch limits

You can set prefetch limits either on the broker or on the consumer and there are three levels of granularity to choose from, as follows:

Per broker

You can define the prefetch limits for all consumers that attach to a particular broker, by setting a destination policy on the broker. To set the destination policy, add a destinationPolicy element as a child of the broker element in the broker's configuration, as follows:

<broker ... >
  ...
  <destinationPolicy>
    <policyMap>
      <policyEntries>
        <policyEntry queue="queue.>" queuePrefetch=”1”/>
        <policyEntry topic="topic.>" topicPrefetch=”1000”/>
      </policyEntries>
    </policyMap>
  </destinationPolicy>
  ...
</broker>

In the preceding example, the queue prefetch limit for all queues whose names start with queue. is set to 1 (the > character is a wildcard symbol that matches one or more name segments); and the topic prefetch limit for all topics whose names start with topic. is set to 1000.

The following attributes of policyEntry are used to specify prefetch limits:

queuePrefetch

Specifies the queue consumer prefetch limit.

queueBrowserPrefetch

Specifies the queue browser prefetch limit.

topicPrefetch

Specifies the topic consumer prefetch limit.

durableTopicPrefetch

Specifies the durable topic subscriber prefetch limit.

Per connection factory

In a consumer, you can specify the prefetch limits on a connection factory by setting properties on an ActiveMQConnectionFactory instance. For example, the following code example shows how to specify the prefetch limits for all consumer types on a connection factory:

// Java
ActiveMQConnectionFactory factory = new ActiveMQConnectionFactory();

Properties props = new Properties();
props.setProperty("prefetchPolicy.queuePrefetch", "1000");
props.setProperty("prefetchPolicy.queueBrowserPrefetch", "500");
props.setProperty("prefetchPolicy.durableTopicPrefetch", "100");
props.setProperty("prefetchPolicy.topicPrefetch", "32766");

factory.setProperties(props);

Per destination

At the finest level of granularity, you can specify the prefetch limit on each destination instance that you create in a consumer. For example, to consume from the queue, TEST.QUEUE, with a prefetch limit of 10, create a MessageConsumer instance as follows:

// Java
Queue queue = 
  new ActiveMQQueue("TEST.QUEUE?consumer.prefetchSize=10");

MessageConsumer consumer = session.createConsumer(queue);
Comments powered by Disqus