18.5. Paging

18.5.1. About Paging

HornetQ supports many message queues with each queue containing millions of messages. The HornetQ server runs with limited memory thereby making it difficult to store all message queues in memory at one time.
Paging is a mechanism used by the HornetQ server to transparently page messages in and out of memory on need basis in order to accomodate large message queues in a limited memory.
HornetQ starts paging messages to disk, when the size of messages in memory for a particular address exceeds the maximum configured message size.

Note

HornetQ paging is enabled by default.
Report a bug

18.5.2. Page Files

There is an individual folder for each address on the file system which stores messages in multiple files. These files which store the messages are called page files. Each file contains messages up to the maximum configured message size (page-size-bytes).
The system navigates the page files as needed and removes the page files as soon as all messages in the page were received by client.

Note

If a consumer has a message selector to read messages from queue, only the messages in memory that match the selector are delivered to the consumer. When the consumer acknowledges delivery of these messages, new messages are depaged and loaded into memory. For performance reasons, HornetQ does not scan paged messages to verify if they match the consumer's message selector and there is no confirmation that new depaged messages match the consumer's message selector. There may be messages that match consumer's selector on disk in page files but HornetQ does not load them into memory until another consumer reads the messages in memory and provides free space. If the free space is not available, the consumer with selector may not receive any new messages.
Report a bug

18.5.3. Configuration of Paging Folder

Global paging parameters are specified in server configuration files (standalone.xml and domain.xml). You can configure the location of the paging directory/folder by using the paging-directory parameter: 
<hornetq-server>
 ...
 <paging-directory>/location/paging-directory</paging-directory>
 ...
</hornetq-server>
The paging-directory parameter is used to specify a location/folder to store the page files. HornetQ creates one folder for each paging address in this paging directory. The page files are stored in these folders.
The default paging directory is EAP_HOME/standalone/data/messagingpaging (standalone mode) and EAP_HOME/domain/servers/SERVERNAME/data/messagingpaging (domain mode).
Report a bug

18.5.4. Paging Mode

When messages delivered to an address exceed the configured size, that address goes into "page/paging mode".

Note

Paging is done individually per address. If you configure a max-size-bytes for an address, it means each matching address will have a maximum size that you specified. However it does not mean that the total overall size of all matching addresses is limited to max-size-bytes.
Even with page mode, the server may crash due to an out-of-memory error. HornetQ keeps a reference to each page file on the disk. In a situation with millions of page files, HornetQ can face memory exhaustion. To minimize this risk, it is important to set the attribute page-size-bytes to a suitable value. You must configure the memory for your JBoss EAP 6 server higher than (number of destinations)*(max-size-bytes), otherwise an out-of-memory error can occur.
You can configure the maximum size in bytes (max-size-bytes) for an address in server configuration files (standalone.xml and domain.xml): 
<address-settings>
   <address-setting match="jms.someaddress">
      <max-size-bytes>104857600</max-size-bytes>
      <page-size-bytes>10485760</page-size-bytes>
      <address-full-policy>PAGE</address-full-policy>
   </address-setting>
</address-settings>
The following table describes the parameters on the address settings:

Table 18.4. Paging Address Settings

Element Default Value Description
max-size-bytes 10485760
This is used to specify the maximum memory size the address can have before entering nto paging mode
page-size-bytes 2097152
This is used to specify the size of each page file used on the paging system.
address-full-policy PAGE
This value of this attribute is used for paging decisions. You can set either of these values for this attribute: PAGE: To enable paging and page messages beyond the set limit to disk, DROP: To silently drop messages which exceed the set limit, FAIL: To drop messages and send an exception to client message producers, BLOCK: To block client message producers when they send messages beyond the set limit
page-max-cache-size 5
The system will keep page files up to page-max-cache-size in memory to optimize Input/Output during paging navigation

Important

If you don't want to page messages when the maximum size is reached, you may choose to configure an address in order to simply drop messages, drop messages with an exception on client side or block producers from sending further messages, by setting the address-full-policy to DROP, FAIL and BLOCK respectively. In the default configuration, all addresses are configured to page messages after an address reaches max-size-bytes.
Addresses with Multiple Queues

When a message is routed to an address that has multiplte queues bound to it, there is only a single copy of the message in memory. Each queue only handles a reference to this original copy of the message. Thus the memory is freed up only when all the queues referencing the original message, have delivered the message.

Note

A single lazy queue/subscription can reduce the Input/Output performance of the entire address as all the queues will have messages being sent through an extra storage on the paging system.
Report a bug