Show Table of Contents
18.5. Paging
18.5.1. About Paging
HornetQ supports many message queues with each queue containing millions of messages. The HornetQ server runs with limited memory thereby making it difficult to store all message queues in memory at one time.
Paging is a mechanism used by the HornetQ server to transparently page messages in and out of memory on need basis in order to accomodate large message queues in a limited memory.
HornetQ starts paging messages to disk, when the size of messages in memory for a particular address exceeds the maximum configured message size.
Note
HornetQ paging is enabled by default.
18.5.2. Page Files
There is an individual folder for each address on the file system which stores messages in multiple files. These files which store the messages are called page files. Each file contains messages up to the maximum configured message size (
page-size-bytes).
The system navigates the page files as needed and removes the page files as soon as all messages in the page were received by client.
Note
If a consumer has a message selector to read messages from queue, only the messages in memory that match the selector are delivered to the consumer. When the consumer acknowledges delivery of these messages, new messages are depaged and loaded into memory. For performance reasons, HornetQ does not scan paged messages to verify if they match the consumer's message selector and there is no confirmation that new depaged messages match the consumer's message selector. There may be messages that match consumer's selector on disk in page files but HornetQ does not load them into memory until another consumer reads the messages in memory and provides free space. If the free space is not available, the consumer with selector may not receive any new messages.
18.5.3. Configuration of Paging Folder
Global paging parameters are specified in server configuration files (standalone.xml and domain.xml). You can configure the location of the paging directory/folder by using the
paging-directory parameter:
<hornetq-server> ... <paging-directory>/location/paging-directory</paging-directory> ... </hornetq-server>The
paging-directory parameter is used to specify a location/folder to store the page files. HornetQ creates one folder for each paging address in this paging directory. The page files are stored in these folders.
The default paging directory is
EAP_HOME/standalone/data/messagingpaging (standalone mode) and EAP_HOME/domain/servers/SERVERNAME/data/messagingpaging (domain mode).
18.5.4. Paging Mode
When messages delivered to an address exceed the configured size, that address goes into "page/paging mode".
Note
Paging is done individually per address. If you configure a
max-size-bytes for an address, it means each matching address will have a maximum size that you specified. However it does not mean that the total overall size of all matching addresses is limited to max-size-bytes.
Even with
page mode, the server may crash due to an out-of-memory error. HornetQ keeps a reference to each page file on the disk. In a situation with millions of page files, HornetQ can face memory exhaustion. To minimize this risk, it is important to set the attribute page-size-bytes to a suitable value. You must configure the memory for your JBoss EAP 6 server higher than (number of destinations)*(max-size-bytes), otherwise an out-of-memory error can occur.
You can configure the maximum size in bytes (
max-size-bytes) for an address in server configuration files (standalone.xml and domain.xml):
<address-settings>
<address-setting match="jms.someaddress">
<max-size-bytes>104857600</max-size-bytes>
<page-size-bytes>10485760</page-size-bytes>
<address-full-policy>PAGE</address-full-policy>
</address-setting>
</address-settings>
The following table describes the parameters on the address settings:
Table 18.4. Paging Address Settings
| Element | Default Value | Description |
|---|---|---|
| max-size-bytes | 10485760 |
This is used to specify the maximum memory size the address can have before entering nto paging mode
|
| page-size-bytes | 2097152 |
This is used to specify the size of each page file used on the paging system.
|
| address-full-policy | PAGE |
This value of this attribute is used for paging decisions. You can set either of these values for this attribute:
PAGE: To enable paging and page messages beyond the set limit to disk, DROP: To silently drop messages which exceed the set limit, FAIL: To drop messages and send an exception to client message producers, BLOCK: To block client message producers when they send messages beyond the set limit
|
| page-max-cache-size | 5 |
The system will keep page files up to
page-max-cache-size in memory to optimize Input/Output during paging navigation
|
Important
If you don't want to page messages when the maximum size is reached, you may choose to configure an address in order to simply drop messages, drop messages with an exception on client side or block producers from sending further messages, by setting the
address-full-policy to DROP, FAIL and BLOCK respectively. In the default configuration, all addresses are configured to page messages after an address reaches max-size-bytes.
Addresses with Multiple Queues
When a message is routed to an address that has multiplte queues bound to it, there is only a single copy of the message in memory. Each queue only handles a reference to this original copy of the message. Thus the memory is freed up only when all the queues referencing the original message, have delivered the message.
Note
A single lazy queue/subscription can reduce the Input/Output performance of the entire address as all the queues will have messages being sent through an extra storage on the paging system.