Chapter 6. Persisting Messages

This chapter describes how persistence works with AMQ Broker and how to configure it.

The broker ships with two persistence options:

  1. Journal-based

    The default. A highly performant option that writes messages to journals on the file system.

  2. JDBC-based

    Uses the broker’s JDBC Store to persist messages to a database of your choice.

Alternatively, you can also configure the broker for zero persistence.

The broker uses a different solution for persisting large messages outside the message journal. See Working with Large Messages for more information. The broker can also be configured to page messages to disk in low memory situations. See Paging Messages for more information.

Note

For current information regarding which databases and network file systems are supported see Red Hat AMQ 7 Supported Configurations on the Red Hat Customer Portal.

6.1. About Journal-based Persistence

A broker’s journal is a set of append only files on disk. Each file is pre-created to a fixed size and initially filled with padding. As messaging operations are performed on the broker, records are appended to end of the journal. Appending records allows the broker to minimize disk head movement and random access operations, which are typically the slowest operation on a disk. When one journal file is full, the broker uses a new one.

The journal file size is configurable, minimizing the number of disk cylinders used by each file. Modern disk topologies are complex, however, and the broker cannot control which cylinder(s) the file is mapped to. Journal file sizing therefore is not an exact science.

Other persistence-related features include:

  • A sophisticated file garbage collection algorithm that determines whether a particular journal file is still in use. If not, the file can be reclaimed and re-used.
  • A compaction algorithm that removes dead space from the journal and that compresses the data. This results in the journal using fewer files on disk.
  • Support for local transactions.
  • Support for XA transactions when using AMQ JMS clients.

The majority of the journal is written in Java. However, the interaction with the actual file system is abstracted, so you can use different, pluggable implementations. AMQ Broker ships with two implementations:

  • Java NIO.

    Uses the standard Java NIO to interface with the file system. This provides extremely good performance and runs on any platform with a Java 6 or later runtime.

  • Linux Asynchronous IO

    Uses a thin native wrapper to talk to the Linux asynchronous IO library (AIO). With AIO, the broker is called back after the data has made it to disk, avoiding explicit syncs altogether. By default the broker tries to use an AIO journal, and falls back to using NIO if AIO is not available.

    Using AIO typically provides even better performance than using Java NIO. For instructions on how to install libaio see Using an AIO journal.

Note

For current information regarding which network file systems are supported see Red Hat AMQ 7 Supported Configurations on the Red Hat Customer Portal.

6.1.1. Using AIO

The Java NIO journal is highly performant, but if you are running the broker using Linux Kernel 2.6 or later, Red Hat recommends using the AIO journal for better persistence performance. It is not possible to use the AIO journal with other operating systems or earlier versions of the Linux kernel.

To use the AIO journal you must install the libaio if it is not already installed.

Procedure

  • Use the yum command to install libaio, as in the example below:
yum install libaio

6.2. Configuring Journal-based Persistence

Persistence configuration is maintained in the file BROKER_INSTANCE_DIR/etc/broker.xml. The broker’s default configuration uses journal based persistence and includes the elements shown below.

<configuration>
  <core>
    ...
    <persistence-enabled>true</persistence-enabled>
    <journal-type>ASYNCIO</journal-type>
    <bindings-directory>./data/bindings</bindings-directory>
    <journal-directory>./data/journal</journal-directory>
    <journal-datasync>true</journal-datasync>
    <journal-min-files>2</journal-min-files>
    <journal-pool-files>-1</journal-pool-files>
    ...
  </core>
</configuration>
persistence-enabled
Specify whether to use the file-based journal for message persistence.
journal-type
Type of journal to use. If set to ASYNCIO, the broker first attempts to use AIO. The broker falls back to NIO if ASYNCIO is not found.
bindings-directory
File system location of the bindings journal. The default setting is relative to BROKER_INSTANCE_DIR.
journal-directory
File system location of the messaging journal. The default setting is relative to BROKER_INSTANCE_DIR.
journal-datasync
Specify whether to use fdatasync to confirm writes to the disk.
journal-min-files
Number of journal files to create when the broker starts.
journal-pool-files
Number of files to keep after reclaiming unused files. The default value of -1 means that no files are deleted during clean up.

6.2.1. The Message Journal

The message journal stores all message-related data, including the messages themselves and duplicate ID caches. The files on this journal are prefixed as activemq-data. Each file has a amq extension and a default size of 10485760 bytes. The location of the message journal is set using the journal-directory configuration element. The default value is BROKER_INSTANCE_DIR/data/journal. The default configuration includes other elements related to the messaging journal:

  • journal-min-files

    The number of journal files to pre-create when the broker starts. The default is 2.

  • journal-pool-files

    The number of files to keep after reclaiming un-used files. The default value, -1, means that no files are deleted once created by the broker. However, the system cannot grow infinitely, so you are required to use paging for destinations that are unbounded in this way. See the chapter on Paging Messages for more information.

There are several other configuration elements available for the messaging journal. See the appendix for a full list.

6.2.2. The Bindings Journal

The bindings journal is used to store bindings-related data, such as the set of queues deployed on the server and their attributes. It also stores data such as ID sequence counters.

The bindings journal always uses NIO because it is typically low throughput when compared to the message journal. Files on this journal are prefixed with activemq-bindings. Each file has a bindings extension and a default size of 1048576 bytes.

Use the following configuration elements in BROKER_INSTANCE_DIR/etc/broker.xml to configure the bindings journal.

  • bindings-directory

    This is the directory in which the bindings journal lives. The default value is BROKER_INSTANCE_DIR/data/bindings.

  • create-bindings-dir

    If this is set to true then the bindings directory is automatically created at the location specified in bindings-directory if it does not already exist. The default value is true

6.2.3. The JMS Journal

The JMS journal stores all JMS-related data, including JMS Queues, Topics, and Connection Factories, as well as any JNDI bindings for these resources. Also, any JMS Resources created via the management API is persisted to this journal, but any resources configured via configuration files are not. The JMS Journal is only created if JMS is being used.

The files on this journal are prefixed as activemq-jms. Each file has a jms extension and and a default size of 1048576 bytes.

The JMS journal shares its configuration with the bindings journal.

6.2.4. Compacting Journal Files

AMQ Broker includes a compaction algorithm that removes dead space from the journal and compresses its data so that it takes up less space on disk. There are two criteria used to determine when to start compaction. After both criteria are met, the compaction process parses the journal and removes all dead records. Consequently, the journal comprises fewer files. The criteria are:

  • The number of files created for the journal.
  • The percentage of live data in the journal’s files.

You configure both criteria in BROKER_INSTANCE_DIR/etc/broker.xml.

Procedure

  • To configure the criteria for the compaction process, add the following two elements, as in the example below.

    <configuration>
      <core>
        ...
        <journal-compact-min-files>15</journal-compact-min-files> 1
        <journal-compact-percentage>25</journal-compact-percentage>  2
        ...
      </core>
    </configuration>
    1
    The minimum number of files created before compaction begins. That is, the compacting algorithm does not start until you have at least journal-compact-min-files. The default value is 10. Setting this to 0 disables compaction, which is dangerous because the journal could grow indefinitely.
    2
    The percentage of live data in the journal’s files. When less than this percentage is considered live data, compacting begins. Remember that compacting does not begin until you also have at least journal-compact-min-files data files on the journal. The default value is 30.
Compacting Journals Using the CLI

You can also use the command-line interface (CLI) to compact journals.

Procedure

  1. As the owner of the BROKER_INSTANCE_DIR, stop the broker. In the example below, the user amq-broker was created during the installation of AMQ Broker.

    su - amq-broker
    cd __BROKER_INSTANCE_DIR__/bin
    $ ./artemis stop
  2. (Optional) Run the following CLI command to get a full list of parameters for the data tool. Note that by default, the tool uses settings found in BROKER_INSTANCE_DIR/etc/broker.xml.

    $ ./artemis help data compact.
  3. Run the following CLI command to compact the data.

    $ ./artemis data compact.
  4. After the tool has successfully compacted the data, restart the broker.

    $ ./artemis run

Related Information

AMQ Broker includes a number of CLI commands for managing your journal files. See command-line Tools in the Appendix for more information.

6.2.5. Disabling Disk Write Cache

Most disks contain hardware write caches. A write cache can increase the apparent performance of the disk because writes are lazily written to the disk later. By default many systems ship with disk write cache enabled. This means that even after syncing from the operating system there is no guarantee the data has actually made it to disk, so if a failure occurs, critical data can be lost.

Some more expensive disks have non-volatile or battery-backed write caches that do not necessarily lose data in event of failure, but you should test them. If your disk does not have such features, you should ensure that write cache is disabled. Be aware that disabling disk write cache can negatively affect performance.

Procedure

  • On Linux, manage your disk’s write cache settings using the tools hdparm (for IDE disks) or sdparm or sginfo (for SDSI/SATA disks).
  • On Windows, manage the cache setting by right-clicking the disk and clicking Properties.

6.3. Configuring JDBC Persistence

The JDBC persistence store uses a JDBC connection to store messages and bindings data in database tables. The data in the tables is encoded using AMQ Broker journal encoding. For information about supported databases, see Red Hat AMQ 7 Supported Configurations on the Red Hat Customer Portal.

Note

An administrator might choose to store messaging data in a database based on the requirements of an organization’s wider IT infrastructure. However, use of a database can negatively effect the performance of a messaging system. Specifically, writing messaging data to database tables via JDBC creates a significant performance overhead for a broker.

Procedure

  1. Add the appropriate JDBC client libraries to the broker runtime. You can do this by adding the relevant jars to the BROKER_INSTANCE_DIR/lib directory.
  2. Create a store element in your BROKER_INSTANCE_DIR/etc/broker.xml configuration file under the core element, as in the example below.

    <configuration>
      <core>
        <store>
           <database-store>
              <jdbc-connection-url>jdbc:oracle:data/oracle/database-store;create=true</jdbc-connection-url>
              <jdbc-user>ENC(5493dd76567ee5ec269d11823973462f)</jdbc-user>
              <jdbc-password>ENC(56a0db3b71043054269d11823973462f)</jdbc-password>
              <bindings-table-name>BINDINGS_TABLE</bindings-table-name>
              <message-table-name>MESSAGE_TABLE</message-table-name>
              <large-message-table-name>LARGE_MESSAGES_TABLE</large-message-table-name>
              <page-store-table-name>PAGE_STORE_TABLE</page-store-table-name>
              <node-manager-store-table-name>NODE_MANAGER_TABLE</node-manager-store-table-name>
              <jdbc-driver-class-name>oracle.jdbc.driver.OracleDriver</jdbc-driver-class-name>
              <jdbc-network-timeout>10000</jdbc-network-timeout>
              <jdbc-lock-renew-period>2000</jdbc-lock-renew-period>
              <jdbc-lock-expiration>20000</jdbc-lock-expiration>
              <jdbc-journal-sync-period>5</jdbc-journal-sync-period>
           </database-store>
        </store>
      </core>
    </configuration>
    jdbc-connection-url
    Full JDBC connection URL for your database server. The connection url should include all configuration parameters and the database name.
    jdbc-user
    Encrypted user name for your database server. For more information about encrypting user names and passwords for use in configuration files, see Section 5.9, “Encrypting passwords in configuration files”.
    jdbc-password
    Encrypted password for your database server. For more information about encrypting user names and passwords for use in configuration files, see Section 5.9, “Encrypting passwords in configuration files”.
    bindings-table-name
    Name of the table in which bindings data is stored. Specifying this table name enables you to share a single database between multiple servers, without interference.
    message-table-name
    Name of the table in which message data is stored. Specifying this table name enables you to share a single database between multiple servers, without interference.
    large-message-table-name
    Name of the table in which large messages and related data are persisted. In addition, if a client streams a large message in chunks, the chunks are stored in this table. Specifying this table name enables you to share a single database between multiple servers, without interference.
    page-store-table-name
    Name of the table in which paged store directory information is stored. Specifying this table name enables you to share a single database between multiple servers, without interference.
    node-manager-store-table-name
    Name of the table in which the shared store high-availability (HA) locks for live and backup brokers and other HA-related data is stored on the broker server. Specifying this table name enables you to share a single database between multiple servers, without interference. Each live-backup pair that uses shared store HA must use the same table name. You cannot share the same table between multiple (and unrelated) live-backup pairs.
    jdbc-driver-class-name
    Fully-qualified class name of the JDBC database driver. For information about supported databases, see Red Hat AMQ 7 Supported Configurations on the Red Hat Customer Portal.
    jdbc-network-timeout
    JDBC network connection timeout, in milliseconds. The default value is 20000 milliseconds. When using a JDBC for shared store HA, it is recommended to set the timeout to a value less than or equal to jdbc-lock-expiration.
    jdbc-lock-renew-period
    Length, in milliseconds, of the renewal period for the current JDBC lock. When this time elapses, the broker can renew the lock. The default value is 2000 milliseconds.
    jdbc-lock-expiration
    Time, in milliseconds, that the current JDBC lock is considered active, even if the jdbc-lock-renew-period time has elapsed. Setting this property to a value greater than jdbc-lock-renew-period ensures that the lock is not immediately lost if the broker that owns the lock experiences an unexpected delay in renewing it. After the expiration time elapses, if the JDBC lock has not been renewed by the broker that currently owns it, another broker can establish a JDBC lock. The default value is 20000 milliseconds.
    jdbc-journal-sync-period
    Duration, in milliseconds, for which the broker journal synchronizes with JDBC. The default value is 5 milliseconds.

6.4. Configuring Zero Persistence

In some situations, zero persistence is sometimes required for a messaging system. Configuring the broker to perform zero persistence is straightforward. Set the parameter persistence-enabled in BROKER_INSTANCE_DIR/etc/broker.xml to false.

Note that if you set this parameter to false, then zero persistence occurs. That means no bindings data, message data, large message data, duplicate ID caches or paging data is persisted.