LibraryPrintFeedback

Configuring Message Broker Persistence

Version 7.1

December 2012
Trademark Disclaimer
Third Party Acknowledgements

Updated: 08 Jan 2014

Table of Contents

1. Introduction to Fuse MQ Enterprise Persistence
2. Using the KahaDB Message Store
Understanding the KahaDB Message Store
Configuring the KahaDB Message Store
Concurrent Store and Dispatch
Optimizing the Metadata Cache
Recovery
3. Using a Distributed KahaDB Persistence Adapter
4. Using JDBC to Connect to a Database Store
Basics of Using the JDBC Persistence Adapter
Using JDBC with the High Performance Journal
Using JDBC without the Journal
Customizing the JDBC Persistence Adapter
5. Message Cursors
Types of Cursors
Configuring the Type of Cursor Used by a Destination
Index

List of Figures

2.1. Overview of the KahaDB Message Store
2.2. Concurrent Store and Dispatch—Slow Consumers
2.3. Concurrent Store and Dispatch—Fast Consumers
2.4. Serialized Store and Dispatch
2.5. Overview of the Metadata Cache and Store
5.1. Store-based Cursors for a Fast Consumer
5.2. Store-based Cursors for a Slow Consumer
5.3. VM Cursors
5.4. File-based Cursors

List of Tables

1.1. Setting a Broker's Persistence
2.1. Configuration Properties of the KahaDB Message Store
4.1. Attributes for Configuring the Journaled JDBC Persistence Adapter
4.2. Attributes for Configuring the Plain JDBC Persistence Adapter
4.3. Statements for Configuring the SQL Statements Used by the JDBC Persistence Adapter
5.1. Elements for Configuring the Type of Cursor to Use for Durable Subscribers
5.2. Elements for Configuring the Type of Cursor to Use for Transient Subscribers
5.3. Elements for Configuring the Type of Cursor to Use for a Queue

List of Examples

1.1. Turning Off a Broker's Persistence
2.1. Configuring the KahaDB Message Store
2.2. KahaDB Configured with Serialized Store and Dispatch
2.3. Configuration for Journal Validation
3.1. Distributed KahaDB Persistence Adapter Configuration
4.1. Configuration for the Apache Derby Database
4.2. Configuration for the Oracle JDBC Driver
4.3. Configuring Fuse MQ Enterprise to use the Journaled JDBC Persistence Adapter
4.4. Configuring Fuse MQ Enterprise to use the Plain JDBC Persistence Adapter
4.5. Fine Tuning the Database Schema
4.6. Configuring a Generic JDBC Provider
5.1. Configuring a Topic's Cursor Usage
5.2. Configuring a Queue's Cursor Usage

By default, brokers are configured to use a persistence layer to ensure that persistent messages will survive a broker failure and meet the once-and-only-once requirement of the JMS specification. Having a broker's persistence layer configured comes with a cost in terms of resources used and speed, so for testing purposes or cases where persistence will never be required, it may make sense to disable a broker's persistence layer.

Deactivating a broker's persistence layer means that a broker will treat all messages as non-persistent. If a producer sets a message's JMSDeliveryMode property to PERSISTENT the broker will not respect the setting. The message will be delivered at-most-once instead of once-and-only-once. This means that persistent messages will not survive broker shutdown.

Persistence in Fuse MQ Enterprise is controlled by a broker's XML configuration file. To change a broker's persistence behavior you modify the configuration's broker element's persistent attribute.


Example 1.1 shows a configuration snippet for turning off a broker's message persistence.


[Important]Important

If you use antivirus software it can interfere with Fuse MQ Enterprise's ability to access the files in the KahaDB message store. You should configure your antivirus software to skip the KahaDB data folders when doing automatic scans.

The KahaDB message store is configured by placing a kahaDB element in the persistenceAdapter element of your broker's configuration. The kahaDB element's attributes are used to configure the message store.

The attributes, listed in Table 2.1, all have reasonable default values, so you are not required to specify values for them. However, you will want to explicitly specify the location of the message store's data files by providing a value for the directory attribute. This will ensure that the broker will not conflict with other brokers.

Example 2.1 shows a basic configuration of the KahaDB message store. The KahaDB files are stored under the activemq-data directory.


Table 2.1 describes the attributes that can be used to configure the KahaDB message store.

Table 2.1. Configuration Properties of the KahaDB Message Store

AttributeDefault ValueDescription
directory activemq-dataSpecifies the path to the top-level folder that holds the message store's data files.
indexWriteBatchSize 1000Specifies the number of B-tree indexes written in a batch. Whenever the number of changed indexes exceeds this value, the metadata cache is written to disk.
indexCacheSize 10000Specifies the number of B-tree index pages cached in memory.
enableIndexWriteAsync falseSpecifies if kahaDB will asynchronously write indexes.
journalMaxFileLength 32mbSpecifies the maximum size of the data log files.
enableJournalDiskSyncs trueSpecifies whether every non-transactional journal write is followed by a disk sync. If you want to satisfy the JMS durability requirement, you must also disable concurrent store and dispatch.
cleanupInterval 30000Specifies the time interval, in milliseconds, between cleaning up data logs that are no longer used.
checkpointInterval 5000Specifies the time interval, in milliseconds, between writing the metadata cache to disk.
ignoreMissingJournalfiles falseSpecifies whether the message store ignores any missing journal files while it starts up. If false, the message store raises an exception when it discovers a missing journal file.
checkForCorruptJournalFiles falseSpecifies whether the message store checks for corrupted journal files on startup and tries to recover them.
checksumJournalFiles falseSpecifies whether the message store generates a checksum for the journal files. If you want to be able to check for corrupted journals, you must set this to true.
archiveDataLogs falseSpecifies if the message store moves spent data logs to the archive directory.
directoryArchive nullSpecifies the location of the directory to archive data logs.
databaseLockedWaitDelay 10000Specifies the time delay, in milliseconds, before trying to acquire the database lock in the context of a shared master/slave failover deployment. See Shared File System Master/Slave in Fault Tolerant Messaging.
maxAsyncJobs 10000Specifies the size of the task queue used to buffer the broker commands waiting to be written to the journal. The value should be greater than or equal to the number of concurrent message producers. See Concurrent Store and Dispatch.
concurrentStoreAndDispatchTopics falseSpecifies if the message store dispatches topic messages to interested clients concurrently with message storage. See Concurrent Store and Dispatch.
concurrentStoreAndDispatchQueues trueSpecifies if the message store dispatches queue messages to clients concurrently with message storage. See Concurrent Store and Dispatch.
archiveCorruptedIndex falseSpecifies if corrupted indexes are archived when the broker starts up.
useLocktrueSpecifies in the adapter uses file locking.

Proper configuration of the metadata cache is one of the key factors affecting the performance of the KahaDB message store. In a production deployment, therefore, you should always take the time to tune the properties of the metadata cache for maximum performance. Figure 2.5 shows an overview of the metadata cache and how it interacts with the metadata store. The most important part of the metadata is the B-tree index, which is shown as a tree of nodes in the figure. The data in the cache is periodically synchronized with the metadata store, when a checkpoint is performed.


Normally, the journal tends to run ahead of the metadata store, because the journal is constantly being updated, whereas the metadata store is written only periodically (for example, whenever there is a checkpoint). Consequently, whenever there is a disorderly shutdown (which prevents the final state of the broker from being saved), it is likely that the stored metadata will be inconsistent with the journal, with the journal containing additional events not reflected in the metadata store.

When the broker restarts after a disorderly shutdown, the KahaDB message store recovers by reading the stored metadata into the cache and then reading the additional journal events not yet taken into account in the stored metadata (KahaDB can easily locate the additional journal events, because the metadata store always holds a reference to the last consistent location in the journal). KahaDB replays the additional journal events in order to recreate the original metadata.

[Note]Note

The KahaDB message store also uses a redo log, db.redo, to reduce the risk of a system failure occurring while updating the metadata store. Before updating the metadata store, KahaDB always saves the redo log, which summarizes the changes that are about to be made to the store. Because the redo log is a small file, it can be written relatively rapidly and is thus less likely to be affected by a system failure. During recovery, KahaDB checks whether the changes recorded in the redo log need to be applied to the metadata.

The distributed KahaDB persistence adapter configuration wraps more than one KahaDB message store configuration.

The distributed KahaDB persistence adapter configuration is specified using the mKahaDB element. The mKahaDB element has a single attribute, directory, that specifies the location where the adapter writes its data stores. This setting is the default value for the directory attribute of the embedded KahaDB message store instances. The individual message stores can override this default setting.

The mKahaDB element has a single child filteredPersistenceAdapters. The filteredPersistenceAdapters element contains multiple filteredKahaDB elements that configure the KahaDB message stores that are used by the persistence adapter.

Each filteredKahaDB element configures one KahaDB message store. The destinations matched to the message store are specified using attributes on the filteredKahaDB element:

  • queue—specifies the name of queues

  • topic—specifies the name of topics

The destinations can be specified either using explicit destination names or using wildcards. For information on using wildcards see Filters. If no destinations are specified the message store will match any destinations that are not matched by other filters.

The KahaDB message store configured by a filteredKahaDB element is configured using the standard KahaDB persistence adapter configuration. It consists of a kahaDB element wrapped in a persistenceAdapter element. For details on configuring a KahaDB message store see Configuring the KahaDB Message Store.

Fuse MQ Enterprise support two types of JDBC store:

The journaled JDBC store features better performance than the plain JDBC store; but the journaled JDBC store is incompatible with the JDBC master/slave failover pattern—see Shared JDBC Master/Slave in Fault Tolerant Messaging.

Example 4.2 shows the configuration for using the Oracle JDBC driver. The persistence adapter configuration refers to the Spring bean element that configures the JDBC driver.


The JDBC drivers are configured using a Spring bean element. The id attribute specifies the name by which you will refer to the driver when configuring the JDBC persistence adapter. The class attribute specifies the class that implements the data source used to interface with the JDBC driver. The destroy-method attribute specifies the name of the method to call when the JDBC driver is shutdown.

In addition to the bean element, the JDBC driver configuration includes a number of property elements. Each property element specifies a property required by the JDBC driver. For information about the configurable properties refer to your JDBC driver's documentation.

Example 4.3 shows a configuration fragment that configures the journaled JDBC adapter to use a MySQL database.

Example 4.3. Configuring Fuse MQ Enterprise to use the Journaled JDBC Persistence Adapter

<beans ... >
  <broker ...>
    ...
1  <persistenceFactory>
2    <journaledJDBC journalLogFiles="5" dataSource="#mysql-ds" />
    </persistenceFactory>
    ...
  <broker>
  ...
3<bean id="mysql-ds"
      class="org.apache.commons.dbcp.BasicDataSource"
      destroy-method="close">
    <property name="driverClassName" value="com.mysql.jdbc.Driver"/>
    <property name="url" value="jdbc:mysql://localhost/activemq?relaxAutoCommit=true"/>
    <property name="username" value="activemq"/>
    <property name="password" value="activemq"/>
    <property name="poolPreparedStatements" value="true"/>
  </bean>

The configuration in Example 4.3 has three noteworthy elements:

1

The persistenceFactory element wraps the configuration for the JDBC persistence adapter.

[Note]Note

In Fuse MQ Enterprise versions prior to 5.4, the persistenceAdapter element was sometimes used to wrap the configuration for the JDBC persistence adapter. This is no longer possible in 5.4, because the broker performs strict schema checking on the configuration file.

2

The journaledJDBC element specifies that the broker will use the JDBC persistence adapter with the high performance journal. The element's attributes configure the following properties:

  • The journal will span five log files.

  • The configuration for the JDBC driver is specified in a bean element with the ID, mysql-ds.

3

The bean element specified the configuration for the MySQL JDBC driver.

Example 4.4 shows a configuration fragment that configures the plain JDBC adapter to use the Apache Derby database.

Example 4.4. Configuring Fuse MQ Enterprise to use the Plain JDBC Persistence Adapter

<beans ... >
  <broker ...>
  ...
1  <persistenceAdapter>
2    <jdbcPersistenceAdapter dataSource="#derby-ds" />
    </persistenceAdapter>
    ...
  <broker>
  ...
3<bean id="derby-ds" class="org.apache.derby.jdbc.EmbeddedDataSource">
    <property name="databaseName" value="derbydb"/>
    <property name="createDatabase" value="create"/>
  </bean>

The configuration in Example 4.4 has three noteworthy elements:

1

The persistenceAdapter element wraps the configuration for the JDBC persistence adapter.

2

The jdbcPersistenceAdapter element specifies that the broker will use the plain JDBC persistence adapter and that the JDBC driver's configuration is specified in a bean element with the ID, derby-ds.

3

The bean element specified the configuration for the Derby JDBC driver.

You can customize the SQL statements that the JDBC persistence adapter uses to access the database. This is done by adding a statements element to the JDBC persistence adapter configuration. Example 4.5 shows a configuration snippet that specifies that long strings are going to be stored as VARCHAR(128).


The first statements element is a wrapper for one or more nested statements elements. Each nested statements element specifies a single configuration statement. Table 4.3 describes the configurable properties.


The properties listed in Table 4.3 configure the default SQL statements used by the JDBC adapter and work with all of the supported databases.

Message data is cached in the broker using message cursors, where a cursor instance is associated with each destination. A message cursor represents a batch of messages cached in memory. When necessary, a message cursor will retrieve persisted messages through the persistence adapter. But the key point you need to understand about message cursors is that the cursors are essentially independent of the persistence layer.

Message cursors provide a means for optimizing a persistent message store. They allow the persistent store to maintain a pointer to the next batch of messages to pull from the persistent message store. Fuse MQ Enterprise has three types of cursors that can be used depending on the needs of your application:

  • Store-based cursors are used by default to handle persistent messages.

  • VM cursors are very fast, but cannot handle slow message consumers.

  • File-based cursors are used by default to handle non-persistent messages. They are useful when the message store is slow and message consumers are relatively fast.

Topics maintain a dispatch queue and a pending cursor for every consumer subscribed to the topic regardless of whether the subscription is durable or transient. You can configure the cursor implementation used by durable subscribers separately from the cursor implementation used by transient subscribers.

You configure the cursor implementation used by durable subscribers by adding PendingDurableSubscriberMessageStoragePolicy child element to the topic's policyEntry element. Table 5.1 describes the possible children of PendingDurableSubscriberMessageStoragePolicy.


You configure the cursor implementation used by transient subscribers by adding pendingSubscriberPolicy child element to the topic's policyEntry element. Table 5.2 describes the possible children of pendingSubscriberPolicy.


Example 5.1 shows a configuration snip-it that configures a topic to use VM cursors for its transient subscribers and file-based cursors for its durable subscribers.


C

configuration
turning persistence on/off, Activating and deactivating persistence
cursors
file-based, File-based cursors
store-based, Store-based cursors
VM, VM cursors

D

destinationPolicy, Overview
distributed Kahadb persistence adapter
transactions, Transactions
durable subscribers
configuring cursors, Configuring topics
using file-based cursors, Configuring topics
using VM cursors, Configuring topics

F

fileCursor, Configuring topics
fileDurableSubscriberCursor, Configuring topics
fileQueueCursor, Configuring queues
filteredKahaDB, Configuration
filteredPersistenceAdapters, Configuration

J

JDBC
using generic providers, Using generic JDBC providers
jdbcPersistenceAdapter, Configuration
adapter attribute, Configuration, Using generic JDBC providers
cleanupPeriod attribute, Configuration
createTablesOnStartup attribute, Configuration
dataDirectory attribute, Configuration
dataSource attribute, Configuration
journaledJDBC, Configuration
adapter attribute, Configuration, Using generic JDBC providers
createTablesOnStartup attribute, Configuration
dataDirectory attribute, Configuration
dataSource attribute, Configuration
journalArchiveDirectory attribute, Configuration
journalLogFiles attribute, Configuration
journalLogFileSize attribute, Configuration
journalThreadPriority attribute, Configuration
useJournal attribute, Configuration

K

kahaDB element, Basic configuration
archiveCorruptedIndex attribute, Configuration attributes
archiveDataLogs attribute, Configuration attributes
checkForCorruptJournalFiles attribute, Configuration attributes
checkpointInterval attribute, Configuration attributes
checksumJournalFiles attribute, Configuration attributes
cleanupInterval attribute, Configuration attributes
concurrentStoreAndDispatchQueues attribute, Configuration attributes
concurrentStoreAndDispatchTopics attribute, Configuration attributes
databaseLockedWaitDelay attribute, Configuration attributes
directory attribute, Configuration attributes
directoryArchive attribute, Configuration attributes
enableIndexWriteAsync attribute, Configuration attributes
enableJournalDiskSyncs attribute, Configuration attributes
ignoreMissingJournalfiles attribute, Configuration attributes
indexCacheSize attribute, Configuration attributes
indexWriteBatchSize attribute, Configuration attributes
journalMaxFileLength attribute, Configuration attributes
maxAsyncJobs attribute, Configuration attributes
KahaDB message store
architecture, Architecture
basic configuration, Basic configuration
configuration attributes, Configuration attributes
data logs, Data logs
distributed, Using a Distributed KahaDB Persistence Adapter
metadata cache, Metadata cache
metadata store, Metadata store

M

mKahaDB, Configuration

P

PendingDurableSubscriberMessageStoragePolicy, Configuring topics
pendingQueuePolicy, Configuring queues
pendingSubscriberPolicy, Configuring topics
persistenceAdapter, Configuring persistence adapter behavior
persistenceFactory, Configuring persistence adapter behavior
policyEntries, Overview
policyEntry, Overview
policyMap, Overview

T

transactions
distributed destination, Transactions
distributed Kahadb persistence adapter, Transactions
multiple journals, Transactions
transient subscribers
configuring cursors, Configuring topics
using file-based cursors, Configuring topics
using VM cursors, Configuring topics

V

vmCursor, Configuring topics
vmDurableCursor, Configuring topics
vmQueueCursor, Configuring queues