Chapter 4. Using the LevelDB Persistence Adapter

Abstract

The LevelDB persistence adapter uses LevelDB as a high-performance message store. It allows for higher throughput speeds than the default message store.
Important
The LevelDB store is supported only on the Red Hat Enterprise Linux (RHEL) platform (for the RHEL versions and configurations specified in Supported Configurations). If you want to store the LevelDB files on a distributed file system, the following configuration is also supported: the broker itself and the LevelDB adapter plugin run on a supported RHEL platform, while the LevelDB files are stored on an NFSv4 file system (the O/S of the server that hosts the NFSv4 file system is irrelevant). Note that the distributed file system must be NFSv4.
For all other JBoss A-MQ-supported platforms, the LevelDB store is provided for technical preview only. For details on what "technical preview" means, see https://access.redhat.com/support/offerings/techpreview/.

Overview

The LevelDB message store is a file based message store implemented using Google's LevelDB library to maintain indexes into log files holding the messages. The main advantages of the LevelDB store include:
  • higher persistent throughput
  • faster recovery times when a broker restarts
  • supports concurrent read access
  • no pausing during garbage collection cycles
  • uses fewer read IO operations to load stored messages
  • supports XA transactions
  • checks for duplicate messages
  • exposes status via JMX for monitoring
  • supports replication

Platform support

LevelDB is implemented in C++, and Red Hat JBoss A-MQ accesses the libraries using a JNI driver.
JBoss A-MQ also provides an experimental pure Java driver for all other platforms.

Basic configuration

The LevelDB message store is configured by placing a levelDB element in the persistenceAdapter element of your broker's configuration. The levelDB element's attributes are used to configure the message store.
All attributes, listed in Table 4.1, “Configuration Properties of the LevelDB Message Store—standard LevelDB attributes”, have reasonable default values, so you are not required to specify values for them. However, you will need to explicitly specify the location of the message store's data files by providing a value for the directory attribute. This will ensure that the broker will not conflict with other brokers.
Example 4.1, “Configuring the LevelDB Message Store” shows a basic configuration of the LevelDB message store. The LevelDB files are stored under the activemq-data directory.

Example 4.1. Configuring the LevelDB Message Store

<broker brokerName="broker" persistent="true" ... >
  ...
  <persistenceAdapter>
    <levelDB directory="activemq-data" />
  </persistenceAdapter>
  ...
</broker>

Configuration attributes

Table 4.1, “Configuration Properties of the LevelDB Message Store—standard LevelDB attributes” describes the attributes that can be used to configure the LevelDB message store.

Table 4.1. Configuration Properties of the LevelDB Message Store—standard LevelDB attributes

AttributeDefault ValueDescription
directory activemq-data Specifies the path to the top-level folder that holds the message store's data files.
readThreads 10 Specifies the number of concurrent IO reads to allow.
sync true Specifies whether syncs log operations to disk.
logSize 104857600 Specifies the maximum size, in bytes, of each data log file before log file rotation occurs.
verifyChecksums false Specifies whether checksum verification is performed on all data that is read from the file system.
paranoidChecks false Specifies whether the store errors out as soon as possible when it detects internal corruption.
indexFactory org.fusesource.leveldbjni.JniDBFactory, org.iq80.leveldb.impl.Iq80DBFactory
Specifies a comma separated list of leveldb API implementation factory classes that the broker will attempt to load. The broker will use the first one that loads successfully.
  • org.fusesource.leveldbjni.JniDBFactory enables the JNI base implementation
  • org.iq80.leveldb.impl.Iq80DBFactory enables the pure Java implementation
indexMaxOpenFiles 1000 Specifies the number of open files that can be used by the index.
indexBlockRestartInterval 16 Specifies the number of keys between restart points for delta encoding of keys.
indexWriteBufferSize 4194304 Specifies the amount, in bytes, of index data to build up in memory before converting to a sorted on-disk file.
indexBlockSize 4096 Specifies the size, in bytes, of index data packed per block.
indexCacheSize 268435456 Specifies the maximum amount, in bytes, of memory to use to cache index blocks.
indexCompression snappy Specifies the type of compression to apply to the index blocks. Can be snappy or none.
logCompression snappy Specifies the type of compression to apply to the log records. Can be snappy or none.

Table 4.2. Configuration Properties of the LevelDB Message Store—pluggable storage locker attributes

AttributeDefault ValueDescription
failIfLocked false Specifies whether the broker will fail on start up if the message store's data files are locked. If the broker does not fail, it blocks until the data files are unlocked.
useLock true Specifies whether the adapter uses file locking.