25.5. Working with Queues in Rsyslog

Figure 25.1. Message Flow in Rsyslog
/etc/rsyslog.conf
are applied. Based on these rules, the rule processor evaluates which actions are to be performed. Each action has its own action queue. Messages are passed through this queue to the respective action processor which creates the final output. Note that at this point, several actions can run simultaneously on one message. For this purpose, a message is duplicated and passed to multiple action processors.
- they serve as buffers that decouple producers and consumers in the structure of rsyslog
- they allow for parallelization of actions performed on messages
Warning
SSH
logging, which in turn can prevent SSH
access. Therefore it is advised to use dedicated action queues for outputs which are forwarded over a network or to a database.
25.5.1. Defining Queues
/etc/rsyslog.conf
:
$objectQueueType queue_type
MainMsg
) or for an action queue (replace object with Action
). Replace queue_type with one of direct
, linkedlist
or fixedarray
(which are in-memory queues), or disk
.
Direct Queues
$objectQueueType Direct
MainMsg
or with Action
to use this option to the main message queue or for an action queue respectively. With direct queue, messages are passed directly and immediately from the producer to the consumer.
Disk Queues
/etc/rsyslog.conf
:
$objectQueueType Disk
MainMsg
or with Action
to use this option to the main message queue or for an action queue respectively. Disk queues are written in parts, with a default size 10 Mb. This default size can be modified with the following configuration directive:
$objectQueueMaxFileSize size
$objectQueueFilename name
In-memory Queues
$ActionQueueSaveOnShutdown
setting to save the data before shutdown. There are two types of in-memory queues:
- FixedArray queue — the default mode for the main message queue, with a limit of 10,000 elements. This type of queue uses a fixed, pre-allocated array that holds pointers to queue elements. Due to these pointers, even if the queue is empty a certain amount of memory is consumed. However, FixedArray offers the best run time performance and is optimal when you expect a relatively low number of queued messages and high performance.
- LinkedList queue — here, all structures are dynamically allocated in a linked list, thus the memory is allocated only when needed. LinkedList queues handle occasional message bursts very well.
$objectQueueType LinkedList
$objectQueueType FixedArray
MainMsg
or with Action
to use this option to the main message queue or for an action queue respectively.
Disk-Assisted In-memory Queues
$objectQueueFileName
directive to define a file name for disk assistance. This queue then becomes disk-assisted, which means it couples an in-memory queue with a disk queue to work in tandem.
$objectQueueHighWatermark number
$objectQueueLowWatermark number
MainMsg
or with Action
to use this option to the main message queue or for an action queue respectively. Replace number with a number of enqueued messages. When an in-memory queue reaches the number defined by the high watermark, it starts writing messages to disk and continues until the in-memory queue size drops to the number defined with the low watermark. Correctly set watermarks minimize unnecessary disk writes, but also leave memory space for message bursts since writing to disk files is rather lengthy. Therefore, the high watermark must be lower than the whole queue capacity set with $objectQueueSize. The difference between the high watermark and the overall queue size is a spare memory buffer reserved for message bursts. On the other hand, setting the high watermark too low will turn on disk assistance unnecessarily often.
Example 25.12. Reliable Forwarding of Log Messages to a Server
UDP
protocol. To establish a fully reliable connection, for example when your logging server is outside of your private network, consider using the RELP protocol described in Section 25.7.4, “Using RELP”.
Procedure 25.2. Forwarding To a Single Server
- Use the following configuration in
/etc/rsyslog.conf
or create a file with the following content in the/etc/rsyslog.d/
directory:$ActionQueueType LinkedList $ActionQueueFileName example_fwd $ActionResumeRetryCount -1 $ActionQueueSaveOnShutdown on *.* @@example.com:6514
Where:$ActionQueueType
enables a LinkedList in-memory queue,$ActionFileName
defines a disk storage, in this case the backup files are created in the/var/lib/rsyslog/
directory with the example_fwd prefix,- the
$ActionResumeRetryCount -1
setting prevents rsyslog from dropping messages when retrying to connect if server is not responding, - enabled
$ActionQueueSaveOnShutdown
saves in-memory data if rsyslog shuts down, - the last line forwards all received messages to the logging server, port specification is optional.
With the above configuration, rsyslog keeps messages in memory if the remote server is not reachable. A file on disk is created only if rsyslog runs out of the configured memory queue space or needs to shut down, which benefits the system performance.
Procedure 25.3. Forwarding To Multiple Servers
- Each destination server requires a separate forwarding rule, action queue specification, and backup file on disk. For example, use the following configuration in
/etc/rsyslog.conf
or create a file with the following content in the/etc/rsyslog.d/
directory:$ActionQueueType LinkedList $ActionQueueFileName example_fwd1 $ActionResumeRetryCount -1 $ActionQueueSaveOnShutdown on *.* @@example1.com $ActionQueueType LinkedList $ActionQueueFileName example_fwd2 $ActionResumeRetryCount -1 $ActionQueueSaveOnShutdown on *.* @@example2.com