22.6. Using Rsyslog Modules

Due to its modular design, rsyslog offers a variety of modules which provide additional functionality. Note that modules can be written by third parties. Most modules provide additional inputs (see Input Modules below) or outputs (see Output Modules below). Other modules provide special functionality specific to each module. The modules may provide additional configuration directives that become available after a module is loaded. To load a module, use the following syntax:
module(load=”MODULE”)
where MODULE represents your desired module. For example, if you want to load the Text File Input Module (imfile) that enables rsyslog to convert any standard text files into syslog messages, specify the following line in the /etc/rsyslog.conf configuration file:
module(load=”imfile”)
rsyslog offers a number of modules which are split into the following main categories:
  • Input Modules — Input modules gather messages from various sources. The name of an input module always starts with the im prefix, such as imfile and imjournal.
  • Output Modules — Output modules provide a facility to issue message to various targets such as sending across a network, storing in a database, or encrypting. The name of an output module always starts with the om prefix, such as omsnmp, omrelp, and so on.
  • Parser Modules — These modules are useful in creating custom parsing rules or to parse malformed messages. With moderate knowledge of the C programming language, you can create your own message parser. The name of a parser module always starts with the pm prefix, such as pmrfc5424, pmrfc3164, and so on.
  • Message Modification Modules — Message modification modules change content of syslog messages. Names of these modules start with the mm prefix. Message Modification Modules such as mmanon, mmnormalize, or mmjsonparse are used for anonymization or normalization of messages.
  • String Generator Modules — String generator modules generate strings based on the message content and strongly cooperate with the template feature provided by rsyslog. For more information on templates, see Section 22.2.3, “Templates”. The name of a string generator module always starts with the sm prefix, such as smfile or smtradfile.
  • Library Modules — Library modules provide functionality for other loadable modules. These modules are loaded automatically by rsyslog when needed and cannot be configured by the user.
A comprehensive list of all available modules and their detailed description can be found at http://www.rsyslog.com/doc/rsyslog_conf_modules.html.

Warning

Note that when rsyslog loads any modules, it provides them with access to some of its functions and data. This poses a possible security threat. To minimize security risks, use trustworthy modules only.

22.6.1. Importing Text Files

The Text File Input Module, abbreviated as imfile, enables rsyslog to convert any text file into a stream of syslog messages. You can use imfile to import log messages from applications that create their own text file logs. To load imfile, add the following into /etc/rsyslog.conf:
        module(load=”imfile”
        				PollingInterval=”int”)
It is sufficient to load imfile once, even when importing multiple files. The PollingInterval module argument specifies how often rsyslog checks for changes in connected text files. The default interval is 10 seconds, to change it, replace int with a time interval specified in seconds.
To identify the text files to import, use the following syntax in /etc/rsyslog.conf:
# File 1
input(type="imfile"
      File="path_to_file"
      Tag="tag:"
      Severity="severity"
      Facility="facility")

# File 2
input(type="imfile"
      File="path_to_file2")
...
Settings required to specify an input text file:
  • replace path_to_file with a path to the text file.
  • replace tag: with a tag name for this message.
Apart from the required directives, there are several other settings that can be applied on the text input. Set the severity of imported messages by replacing severity with an appropriate keyword. Replace facility with a keyword to define the subsystem that produced the message. The keywords for severity and facility are the same as those used in facility/priority-based filters, see Section 22.2.1, “Filters”.

Example 22.15. Importing Text Files

The Apache HTTP server creates log files in text format. To apply the processing capabilities of rsyslog to apache error messages, first use the imfile module to import the messages. Add the following into /etc/rsyslog.conf:
module(load=”imfile”)
input(type="imfile"
      File="/var/log/httpd/error_log"
      Tag="apache-error:")

22.6.2. Exporting Messages to a Database

Processing of log data can be faster and more convenient when performed in a database rather than with text files. Based on the type of DBMS used, choose from various output modules such as ommysql, ompgsql, omoracle, or ommongodb. As an alternative, use the generic omlibdbi output module that relies on the libdbi library. The omlibdbi module supports database systems Firebird/Interbase, MS SQL, Sybase, SQLite, Ingres, Oracle, mSQL, MySQL, and PostgreSQL.

Example 22.16. Exporting Rsyslog Messages to a Database

To store the rsyslog messages in a MySQL database, add the following into /etc/rsyslog.conf:
module(load=”ommysql”)

*.*  action(type”ommysql”
server=”database-server”
db=”database-name”
uid=”database-userid”
pwd=”database-password”
serverport=”1234”)
First, the output module is loaded, then the communication port is specified. Additional information, such as name of the server and the database, and authentication data, is specified on the last line of the above example.

22.6.3. Enabling Encrypted Transport

Confidentiality and integrity in network transmissions can be provided by either the TLS or GSSAPI encryption protocol.
Transport Layer Security (TLS) is a cryptographic protocol designed to provide communication security over the network. When using TLS, rsyslog messages are encrypted before sending, and mutual authentication exists between the sender and receiver. For configuring TLS, see the section called “Configuring Encrypted Message Transfer with TLS”.
Generic Security Service API (GSSAPI) is an application programming interface for programs to access security services. To use it in connection with rsyslog you must have a functioning Kerberos environment. For configuring GSSAPI, see the section called “Configuring Encrypted Message Transfer with GSSAPI”.

Configuring Encrypted Message Transfer with TLS

To use encrypted transport through TLS, you need to configure both the server and the client.
  1. Create public key, private key and certificate file, see Section 14.1.11, “Generating a New Key and Certificate”.
  2. On the server side, configure the following in the /etc/rsyslog.conf configuration file:
    1. Set the gtls netstream driver as the default driver:
      global(defaultnetstreamdriver="gtls")
    2. Provide paths to certificate files:
      global(defaultnetstreamdrivercafile="path_ca.pem"
      defaultnetstreamdrivercertfile="path_cert.pem"
      defaultnetstreamdriverkeyfile="path_key.pem")
      
      You can merge all global directives into single block if you prefer a less cluttered configuration file.
      Replace:
      • path_ca.pem with a path to your public key
      • path_cert.pem with a path to the certificate file
      • path_key.pem with a path to the private key
    3. Load the imtcp moduleand set driver options:
      module(load=”imtcp”
      StreamDriver.Mode=“number”
      StreamDriver.AuthMode=”anon”)
      
    4. Start a server:
      input(type="imtcp" port="port″)
      
      Replace:
      • number to specify the driver mode. To enable TCP-only mode, use 1
      • port with the port number at which to start a listener, for example 10514
      The anon setting means that the client is not authenticated.
  3. On the client side, configure the following in the /etc/rsyslog.conf configuration file:
    1. Load the public key:
      global(defaultnetstreamdrivercafile="path_ca.pem")
      Replace path_ca.pem with a path to the public key.
    2. Set the gtls netstream driver as the default driver:
       global(defaultnetstreamdriver="gtls")
    3. Configure the driver and specify what action will be performed:
      module(load=”imtcp”
          streamdrivermode=”number”
          streamdriverauthmode=”anon”)
      input(type=”imtcp”
          address=”server.net”
          port=”port”)
      
      Replace number, anon, and port with the same values as on the server.
      On the last line in the above listing, an example action forwards messages from the server to the specified TCP port.

Configuring Encrypted Message Transfer with GSSAPI

In rsyslog, interaction with GSSAPI is provided by the imgssapi module. To turn on the GSSAPI transfer mode:
  1. Put the following configuration in /etc/rsyslog.conf:
    $ModLoad imgssapi
    This directive loads the imgssapi module.
  2. Specify the input as follows:
    $InputGSSServerServiceName name
    $InputGSSServerPermitPlainTCP on
    $InputGSSServerMaxSessions number
    $InputGSSServerRun port
    • Replace name with the name of the GSS server.
    • Replace number to set the maximum number of sessions supported. This number is not limited by default.
    • Replace port with a selected port on which you want to start a GSS server.
    The $InputGSSServerPermitPlainTCP on setting permits the server to receive also plain TCP messages on the same port. This is off by default.

Note

The imgssapi module is initialized as soon as the configuration file reader encounters the $InputGSSServerRun directive in the /etc/rsyslog.conf configuration file. The supplementary options configured after $InputGSSServerRun are therefore ignored. For configuration to take effect, all imgssapi configuration options must be placed before $InputGSSServerRun.

Example 22.17. Using GSSAPI

The following configuration enables a GSS server on the port 1514 that also permits to receive plain tcp syslog messages on the same port.
$ModLoad imgssapi
$InputGSSServerPermitPlainTCP on
$InputGSSServerRun 1514

22.6.4. Using RELP

Reliable Event Logging Protocol (RELP) is a networking protocol for data logging in computer networks. It is designed to provide reliable delivery of event messages, which makes it useful in environments where message loss is not acceptable.

Configuring RELP

To configure RELP, you need to configure both the server and the client using the /etc/rsyslog.conf file.
  1. To configure the client:
    1. Load the required modules:
      module(load="imuxsock")
      module(load="omrelp")
      module(load="imtcp")
    2. Configure the TCP input as follows:
      input(type="imtcp" port="port″)
      Replace port to start a listener at the required port.
    3. Configure the transport settings:
      action(type="omrelp" target="target_IP″ port="target_port″)
      Replace target_IP and target_port with the IP address and port that identify the target server.
  2. To configure the server:
    1. Configure loading the module:
      module(load="imuxsock")
      module(load="imrelp" ruleset="relp")
      
    2. Configure the TCP input similarly to the client configuration:
      input(type="imrelp" port="target_port″)
      Replace target_port with the same value as on the clients.
    3. Configure the rules and choose an action to be performed. In the following example, log_path specifies the path for storing messages:
      ruleset (name="relp") {
      action(type="omfile" file="log_path")
      }

Configuring RELP with TLS

To configure RELP with TLS, you need to configure authentication. Then, you need to configure both the server and the client using the /etc/rsyslog.conf file.
  1. Create public key, private key and certificate file. For instructions, see Section 14.1.11, “Generating a New Key and Certificate”.
  2. To configure the client:
    1. Load the required modules:
      module(load="imuxsock")
      module(load="omrelp")
      module(load="imtcp")
    2. Configure the TCP input as follows:
      input(type="imtcp" port="port″)
      Replace port to start a listener at the required port.
    3. Configure the transport settings:
      action(type="omrelp" target="target_IP″ port="target_port″ tls="on"
      tls.caCert="path_ca.pem"
      tls.myCert="path_cert.pem"
      tls.myPrivKey="path_key.pem"
      tls.authmode="mode"
      tls.permittedpeer=["peer_name"]
      )
      Replace:
      • target_IP and target_port with the IP address and port that identify the target server.
      • path_ca.pem, path_cert.pem, and path_key.pem with paths to the certification files
      • mode with the authentication mode for the transaction. Use either "name" or "fingerprint"
      • peer_name with a certificate fingerprint of the permitted peer. If you specify this, tls.permittedpeer restricts connection to the selected group of peers.
      The tls="on" setting enables the TLS protocol.
  3. To configure the server:
    1. Configure loading the module:
      module(load="imuxsock")
      module(load="imrelp" ruleset="relp")
      
    2. Configure the TCP input similarly to the client configuration:
      input(type="imrelp" port="target_port″ tls="on"
      tls.caCert="path_ca.pem"
      tls.myCert="path_cert.pem"
      tls.myPrivKey="path_key.pem"
      tls.authmode="name"
      tls.permittedpeer=["peer_name","peer_name1","peer_name2"]
      )
      Replace the highlighted values with the same as on the client.
    3. Configure the rules and choose an action to be performed. In the following example, log_path specifies the path for storing messages:
      ruleset (name="relp") {
      action(type="omfile" file="log_path")
      }