Chapter 6. Using the Logging System Role

As a system administrator, you can use the Logging System Role to configure a RHEL host as a logging server to collect logs from many client systems.

6.1. The Logging System Role

With the Logging System Role, you can deploy logging configurations on local and remote hosts.

To apply a Logging System Role on one or more systems, you define the logging configuration in a playbook. A playbook is a list of one or more plays. Playbooks are human-readable, and they are written in the YAML format. For more information about playbooks, see Working with playbooks in Ansible documentation.

The set of systems that you want Ansible to configure according to the playbook is defined in an inventory file. For more information on creating and using inventories, see How to build your inventory in Ansible documentation.

Logging solutions provide multiple ways of reading logs and multiple logging outputs.

For example, a logging system can receive the following inputs:

  • local files,
  • systemd/journal,
  • another logging system over the network.

In addition, a logging system can have the following outputs:

  • logs are stored in the local files in the /var/log directory,
  • logs are sent to Elasticsearch,
  • logs are forwarded to another logging system.

With the logging system role, you can combine the inputs and outputs to fit your needs. For example, you can configure a logging solution that stores inputs from journal in a local file, whereas inputs read from files are both forwarded to another logging system and stored in the local log files.

6.2. Logging System Role parameters

In a Logging System Role playbook, you define the inputs in the logging_inputs parameter, outputs in the logging_outputs parameter, and the relationships between the inputs and outputs in the logging_flows parameter. The Logging System Role processes these variables with additional options to configure the logging system. You can also enable encryption.

Note

Currently, the only available logging system in the Logging System Role is Rsyslog.

  • logging_inputs - List of inputs for the logging solution.

    • name - Unique name of the input. Used in the logging_flows inputs list and a part of the generated config file name.

    • type - Type of the input element. The type specifies a task type which corresponds to a directory name in roles/rsyslog/{tasks,vars}/inputs/.

      • basics - Inputs configuring inputs from systemd journal or unix socket.

        • kernel_message - Load imklog if set to true. Default to false.
        • use_imuxsock - Use imuxsock instead of imjournal. Default to false.
        • ratelimit_burst - Maximum number of messages that can be emitted within ratelimit_interval. Default to 20000 if use_imuxsock is false. Default to 200 if use_imuxsock is true.
        • ratelimit_interval - Interval to evaluate ratelimit_burst. Default to 600 seconds if use_imuxsock is false. Default to 0 if use_imuxsock is true. 0 indicates rate limiting is turned off.
        • persist_state_interval - Journal state is persisted every value messages. Default to 10. Effective only when use_imuxsock is false.
      • files - Inputs configuring inputs from local files.
      • remote - Inputs configuring inputs from the other logging system over network.
    • state - State of the configuration file. present or absent. Default to present.

  • logging_outputs - List of outputs for the logging solution.

    • files - Outputs configuring outputs to local files.
    • forwards - Outputs configuring outputs to another logging system.
    • remote_files - Outputs configuring outputs from another logging system to local files.

  • logging_flows - List of flows that define relationships between logging_inputs and logging_outputs. The logging_flows variable has the following keys:

    • name - Unique name of the flow
    • inputs - List of logging_inputs name values
    • outputs - List of logging_outputs name values.

Additional resources

  • Documentation installed with the rhel-system-roles package in /usr/share/ansible/roles/rhel-system-roles.logging/README.html

6.3. Applying a local Logging System Role

Follow these steps to prepare and apply a Red Hat Ansible Engine playbook to configure a logging solution on a set of separate machines. Each machine will record logs locally.

Prerequisites

  • You have Red Hat Ansible Engine installed on the system from which you want to run the playbook.

    Note

    You do not have to have Red Hat Ansible Engine installed on the systems on which you want to deploy the logging solution.

  • You have the rhel-system-roles package on the system from which you want to run the playbook.

    Note

    You do not have to have rsyslog installed, because the system role installs rsyslog when deployed.

  • You have an inventory file listing the systems on which you want to configure the logging solution.

Procedure

  1. Create a playbook that defines the required role:

    1. Create a new YAML file and open it in a text editor, for example: 

      # vi logging-playbook.yml

    2. Insert the following content: 

      ---
- name: Deploying basics input and implicit files output
  hosts: all
  roles:
    - linux-system-roles.logging
  vars:
    logging_inputs:
      - name: system_input
        type: basics
    logging_outputs:
      - name: files_output
        type: files
    logging_flows:
      - name: flow1
        inputs: [system_input]
        outputs: [files_output]

  2. Execute the playbook on a specific inventory: 

    # ansible-playbook -i inventory-file /path/to/file/logging-playbook.yml

    Where:

    • inventory-file is the inventory file.
    • logging-playbook.yml is the playbook you use.

Verification

  1. Test the syntax of the /etc/rsyslog.conf file: 

    # rsyslogd -N 1
rsyslogd: version 8.1911.0-6.el8, config validation run (level 1), master config /etc/rsyslog.conf
rsyslogd: End of config validation run. Bye.

  2. Verify that the system sends messages to the log:

    1. Send a test message: 

      # logger test

    2. View the /var/log/messages log, for example: 

      # cat /var/log/messages
Aug  5 13:48:31 hostname root[6778]: test

      Where `hostname` is the host name of the client system. Note that the log contains the user name of the user that entered the logger command, in this case root.

6.4. Applying a remote logging solution using the Logging System Role

Follow these steps to prepare and apply a Red Hat Ansible Engine playbook to configure a remote logging solution. In this playbook, one or more clients take logs from systemd-journal and forward them to a remote server. The server receives remote input from remote_rsyslog and remote_files and outputs the logs to local files in directories named by remote host names.

Prerequisites

  • You have Red Hat Ansible Engine installed on the system from which you want to run the playbook.

    Note

    You do not have to have Red Hat Ansible Engine installed on the systems on which you want to deploy the logging solution.

  • You have the rhel-system-roles package on the system from which you want to run the playbook.

    Note

    You do not have to have rsyslog installed, because the system role installs rsyslog when deployed.

  • You have at least two systems:

    • At least one will be the logging server.
    • At least one will be the logging client.

Procedure

  1. Create a playbook that defines the required role:

    1. Create a new YAML file and open it in a text editor, for example: 

      # vi logging-playbook.yml

    2. Insert the following content into the file: 

      ---
- name: Deploying remote input and remote_files output
  hosts: server
  roles:
    - linux-system-roles.logging
  vars:
    logging_inputs:
      - name: remote_udp_input
        type: remote
        udp_ports: [ 601 ]
      - name: remote_tcp_input
        type: remote
        tcp_ports: [ 601 ]
    logging_outputs:
      - name: remote_files_output
        type: remote_files
    logging_flows:
      - name: flow_0
        inputs: [remote_udp_input, remote_tcp_input]
        outputs: [remote_files_output]

- name: Deploying basics input and forwards output
  hosts: clients
  roles:
    - linux-system-roles.logging
  vars:
    logging_inputs:
      - name: basic_input
        type: basics
    logging_outputs:
      - name: forward_output0
        type: forwards
        severity: info
        target: host1.example.com
        udp_port: 601
      - name: forward_output1
        type: forwards
        facility: mail
        target: host1.example.com
        tcp_port: 601
    logging_flows:
      - name: flows0
        inputs: [basic_input]
        outputs: [forward_output0, forward_output1]

[basic_input]
[forward_output0, forward_output1]

      Where host1.example.com is the logging server.

      Note

      You can modify the parameters in the playbook to fit your needs.

      Warning

      The logging solution works only with the ports defined in the SELinux policy of the server or client system and open in the firewall. The default SELinux policy includes ports 601, 514, 6514, 10514, and 20514. To use a different port, modify the SELinux policy on the client and server systems . Configuring the firewall through system roles is not yet supported.

  2. Create an inventory file that lists your servers and clients:

    1. Create a new file and open it in a text editor, for example: 

      # vi inventory.ini

    2. Insert the following content into the inventory file: 

      [servers]
server ansible_host=host1.example.com
[clients]
client ansible_host=host2.example.com

      Where: * host1.example.com is the logging server. * host2.example.com is the logging client.

  3. Execute the playbook on your inventory. 

    # ansible-playbook -i /path/to/file/inventory.ini /path/to/file/_logging-playbook.yml

    Where:

    • inventory.ini is the inventory file.
    • logging-playbook.yml is the playbook you created.

Verification steps

  1. On both the client and the server system, test the syntax of the /etc/rsyslog.conf file: 

    # rsyslogd -N 1
rsyslogd: version 8.1911.0-6.el8, config validation run (level 1), master config /etc/rsyslog.conf
rsyslogd: End of config validation run. Bye.

  2. Verify that the client system sends messages to the server:

    1. On the client system, send a test message: 

      # logger test

    2. On the server system, view the /var/log/messages log, for example: 

      # cat /var/log/messages
Aug  5 13:48:31 host2.example.com root[6778]: test

      Where host2.example.com is the host name of the client system. Note that the log contains the user name of the user that entered the logger command, in this case root.

Additional resources

6.5. Additional resources