Chapter 4. Red Hat Insights client data redaction

The Red Hat Insights client provides data redaction options. Depending on your version of RHEL, there are two methods for controlling data redaction.

Table 4.1. Data redaction and RHEL versions

RHEL VersionRedaction method

RHEL 6.9, 7.8, 8.2, and earlier

Configuration file

remove.conf

RHEL RHEL 6.10, 7.9, 8.3 and later

YAM files

file-redaction.yaml

file-content-redaction.yaml

You must create the remove.conf configuration file or YAML files. They are not installed by default.

4.1. Configuring Red Hat Insights client redaction

The Red Hat Insights client provides data redaction options. Depending on your version of RHEL, there are two methods for controlling data redaction.

4.2. Redaction and remove.conf file use

When you use a configuration file, redaction is controlled by the contents of /etc/insights-client/remove.conf. You can optionally configure the Insights client to use a different redaction configuration file.

Based on your entries in the redaction configuration file, you can specify one or more of the following actions:

  • Eliminate specific files and their content from data collecting
  • Eliminate selected command output from data collecting
  • Eliminate information that matches a pattern
  • Substitute specific strings with a default keyword string

When you configure redaction by elimination, the redacted information is never recorded in the archive file. Redaction is performed by preprocessing the data before it is captured in the archive file.

For redaction by string substitution, the archive file is processed by a Python SoS process before it is sent to Red Hat Insights.

NOTE
Regular expression matching is not supported by the remove.conf file.

You can use command line options to control the archive file output. For example, you can generate the archive file but not send it to Red Hat Insights. You can inspect and verify the redaction results before the archive is sent .

NOTE
When you redact files and command output, that information is not available to compare against the Insights rules. These omissions might cause Insights not to identify issues that apply to your system.

4.3. Configuring Red Hat Insights client redaction using remove.conf

The /etc/insights-client/remove.conf file controls redaction. You must create this file before you can use Insights client redaction.

Procedure

  1. Use an editor to create the /etc/insights-client/remove.conf file template.

    [remove]
    files=/etc/cluster/cluster.conf,/etc/hosts
    commands=/bin/dmesg,/bin/hostname
    patterns=password,username
    keywords=super$ecret,ultra$ecret+
  2. Optionally, delete any lines that you do not want to apply to archive redaction.
  3. Make sure the remove.conf file permissions are set for root owner only.

    [root@insights]# ll remove.conf
    -rw-------. 1 root root 145 Sep 25 17:39 remove.conf
  4. Refer to the additional resources for procedures on how to apply each available redaction option.

4.3.1. Redacting specific file content

You can select specific files that are redacted by using the remove.conf file. The files you select and their content are not included in the archive file.

Prerequisites

Procedure

  1. Use an editor and open the /etc/insights-client/remove.conf file.

    [remove]
    files=/etc/cluster/cluster.conf,/etc/hosts
    commands=/bin/dmesg,/bin/hostname
    patterns=password,username
    keywords=super$ecret,ultra$ecret+
  2. On the files= line, add or remove the files that you want to redact from the archive file.

    Note

    Each file name is separated by a single comma. Do not use spaces.

  3. To redact no files from the Insights client archive, remove the files= line.
  4. Save and close the file.

4.3.2. Redacting specific commands

You can select specific commands that are redacted by using the remove.conf file. The output of these commands is not included in the archive file.

Prerequisites

Procedure

  1. Use an editor and open the /etc/insights-client/remove.conf file.

    [remove]
    files=/etc/cluster/cluster.conf,/etc/hosts
    commands=/bin/dmesg,/bin/hostname
    patterns=password,username
    keywords=super$ecret,ultra$ecret+
  2. On the commands= line, add or remove the commands that you want to redact from the archive file.

    Note

    Each command name is separated by a single comma. Do not use spaces.

  3. To redact no command from the Insights client archive, remove the command= line.
  4. Save and close the file.

4.3.3. Redacting string patterns

You can select specific string patterns that are redacted by using the remove.conf file. The string pattern that you specify is redacted from the archive file by removing the entire line. For example, if the string pattern is name, that pattern matches and redacts hostname, filename, username.

Note

Regular expressions and wildcard matching (egrep) are not supported.

Prerequisites

Procedure

  1. Use an editor and open the /etc/insights-client/remove.conf file.

    [remove]
    files=/etc/cluster/cluster.conf,/etc/hosts
    commands=/bin/dmesg,/bin/hostname
    patterns=password,username
    keywords=super$ecret,ultra$ecret+
  2. On the patterns= line, add any string patterns that you want to redact from the archive file.

    Note

    Each pattern is separated by a single comma. Do not use spaces.

  3. To redact no patterns from the Insights client archive, remove the patterns= line.
  4. Save and close the file.

4.3.4. Redacting keywords

You can select specific keywords that are redacted by using the remove.conf file. The keywords you specify are replaced with keyword0, keyword1, keyword2, etc., in the archive file.

Prerequisites

Procedure

  1. Use an editor and open the /etc/insights-client/remove.conf file.

    [remove]
    files=/etc/cluster/cluster.conf,/etc/hosts
    commands=/bin/dmesg,/bin/hostname
    patterns=password,username
    keywords=super$ecret,ultra$ecret+
  2. On the keywords= line, add any keywords that you want to redact from the archive file.

    Note

    Each keyword is separated by a single comma. Do not use spaces.

  3. To redact no keywords from the Insights client archive, remove the keyword= line.
  4. Save and close the file.

4.3.5. Validating the remove.conf file

You can validate the remove.conf file to make sure its syntax is correct before using it for redaction.

Prerequisites

Procedure

  1. Enter the insights-client command with the --validate option.

    [root@insights]# insights-client --validate
  2. Correct any errors that the command displays.

4.4. Redaction and YAML file use

When you use YAML files for redaction, two files control the redaction actions. You can use one or both files, depending on the content you want to redact. The specified content is redacted before it is captured in the archive file.

Table 4.2. Redaction and YAML files

YAML fileDescription

/etc/insights-client/file-redaction.yaml

This file lists commands and files that you want redacted. The output of the listed commands or files is readacted.

/etc/insights-client/file-content-redaction.yaml

This file defines pattern redaction and keyword replacement. Pattern redaction is done by pattern match or regular expression match. Keyword replacement is done by a Python SoS process that replaces the keyword with a generic identifier.

4.5. Configuring Red Hat Insights client redaction using YAML files

Two YAML files control Insights client redaction. You must create each YAML file before you can use redaction in RHEL 6.10, 7.9, 8.3 and later.

4.5.1. Configuring YAML command and file redaction

The /etc/insights-client/file-redaction.yaml file is a YAML file. It lists the commands and system files that you want redacted. The output of the listed commands or files is not included in the uploaded archive file.

If you want to redact based on keyword replacement or pattern matching, see Section 4.5.2, “Configuring YAML pattern and keyword redaction”.

Prerequisites

  • You must be familiar with the basics of YAML syntax. Explaining YAML is beyond the scope of this procedure.
  • You must have root permission or its equivalent to create files in /etc/insights-client/

Procedure

  1. Use an editor to create the /etc/insights-client/file-redaction.yaml file.

    Example

    # file-redaction.yaml
    ---
    # Exclude the entire output of commands
    #   Specify the full command path or the symbolic name in .cache.json
    
     commands:
    - /bin/rpm -qa
    - /bin/ls
    - ethtool_i
    
    # Exclude the entire output of files
    #  Specify the full filename path or the symbolic name in .cache.json
    
    files:
    - /etc/audit/auditd.conf
    - cluster_conf

  2. Make sure the file-redaction.yaml file permissions are set for root owner only.

    [root@insights]# ll file-redaction.yaml
    -rw-------. 1 root root 145 Sep 25 17:39 file-redaction.yaml

4.5.2. Configuring YAML pattern and keyword redaction

The /etc/insights-client/file-content-redaction.yaml file is a YAML file that defines redaction based on pattern redaction and keyword replacement. Pattern redaction is done by pattern match or regular expression match. Keyword replacement is done by a Python SoS process that replaces the keyword with a generic identifier.

If you want to redact based on command output or specific files, see Section 4.5.1, “Configuring YAML command and file redaction”.

Prerequisites

  • You must be familiar with the basics of YAML syntax. Explaining YAML is beyond the scope of this procedure.
  • You must have root permission or its equivalent to create files in /etc/insights-client/

Procedure

  1. Use an editor to create the /etc/insights-client/file-content-redaction.yaml file.

    Example

    # file-content-redaction.yaml
    ---
    # Pattern redaction per matching line
    #  Lines that match a pattern are excluded from files and command output.
    #  Patterns are processed in the order that they are listed.
    # Example
    
    patterns:
     - "a_string_1"
     - "a_string_2"
    
    # Regular expression pattern redaction per line
    #  Patterns with regular expressions (regex) are wrapped with "regex:"
    # Example
    
    patterns:
     regex:
     - "abc.*def"
     - "localhost[[:digit:]]"
    
    
    # Keyword replacement redaction
    #  Replace keywords in files and command output with generic identifiers
    #  Keyword does not support regex
    # Example
    
    keywords:keywords:
    - "1.1.1.1"
    - "My Name"
    - "a_name"

  2. Make sure the file-content-redaction.yaml file permissions are set for root owner only.

    [root@insights]# ll file-content-redaction.yaml
    -rw-------. 1 root root 145 Sep 25 17:39 file-content-redaction.yaml

4.6. Verifying the Insights client archive

You can verify the contents of the archive file. By inspecting the archive file, you can confirm what data is sent to Red Hat Insights.

4.6.1. Verifying the archive before upload

You can inspect the archive before it is sent to Red Hat Insights by running the client and saving the file without uploading it. This allows you to view what information the client sends to Insights, and to verify obfuscation or redaction settings.

The archive is stored in the /var/tmp/ directory. The file name is displayed when insights-client completes.

Prerequisites

Procedure

  1. Enter the insights-client command with the --no-upload option.

    [root@insights]# insights-client --no-upload

    The command displays informational messages when redaction or obfuscation is applied.

    WARNING: Excluding data from files
    Starting to collect Insights data for ITC-4
    WARNING: Skipping patterns found in remove.conf
    WARNING: Skipping command /bin/dmesg
    WARNING: Skipping command /bin/hostname
    WARNING: Skipping file /etc/cluster/cluster.conf
    WARNING: Skipping file /etc/hosts
    Archive saved at /var/tmp/qsINM9/insights-ITC-4-20190925180232.tar.gz
  2. Navigate to the temporary storage directory as shown in the Archive saved at message.

    [root@insights]# cd /var/tmp/qsINM9/
  3. Unpack the compressed tar.gz file.

    [root@insights]# tar -xzf insights-ITC-4-20190925180232.tar.gz

    The result will be a new directory containing the files.

4.6.2. Verifying the Insights client archive after upload

You can keep the archive for inspection after it is sent to Red Hat Insights by running the client and saving the file. This allows you to verify what information the client sends Insights, and to verify obfuscation or redaction settings.

Prerequisites

Procedure

  1. Enter the insights-client command with the --keep-archive option.

    [root@insights]# insights-client --keep-archive

    The command displays informational messages.

    Starting to collect Insights data for ITC-4
    Uploading Insights data.
    Successfully uploaded report from ITC-4 to account 6229994.
    Insights archive retained in /var/tmp/ozM8bY/insights-ITC-4-20190925181622.tar.gz
  2. Navigate to the temporary storage directory as shown in the Insights archive retained in message.

    [root@insights]# cd /var/tmp/ozM8bY/
  3. Unpack the compressed tar.gz file.

    [root@insights]# tar -xzf insights-ITC-4-20190925181622.tar.gz

    The result will be a new directory containing the files.