Back in the Hood!!
After a crazy and exciting week of innovations in San Francisco, here I am again to tell you a bit more on how to customize Red Hat Insights to your needs!
To blacklist or not to blacklist, that is the question
As explained in my previous post Red Hat Insights 102, you can control the data Red Hat Insights sends to Red Hat servers, how data is sent, and when it is sent. But deviating from the default has its drawbacks too.
We want to provide our customers with the necessary options and controls to the data that Insights collects. However, with each modification to the default payload sent to Red Hat, you may adversely change the level of analysis Insights provides about your environment. Since Insights collects what is minimally needed for analysis. Each removal of data from the collection payload can impact our rules capabilities to detect the issue within your infrastructure.
Before we continue, I want to add a few words about the differences in commands required for RHEL 6 vs. RHEL 7 systems. This will affect many of the commands shared below. The name of the Red Hat Insights client has changed for RHEL 7.5+ but has not yet done so for RHEL 6.
For RHEL 7.5+ users, the name of the client (and all commands and locations that use the name of the client) is
insights-client. For older RHEL users, the name and other changes will come to RHEL 6 with the release of RHEL 6.10. For now though, RHEL 6 users should continue to use the legacy client name:
Blacklisting information, or how to deny access to certain data
Red Hat Insights collects metadata about the runtime configuration of a system (approximately 1% of what would be collected via sosreport during a support case) but this data can be customized, anonymized, blacklisted and obfuscated in the way that you need.
You can decide to exclude entire files, specific commands, specific patterns, and specific keywords from the data that is sent to Red Hat. To enable these exclusions, you simply need to create a file called
/etc/insights-client/remove.conf if you are already using RHEL 7.5) and specify this file in the
remove_file line of
/etc/insights-client/insights-client.conf if you are already using RHEL 7.5), as in the following example:
[remove] files=/etc/cluster/cluster.conf,/etc/hosts commands=/bin/dmesg patterns=password,username keywords=super$ecret,ultra$ecret
Let's dig into the directives from the example above.
files: A comma-separated list of files to be excluded. Each element in the list of files must be the absolute path to the file. To ensure exclusion, file names listed here must match exactly what is shown in the collection rules.
commands: A comma-separated list of commands that should not be executed and whose output should not be sent. To ensure exclusion, command names listed here must match exactly what is shown in the collection rules.
patterns: A comma-separated list of patterns that should not be sent.
keywords: A comma-separated list of keywords that should not be sent. Matching keywords will be replaced with the literal keyword. For this option to take effect, the
obfuscateoption must be set to
Patterns affect entire lines so any line that includes a matching pattern will not be sent.
As explained in my previous blog post, you can validate data to be sent with the
redhat-access-insights --no-upload command.
Let's get our hands dirty!
Let's take a look at what's collected and sent to Red Hat by the Red Hat Insights client using the default configuration.
To see what’s collected, without actually taking the action of collecting the data, we simply execute the agent with the
--no-upload option as follows:
[root@gherkin ~]# redhat-access-insights --no-upload Starting to collect Insights data See Insights data in /var/tmp/oLUbKq/insights-gherkin-20180521110933.tar.gz
To inspect what's being collected, we simply have to unzip the generated tar.gz file and dig into it, through commands and files:
[root@gherkin ~]# tar xvfz /var/tmp/oLUbKq/insights-gherkin-20180521110933.tar.gz [root@gherkin ~]# cd ./insights-gherkin-20180521110933/
We can see the hostname being collected:
[root@gherkin insights-gherkin-20180521110933]# cat insights_commands/hostname gherkin
NIC information is also collected:
[root@gherkin insights-gherkin-20180521110933]# cat insights_commands/ethtool_eno1 Settings for eno1: Supported ports: [ TP ] Supported link modes: 10baseT/Half 10baseT/Full 100baseT/Half 100baseT/Full 1000baseT/Full Supported pause frame use: No Supports auto-negotiation: Yes Advertised link modes: 10baseT/Half 10baseT/Full 100baseT/Half 100baseT/Full 1000baseT/Full Advertised pause frame use: No Advertised auto-negotiation: Yes Speed: 1000Mb/s Duplex: Full Port: Twisted Pair PHYAD: 1 Transceiver: internal Auto-negotiation: on MDI-X: off (auto) Supports Wake-on: pumbg Wake-on: g Current message level: 0x00000007 (7) drv probe link Link detected: yes
Kernel version information, architecture, and OS version is collected too:
[root@gherkin insights-gherkin-20180521110933]# cat insights_commands/uname_-a Linux gherkin 3.10.0-693.11.1.el7.x86_64 #1 SMP Fri Oct 27 05:39:05 EDT 2017 x86_64 x86_64 x86_64 GNU/Linux [root@gherkin insights-gherkin-20180521110933]# cat insights_commands/uptime 11:09:39 up 53 days, 10:27, 3 users, load average: 0.40, 1.24, 0.90
Filesystems currently mounted on our machine:
[root@gherkin insights-gherkin-20180521110933]# cat insights_commands/mount sysfs on /sys type sysfs (rw,nosuid,nodev,noexec,relatime) proc on /proc type proc (rw,nosuid,nodev,noexec,relatime) devtmpfs on /dev type devtmpfs (rw,nosuid,size=16323324k,nr_inodes=4080831,mode=755) securityfs on /sys/kernel/security type securityfs (rw,nosuid,nodev,noexec,relatime) tmpfs on /dev/shm type tmpfs (rw,nosuid,nodev) [...]
Insights also collects the hostnames and IPs our system knows about (these are my home LAN's machines):
[root@gherkin insights-gherkin-20180521110933]# cat etc/hosts 127.0.0.1 localhost localhost.localdomain localhost4 localhost4.localdomain4 ::1 localhost localhost.localdomain localhost6 localhost6.localdomain6 # Máquinas virtuales residentes en gherkin 192.168.100.250 ansible-tower ansible-tower.tortilla.org 192.168.100.189 ansible-node1 ansible-node1.tortilla.org 192.168.100.199 icg1 icg1.example.com 192.168.100.202 icg2 icg2.example.com 192.168.100.147 icg3 icg3.example.com 192.168.100.194 icg4 icg4.example.com
But what if I don't want to send all that data to Red Hat? Well, let's configure our host so it won't send the previous data.
First of all, we need to configure
/etc/redhat-access-insights/redhat-access-insights.conf to tell the Insights agent where the
remove.conf file to be used is located:
[root@gherkin ~]# vi /etc/redhat-access-insights/redhat-access-insights.conf [redhat-access-insights] remove_file=/etc/redhat-access-insights/remove.conf
Since we don't want to send our own machine name or IP to Red Hat, in the very same file, we need to specify to obfuscate them as follows:
# Obfuscate IP addresses obfuscate=True # Obfuscate hostname obfuscate_hostname=True
Another thing Red Hat Insights collects is the sshd configuration, for example, the access type:
[root@gherkin insights-gherkin-20180521110933]# cd etc/ssh/ [root@gherkin ssh]# grep -E '[P|p]assword' * #PermitEmptyPasswords no # the setting of "PermitRootLogin without-password".
Now it's the time to tune the
remove.conf file to our needs:
[root@gherkin ~]# vi /etc/redhat-access-insights/remove.conf [remove] files=/etc/hosts,/proc/cpuinfo commands=/bin/dmesg,/bin/mount,/bin/mount,/bin/uname -a,/sbin/ethtool,/usr/bin/uptime patterns=password,username
In this way, all the previous information won't be collected and sent. To verify it, we just need to re run the agent with
--no-upload option, and see its results:
[root@gherkin ~]# redhat-access-insights --no-upload WARNING: Excluding data from files Starting to collect Insights data WARNING: Skipping command /bin/dmesg WARNING: Skipping command /sbin/ethtool WARNING: Skipping command /bin/mount WARNING: Skipping command /bin/uname -a WARNING: Skipping command /usr/bin/uptime WARNING: Skipping file /proc/cpuinfo WARNING: Skipping file /etc/hosts See Insights data in /var/tmp/eJ9LoU/soscleaner-3125813513575232.tar.gz
By executing the agent, you can see the list of commands that will be skipped, as specified in the remove.conf file).
In addition, we'll inspect each of the previous commands and files from the
[root@gherkin soscleaner-3125813513575232]# cat insights_commands/hostname host0 [root@gherkin soscleaner-3125813513575232]# cat proc/cpuinfo cat: proc/cpuinfo: No such file or directory [root@gherkin soscleaner-3125813513575232]# cat etc/hosts cat: etc/hosts: No such file or directory [root@gherkin soscleaner-3125813513575232]# cat insights_commands/ethtool_eno1 cat: insights_commands/ethtool_eno1: No such file or directory [root@gherkin soscleaner-3125813513575232]# cat insights_commands/mount cat: insights_commands/mount: No such file or directory [root@gherkin soscleaner-3125813513575232]# cat insights_commands/uptime cat: insights_commands/uptime: No such file or directory
And finally, let's verify how patterns also work with the example of the sshd configuration:
[root@gherkin soscleaner-3125813513575232]# cd etc/ssh/ [root@gherkin ssh]# ls sshd_config [root@gherkin ~]# grep -E '[P|p]assword' sshd_config
Red Hat Insights has been designed to ensure that minimal, specific data is collected.
We encourage customers to check what is collected with the
--no-upload option prior to making modifications.
Find out more
If you want to know about the patterns and files and commands Insights collects data from as well as the the whole list of files and commands that are used to collect data from, you can take a look at this document.