Comparing System Configurations and Baselines in Red Hat Insights Inventory

Red Hat Insights 2020-10

An overview of the System Comparison service and baselines for drift analysis

Red Hat Customer Content Services

Abstract

Compare system configurations of a system over time, or to other systems and baselines to identify discrepancies in your environment, and perform drift analysis.
Providing Feedback:
If you have a suggestion to improve this document or find an error, submit a Bugzilla report at http://bugzilla.redhat.com against Cloud Software Services (cloud.redhat.com) for the System Comparison component.

Chapter 1. Overview

The System Comparison service enables you to compare the system configuration of one system to other systems in your Red Hat Insights inventory. You can also compare the system configurations over time to track and analyze changes. In addition, you can set a reference point and compare all systems against that reference. If needed, you can use any of these comparisons to troubleshoot systems at various points in time throughout their use.

When comparing across different systems, you can filter the displayed profile facts to highlight the ones that match, those that are different, or where information is missing or considered problematic.

You can also define and manage baselines. You can use defined Insights baselines to compare baselines and system configurations. In addition, you can generate CSV output of the systems and baselines you are comparing.

Chapter 2. Installing the System Comparison Service

The System Comparison service uses the Insights client and does not require any additional tool to install or command to run. The System Comparison service enables users to compare systems registered in their Insights inventory.

Note

For steps on how to install the Insights client and register your systems to Red Hat Insights, see Get Started.

Chapter 3. Accessing the System and Baselines Comparison Service

The System Comparison service is part of Red Hat Insights. Access this service via https://cloud.redhat.com.

Procedure

  1. On https://cloud.redhat.com, in the Red Hat Insights panel, click Drift. The Comparison screen opens.

    drift comparison screen

  2. Cick Add to comparison. The Add to comparison screen opens, where you can add systems or baselines to compare.

    drift add to comparison screen

    • On the Systems tab, you can add any systems you want to compare. This screen also lists any baselines that already exist in your Insights inventory.
    • On the Baselines tab, you can create baselines. This screen also lists any baselines that already exist in your inventory.

      drift baselines screen

  3. Click Submit to add your systems or baselines for comparison.

Once you add them, you can start comparing the facts of systems and baselines in your Insights inventory.

Chapter 4. Adding Systems Manually within System Comparison

Within the System Comparison service, you can add systems registered in your Insights inventory.

Procedure

  1. On the Insights user interface, click DriftComparison in the left-side navigation menu. The Comparison service opens to the Comparison screen.
  2. Click Add to comparison.
  3. Select the systems to compare from the list. Alternatively, enter the system name in the search box to find by name, and then select the system.
  4. Click Submit.
Note
  • At any time, you can add more systems by clicking the Add to comparison button near the top of the Comparison screen.
  • Similarly, you can remove a particular system under comparison by clicking the X symbol on the upper-right corner of the individual system name, or you can remove all systems under comparison by clicking the options menu drift options menu located at the top, then clicking Clear all comparisons to start again.

Chapter 5. Comparing System Profiles

After you have added all the systems you want to compare, use the Comparison screen to compare the facts in the systems.

Procedure

  1. At https://cloud.redhat.com, in the Red Hat Insights panel, click Drift.
  2. In the left-side navigation menu, click Drift and select Comparison.
  3. On the Comparison screen, click Add to comparison.
  4. On the Add to Comparison screen, select the Systems tab to see the systems that are checked into your Insights inventory.
  5. In the Name column, select the checkbox for two or more systems and click Submit. The Comparison screen opens, showing the state of facts in the systems.
  6. In the View drop-down list, select Same, Different, and/or Incomplete data to choose which filtered facts the system displays. Fact values are shown in different colors. The red drift red examine icon icon indicates an issue you should examine, and the green drift good expected icon denotes an expected state or value. When the expected state is unknown, the application shows the state in black, marked with the drift question mark icon.
  7. In the Filter by fact column, click drift options menu to clear all comparisons, and in the State area, you can dismiss one or two of the fact filters, or click Clear filters to clear all filters.
  8. Click Add to comparison to add additional systems or a baseline for comparison. In the Systems or Baselines column, select the systems or baselines you want to add, and click Submit. The screen displays the systems you selected in columns. At the top of each column is the system name and date the system data was captured. Below are the values for each fact listed in the Fact column.
  9. Click drift export as csv icon to export the report as a CSV file.

Chapter 6. Comparing System Profiles Over Time

Insights provides the ability to inspect system profiles and your RHEL configuration. With the addition of the Drift service, you can select and view system profiles over time and use them in your comparisons. By examining the profiles, you can understand system characteristics such as:

  • system hardware and any changes associated with it
  • system configuration changes that may have occurred
  • applied updates you may need to validate
  • operational issues you may need to troubleshoot

6.1. Viewing a History of System Profiles

Every time you submit a system for comparison, the submission checks in the profile and marks it with a time stamp. By examining the different profile versions, you can see a view of the system over time.

Note

On the Choose systems screen, if a system is marked with a time stamp drift clock icon icon, you may be able to open that system directly. Otherwise, add the system you want to compare into Drift, as explained in Comparing System Profiles before following step 1 below.

Procedure

  1. On the Choose systems screen, in the Systems tab, select the check box for the systems you want to compare and click Submit.
  2. On the Comparison screen, the box at the top of the screen identifies the system by its system name. Below the name is the date and time the version was checked in.
  3. Next to the time stamp, click the clock drift clock icon icon to open the drop-down list of profiles that have been checked in. The most recent profile is listed first.
  4. Select the check box for the checked in profiles you want to examine and click Compare.

    When you have selected all profiles you want to examine, you are ready to view fact values to understand how the system has changed over time. You can also determine if there are system facts that need your attention. For information about system facts, see Accessing and Using System Facts.

Chapter 7. Comparing Systems to a Single Reference Point

At times, you may want to compare all systems to a single reference point rather than compare all systems with each other, or compare them as a group. For example, you may need to compare all systems to a baseline, so that all systems are calculated against that baseline. You can also compare a system against time-stamped profiles to understand where and when changes have occurred.

You might also want to invert a comparison. For example, instead of comparing profiles with old time stamps against the latest profile, you may want to compare all profiles against the oldest known working version of a system. Such a comparison enables you to identify changes that deviate from the reference point.

Procedure

  • On the Comparison screen, click the drift set reference point icon icon on the system header.

The System Comparison service compares each of your selected systems with the fact values in the reference system, displayed in the first position column, with its header highlighted in blue.

Drift displays the fact values with every difference highlighted in red.

Note

A fact category with multiple values has no highlights until you expand the fact to view details. When you do so, Drift shows the specific fact details highlighted in red.

Chapter 8. Accessing and Using System Facts

A set of system facts defines system configurations and baselines. System facts are name/value pair properties that contain details about configuration items, such as packages and network interface settings of a system. You can access, filter, and use system facts as part of your system configuration and baseline comparison.

Comparison states based on observed fact values provide guidance in managing your system. The application indicates facts whose behavior differs from expectations, identifies facts whose state is unknown, and alerts users to facts that require attention.

The application displays the observed fact values in different colors. A red drift red examine icon icon indicates an issue you should examine, and a green drift good expected icon icon denotes an expected state or value. When the expected state is unknown, the application shows the state in black, marked with the drift question mark icon.

Some facts are system-specific and are considered unique. Their state is marked in red if values are equal for a given comparison. This is the case for fqdn and IP addresses. When marked in red, these facts require your attention.

The application expects other facts, such as last_boot_time, to be different for all compared systems. For such facts, it does not highlight the differences, and it marks the comparison state as unknown (no opinion).

8.1. Available System Facts and Their Functions

Table 8.1. System Facts

Fact NameDescriptionExample Value

arch

System architecture

x86_64

bios_release_date

BIOS release date; typically MM/DD/YYYY

01/01/2011

bios_vendor

BIOS vendor name

LENOVO

bios_version

BIOS version

1.17.0

cloud_provider

Cloud vendor. Values are google, azure, aws, alibaba, or empty

google

cores_per_socket

Number of CPU cores per socket

2

cpu_flags

Category with a list of CPU flags. Each name is the CPU flag (ex: vmx), and the value is always enabled.

vmx, with a value of enabled.

enabled_services

Category with a list of enabled services. Each name in the category is the service name (ex: crond), and the value is always enabled .

crond, with a value of enabled.

fqdn

System Fully Qualified Domain Name

system1.example.com

infrastructure_type

System infrastructure; common values are virtual or physical

virtual

infrastructure_vendor

Infrastructure vendor; common values are kvm, vmware, baremetal, etc.

kvm

installed_packages

List of installed RPM packages. This is a category.

bash, with a value of 4.2.46-33.el7.x86_64.

installed_services

Category with a list of installed services. Each name in the category is the service name (ex: crond), and the value is always installed.

crond, with a value of installed.

kernel_modules

List of kernel modules. Each name in the category is the kernel module (ex: nfs), and the value is enabled.

nfs, with a value of enabled.

last_boot_time

The boot time in YYYY-MM-DDTHH:MM:SS format. Informational only; we do not compare boot times across systems.

2019-09-18T16:54:56

network_interfaces

List of facts related to network interfaces.

 
 

There are six facts for each interface: ipv6_addresses, ipv4_addresses, mac_address, mtu, state and type. The two address fields are comma-separated lists of IP addresses. The state field is either UP or DOWN. The type field is the interface type (ex: ether, loopback, bridge, etc.).

 
 

Each interface (ex: lo, em1, etc) is prefixed to the fact name. For example, em1’s mac address would be the fact named em1.mac_address.

 
 

Most network interface facts are compared to ensure they are equal across systems. However, ipv4_addresses, ipv6_addresses, and mac_address are checked to ensure they are different across systems. A subexception for lo should always have the same IP and mac address on all systems.

 

number_of_cpus

Total number of CPUs

1

number_of_sockets

Total number of sockets

1

os_kernel_version

Kernel version

4.18.0

os_release

Kernel release

8.1

running_processes

List of running processes. The fact name is the name of the process, and the value is the instance count.

crond, with a value of 1.

sap_instance_number

SAP instance number

42

sap_sids

SAP system ID (SID)

A42

sap_system

Boolean field that indicates if SAP is installed on the system

True

sap_version

SAP version number

2.00.052.00.1599235305

satellite_managed

Boolean field that indicates is a system is registered to a Satellite server.

FALSE

selinux_current_mode

Current SELinux mode

enforcing

selinux_config_file

SELinux mode set in the config file

enforcing

system_memory

Total system memory in human-readable form

3.45 GiB

tuned_profile

Current profile resulting from the command tuned-adm active

desktop

yum_repos

List of yum repositories. The repository name is added to the beginning of the fact. Each repository has the associated facts base_url,enabled, and gpgcheck.

Red Hat Enterprise Linux 7 Server (RPMs).base_url would have the value https://cdn.redhat.com/content/dist/rhel/server/7/$releasever/$basearch/os

8.2. Filtering System Facts

You can filter system facts in several ways: by the fact comparison state, by fact name, and/or by fact category.

Filtering by comparison state

Click the View drop-down list and select Same to show only the facts for which values are the same, Different to show only the facts that are different, or Incomplete Data to show only the facts where the information is incomplete. You can also select a combination of Same, Different, and Incomplete data states, and clear selections as necessary. When you first add systems for comparison, all three options are selected by default.

filter view options

Filtering by fact name
Enter the fact name in the search box at the top to filter by specific fact name. For example, entering kernel as a filter displays all facts containing kernel in their name. Or enter installed_packages in the search box to view all packages.
Filtering by fact category
Enter a fact category in the search box to compare systems by that category. Examples include installed_packages, installed_services, kernel_modules, network_interfaces, yum_repos, cpu_flags, and enabled_services.

In the following example screen, you can see the system comparison data filtered by facts that show a difference across systems. Some facts, such as the fqdn, are expected to be different for each system, but the installed packages are expected to be the same. Over time, some packages have been upgraded on system 1, but not on system 2 and system 3, as shown by expanding the fact category installed_packages.

drift comparison with diffs screen

8.3. Sorting System Facts

You can sort system facts alphabetically. Click the arrow next to Fact ( fact sort1 ) to switch sorting between ascending and descending order. Note that facts are shown in ascending order by default. You can also sort system facts by the comparison State. Click the arrow next to State ( drift state sort ) to switch sorting by state.

Note

Sorting works in combination with any applied filters. That is, if you have filtered for installed packages or viewing facts by comparison state, the filtered data can be sorted alphabetically or by comparison state.

Chapter 9. Exporting System Comparison Output

The Comparison service allows you to export system comparison output, along with any current selections such as filters, to a comma-separated values (CSV) file.

Note

The exported CSV report preserves all your current selections on the system comparison output, including any filters applied. That is, it follows the WYSIWYG (What You See Is What You Get) paradigm. Therefore, you will need to expand any nested fact categories (installed_packages, for example) to be exported in the report.

Procedure

  1. On the Comparison output screen for two or more systems, click the Export as CSV icon ( drift export as csv icon ), and then select either Export as CSV, or Clear all comparisons.

    export csv1

  2. Open the CSV file with the tool of your choice so that you can easily compare exported facts and analyze discrepancies in systems.

Chapter 10. Reviewing and Managing Baselines

Baselines are configurations (sets of name / value facts) that can be created from scratch, as a copy of an existing baseline, a copy of an existing system configuration, or a copy of a historical system profile. You use defined baselines in the Comparison service to compare baselines and system configurations.

You can manage system profile baseline definitions for your organization. That is, you can edit baselines by changing values on facts, or by deleting them. You can use the Insights user interface to create, copy, edit, delete, and export baselines.  

Note

You can also use the baselines API to create and manage baselines. Refer to the System Comparison API documentation for information about using the REST API to query and edit baselines. See https://access.redhat.com/documentation/en-us/red_hat_insights/2020-10/html/system_comparison_api_documentation/index for information.

10.1. Creating Baselines

Note

When creating baselines, you can use a variety of system facts. For information about system facts, see Accessing and Using System Facts.

Choose the method you want to use to create a new baseline. You can:

  • Create a baseline from scratch
  • Copy an existing baseline
  • Copy an existing system
  • Copy an historical system profile

10.1.1. Creating a New Baseline from Scratch

Procedure

  1. On the Insights GUI, click DriftBaselines in the left-side navigation menu. The Baselines screen opens.
  2. Click Create baseline. The Create baseline screen opens.

    drift create new baseline

  3. Select Create baseline from scratch.
  4. In the Baseline name field, enter a name for the baseline you want to create.
  5. Click Create baseline. The Edit screen for the new baseline opens.
  6. Click Add fact or category. The Fact add/edit screen for the new baseline opens.

    drift add fact

  7. Enter the Fact name and Value, then click Save.

    Note

    If this is a category (parent fact) under which you will add sub fact(s), select This is a category and only enter the Category name. Then click Save.

    drift add category

  8. To add sub facts under a category, click the more options menu drift options menu next to a category and select Add sub fact.
  9. You can also edit a fact, sub fact, or category using the more options menu.
Note

Any changes performed are saved automatically. Therefore, once you have completed adding all facts, sub facts and their values, you can view and manage the baseline you have created by navigating to DriftBaselines.

10.1.2. Copying an Existing Baseline to Create a New Baseline

You can copy an existing baseline to create a new baseline.

Procedure

  1. On the Insights user interface, click DriftBaselines in the left-side navigation menu. The Baselines screen opens.
  2. Click Create baseline. The Create baseline screen opens.
  3. Select Copy an existing baseline.
  4. In the Baseline name field, enter a name for the new baseline.
  5. From the list of existing baselines, choose the baseline you want to copy from and click Create baseline. The screen that opens has the newly created baseline with all facts populated.
  6. Edit or delete existing facts using the more options menu drift options menu next to a fact.
  7. You can also add a new fact or category by clicking Add fact. Similarly, you can add a sub-fact to a category.
Note

Based on typical expectations about fact behavior, Drift alerts users to facts that may require attention. It does not flag facts that differ from each other if the differences are expected or are of no importance. It flags only important differences or similarities that may be problematic. You can then address these exceptions to the typical fact behavior.

Once you have completed adding or editing facts, sub facts and their values, you can view and manage the baseline you have created by navigating to DriftBaselines.

10.1.3. Copying an Existing System to Create a New Baseline

You can copy an existing system to create a new baseline.

Procedure

  1. On the Insights user interface, click DriftBaselines in the left-side navigation menu. The Baselines screen opens.
  2. Click Create baseline. The Create baseline screen opens.
  3. Select Copy an existing system.
  4. In the Baseline name field, enter a name for the new baseline.
  5. From the existing list of systems, select the system you want to copy from and click Create baseline. The screen that opens has the newly created baseline with all facts populated.
  6. Edit or delete existing facts using the more options menu drift options menu next to a fact.
  7. You can also add a new fact or category by clicking Add fact. Similarly, you can add a sub fact to a category.
Note

Based on typical expectations about fact behavior, Drift alerts users to facts that may require attention. It does not flag facts that differ from each other if the differences are expected or are of no importance. It flags only important differences or similarities that may be problematic. You can then address these exceptions to the typical fact behavior in a baseline.

Once you have completed adding or editing facts, sub facts and their values, you can view and manage the baseline you have created by navigating to DriftBaselines.

10.1.4. Copying an Historical System Profile to Create a New Baseline

You can copy an historical system profile to create a new baseline.

Procedure

  1. On the Insights user interface, click DriftBaselines in the left-side navigation menu. The Baselines screen opens.
  2. Click Create baseline. The Create baseline screen opens.
  3. Select Copy an existing system.
  4. In the Baseline name field, enter a name for the new baseline.
  5. From the existing list of systems, select the system you want to copy from. In the Historical profile column, click the historical profile icon ( drift hsp icon ).
  6. From the drop-down menu, select the profile you want to copy, based on the date and time stamp. After selection, {ServiceName_short} displays the timestamp of the profile you selected under the name of the system in the Name column. If you change your mind and want to select a different historical profile, click the X next to the profile timestamp, and select a different historical profile.
  7. When you are satisfied that you have the particular profile you want to copy, click Create baseline. The screen that opens has the newly created baseline with all facts populated.
  8. Edit or delete existing facts using the more options menu drift options menu next to a fact.
  9. You can also add a new fact or category by clicking Add fact. Similarly, you can add a sub fact to a category.
Note

Based on typical expectations about fact behavior, Drift alerts users to facts that may require attention. It does not flag facts that differ from each other if the differences are expected or are of no importance. It flags only important differences or similarities that may be problematic. You can then address these exceptions to the typical fact behavior in a baseline.

Once you have completed adding or editing facts, sub facts and their values, you can view and manage the baseline you have created by navigating to DriftBaselines.

10.2. Editing or Deleting Baselines

You can edit baselines by renaming the baseline, and changing, adding, or deleting facts, categories, and sub facts.

10.2.1. Deleting a Baseline

Procedure

  1. On the Insights user interface, click DriftBaselines in the left-side navigation menu. The Baselines screen opens.
  2. Use the drop-down check box to select all or any baselines. When you make a selection, the drift num baselines field field indicates the number of baselines you have selected.
  3. To delete a baseline in the list, click Delete baselines on the more options menu drift options menu at the top of the screen. You can also select Delete from the more options menu drift options menu on the line that displays the named baseline you want to delete.

10.2.2. Editing a Baseline

Procedure

  1. To edit a baseline, click the baseline name. The edit screen for the chosen baseline opens.
  2. Click Add fact or category to add facts or categories to the baseline.
  3. To edit facts or categories, select the items you want to edit and click Edit fact on the more options on the right side of the line containing the fact. The Edit fact screen opens. Change the name and/or value and click Save. If the item is a category, the Edit category screen opens. Change the category name and slick Save.
  4. To edit a baseline by deleting facts or categories, select the items you want to delete and click Delete fact on the more options on the right side of the line containing the fact.
  5. For fact categories, click the more options menu on the right side of the line containing the fact category. Click to Add sub fact, Edit category, or Delete category, and the appropriate screen opens. Provide or edit the information requested and click Save.
  6. If you chose to delete facts or a category, a popup window opens, asking you to confirm that you want to delete the facts or categories. Click to confirm your action, or click Cancel if you decide not to make the change.

10.3. Exporting Baselines

You can export all system baselines or individual baseline facts or categories to a CSV file and analyze them externally. Note that any search filters applied are preserved in the downloaded CSV file.

10.3.1. Exporting All System Baselines

Procedure

  1. Click DriftBaselines in the left-side navigation menu. The Baselines screen opens.
  2. Click the Export to CSV icon ( drift export as csv icon ) next to the Create baseline button.
  3. Click Export as CSV.

10.3.2. Exporting Individual Baselines, Facts, and Categories

Procedure

  1. Click DriftBaselines in the left-side navigation menu. The Baselines screen opens.
  2. From the list of baselines, select a baseline from the list.
  3. On the Edit screen for the baseline you selected, click the Export to CSV icon ( drift export as csv icon ) next to the Add fact or category button.
  4. Click Export as CSV.
Note

Use this same procedure to export individual facts and/or categories to CSV.

Chapter 11. Comparing System Configurations to Baselines

You can compare system configurations to one or more baselines to identify discrepancies in your environment and perform drift analysis.

Procedure

  1. On the Insights user interface, click Drift and select Comparison in the left-side navigation menu. The Comparison screen opens.
  2. Click Add to comparison.
  3. On the Systems tab, select one or more systems from the list. Alternatively, you can search for a system by name, then select the system.
  4. On the Baselines tab, select one or more baselines from the list. Alternatively, you can search for a baseline by name, then select the baseline.
  5. Click Submit.

At any time, you can add more systems and baselines by clicking the Add System button on the right side of the systems already added for comparison.

drift system baseline comparison

Similarly, to remove a particular system or baseline under comparison, click the X sign in the upper-right corner of the individual system or baseline name panel.

To remove all systems and baselines under comparison, click Clear all comparisons from the options menu drift options menu located at the top, and start again.

You can view the displayed comparison result and filter as necessary by fact name, comparison state, and category. To export the result, along with any current selections such as filters, to a comma-separated values (CSV) file for further analysis, click the Export as CSV option in the menu.

Legal Notice

Copyright © 2020 Red Hat, Inc.
The text of and illustrations in this document are licensed by Red Hat under a Creative Commons Attribution–Share Alike 3.0 Unported license ("CC-BY-SA"). An explanation of CC-BY-SA is available at http://creativecommons.org/licenses/by-sa/3.0/. In accordance with CC-BY-SA, if you distribute this document or an adaptation of it, you must provide the URL for the original version.
Red Hat, as the licensor of this document, waives the right to enforce, and agrees not to assert, Section 4d of CC-BY-SA to the fullest extent permitted by applicable law.
Red Hat, Red Hat Enterprise Linux, the Shadowman logo, the Red Hat logo, JBoss, OpenShift, Fedora, the Infinity logo, and RHCE are trademarks of Red Hat, Inc., registered in the United States and other countries.
Linux® is the registered trademark of Linus Torvalds in the United States and other countries.
Java® is a registered trademark of Oracle and/or its affiliates.
XFS® is a trademark of Silicon Graphics International Corp. or its subsidiaries in the United States and/or other countries.
MySQL® is a registered trademark of MySQL AB in the United States, the European Union and other countries.
Node.js® is an official trademark of Joyent. Red Hat is not formally related to or endorsed by the official Joyent Node.js open source or commercial project.
The OpenStack® Word Mark and OpenStack logo are either registered trademarks/service marks or trademarks/service marks of the OpenStack Foundation, in the United States and other countries and are used with the OpenStack Foundation's permission. We are not affiliated with, endorsed or sponsored by the OpenStack Foundation, or the OpenStack community.
All other trademarks are the property of their respective owners.