Comparing System Configurations and Baselines in Cloud Management Services Inventory

Red Hat Insights 2020-04

An overview of the System Comparison service and baselines for drift analysis

Red Hat Customer Content Services

Abstract

Compare system configurations of a system over time, or to other systems and baselines to identify discrepancies in your environment, and perform drift analysis.
Providing Feedback:
If you have a suggestion to improve this document or find an error, submit a Bugzilla report at http://bugzilla.redhat.com against Cloud Software Services (cloud.redhat.com) for the System Comparison component.

Chapter 1. Overview

System Comparison enables you to compare the system configuration of one system to other systems in your cloud management services inventory. You can also compare the system configurations over time to track and analyze changes. In addition, you can set a reference point and compare all systems against that reference. If needed, you can use any of these comparisons to troubleshoot systems at various times throughout their use.

When comparing across different systems, you can filter the displayed profile facts to highlight the ones that match, those that are different, or where information is missing. You can also define and manage baselines. Once defined, you can use baselines in System Comparison to compare baselines and system configurations. In addition, you can generate CSV output of the systems and baselines you are comparing.

Chapter 2. Installing the System Comparison Service

The System Comparison service uses the insights-client and does not require any additional tool to install or command to run. Users are able to compare systems registered in cloud management services inventory.

Note

For steps on how to install the Insights client and register your systems to Red Hat Insights, see Get Started.

Chapter 3. Accessing the System and Baselines Comparison Service

The System Comparison service is part of Red Hat Insights and can be accessed via https://cloud.redhat.com.

Procedure

  1. On https://cloud.redhat.com, in the Cloud Management Services for Red Hat Enterprise Linux panel, click Drift Analysis. The Cloud Management Services dashboard opens.
  2. Click Drift in the left-side navigation menu to view the Drift options: Comparison and Baselines.

    • Click Add to comparison to open the Comparison screen where you can add systems or baselines to compare.
      drift comparison screen
    • Click Baselines to open the Baselines screen, where you can create baselines. This screen also lists any baselines that exist.

    drift baselines screen

You can now start adding systems and baselines to compare their facts.

Chapter 4. Adding Systems Manually within System Comparison

You can add systems registered in your cloud management services inventory within the System Comparison service.

Procedure

  1. On the Red Hat Insights user interface, click Drift AnalysisComparison in the left-side navigation menu.
  2. On the Comparison screen, click Add System.
  3. Select the systems to compare from the list. Alternatively, enter the system name in the search box to find by name, then select the system.
  4. Click Submit.
Note
  • At any time, you can add more systems by clicking on the Add System button on the right-side of the systems you have already added for comparison.
  • Similarly, you can remove a particular system under comparison by clicking the cross sign on the upper-right corner of the individual system name, or, you can remove all systems under comparison by clicking the options menu drift options menu located at the top, then clicking Clear all comparisons to start again.

Chapter 5. Comparing System Profiles

After you have added all the systems you want to compare, use the Comparison screen to compare the facts in the systems.

Procedure

  1. At https://cloud.redhat.com, in the Cloud Management Services for Red Hat Enterprise Linux panel, click Drift Analysis.
  2. Click Drift in the left-side navigation menu to view the Drift options: Comparison and Baselines.
  3. Click Comparison to open the Comparison screen where you can add systems or baselines to compare.
  4. Click Add systems or baselines.
  5. On the Choose systems screen, select the Systems tab to see the systems that are checked in.
  6. In the Name column, select the checkbox for two or more systems and click Submit. The Comparison screen opens, showing the state of facts in the systems.
  7. In the View drop-down list, select Same, Different, or Incomplete data to choose which facts are filtered and displayed. Fact values are shown in different colors. The red drift red examine icon icon indicates an issue you should examine, and green drift good expected icon denotes an expected state or value. When the expected state is unknown, the application shows the state in black, marked with the drift question mark icon.
  8. In the Filter by fact column, click drift options menu to clear all comparisons, and in the State area, you can dismiss one or two of the fact filters.
  9. Click Add systems or baselines to add additional systems, or add a baseline for comparison.
  10. Click drift export as csv icon to export the report as a CSV file.

The screen displays the systems you selected in columns. At the top of each column is the system name and date the system data was captured. Below are the values for each fact shown in the Fact column.

Chapter 6. Comparing System Profiles Over Time

Insights provides the ability to inspect system profiles and your RHEL configuration. With the addition of the Drift service, you can select and view system profiles over time and use them in your comparisons. By examining the profiles, you can understand system characteristics such as:

  • system hardware and any changes associated with it
  • system configuration changes that may have occurred
  • applied updates you may need to validate
  • operational issues you may need to troubleshoot

6.1. Viewing a History of System Profiles

Every time you submit a system for comparison, the submission checks in the profile and marks it with a time stamp. By examining the different profile versions, you can see the view of the system over time.

Note

On the Choose systems screen, if a system is marked with a time stamp drift clock icon icon, you may be able to open that system directly. Otherwise, add the system you want to compare into Drift, as explained in Comparing System Profiles before following step 1 below.

Procedure

  1. On the Choose systems screen, in the Systems tab, select the check box for the systems you want to compare and click Submit.
  2. On the Comparison screen, the box at the top of the screen identifies the system by its system name. Below the name is the date and time the version was checked in.
  3. Next to the time stamp, click the clock drift clock icon icon to open the drop-down list of profiles that have been checked in. The most recent profile is listed first.
  4. Select the check box for the checked in profiles you want to examine and click Compare.

    When you have selected all profiles you want to examine, you are ready to view fact values to understand how the system has changed over time. You can also determine if there are system facts that need your attention. For information about system facts, see Accessing and Using System Facts.

Chapter 7. Comparing Systems to a Single Reference Point

At times, you may want to compare all systems to a single reference point rather than compare all systems with each other, or compare them as a group. For example, you may need to compare all systems to a baseline, so that all systems are calculated against that baseline. You can also compare a system against time-stamped profiles to understand where and when changes have occurred.

You might also want to invert a comparison. For example, instead of comparing profiles with old time stamps against the latest profile, you may want to compare all profiles against the oldest known working version of a system. Such a comparison enables you to identify changes that deviate from the reference point.

Procedure

  • On the Comparison screen, click the drift set reference point icon icon on the system header.

The Drift service compares each of your selected systems with the fact values in the reference system, displayed in the first position column, with its header highlighted in blue.

Drift displays the fact values with every difference highlighted in red.

Note

A fact category with multiple values has no highlights until you expand the fact to view details. When you do so, Drift shows the specific fact details highlighted in red.

Chapter 8. Accessing and Using System Facts

A set of system facts defines system configurations and baselines. System facts are name/value pair properties that contain details about configuration items, such as packages and network interface settings of a system. You can access, filter, and use system facts as part of your system configuration and baseline comparison.

Comparison states based on observed fact values provide guidance in managing your system. The application indicates facts whose behavior differs from expectations, identifies facts whose state is unknown, and alerts users to facts that require attention.

The application displays the observed fact values in different colors. A red drift red examine icon icon indicates an issue you should examine, and a green drift good expected icon icon denotes an expected state or value. When the expected state is unknown, the application shows the state in black, marked with the drift question mark icon.

Some facts are system-specific and are considered unique. Their state is marked in red if values are equal for a given comparison. This is the case for fqdn and IP addresses. When marked in red, these facts require your attention.

The application expects other facts, such as last_boot_time, to be different for all compared systems. It does not highlight the differences, and it marks the comparison state as unknown (no opinion).

8.1. Available System Facts and Their Functions

Table 8.1. System Facts and Their Functions

Fact NameDescriptionExample Value

arch

System architecture

x86_64

bios_release_date

BIOS release date; typically MM/DD/YYYY

01/01/2011

bios_vendor

BIOS vendor name

LENOVO

bios_version

BIOS version

1.17.0

cloud_provider

Cloud vendor. Values are google, azure, aws, alibaba, or empty

google

cores_per_socket

Number of CPU cores per socket

2

cpu_flags

Category with a list of CPU flags. Each name is the CPU flag (ex: vmx), and the value is always enabled.

vmx, with a value of enabled.

enabled_services

Category with a list of enabled services. Each name in the category is the service name (ex: crond), and the value is always enabled .

crond, with a value of enabled.

fqdn

System Fully Qualified Domain Name

system1.example.com

infrastructure_type

System infrastructure; common values are virtual or physical

virtual

infrastructure_vendor

Infrastructure vendor; common values are kvm, vmware, baremetal, etc.

kvm

installed_packages

List of installed RPM packages. This is a category.

bash, with a value of 4.2.46-33.el7.x86_64.

installed_services

Category with a list of installed services. Each name in the category is the service name (ex: crond), and the value is always installed.

crond, with a value of installed.

kernel_modules

List of kernel modules. Each name in the category is the kernel module (ex: nfs), and the value is enabled.

nfs, with a value of enabled.

last_boot_time

The boot time in YYYY-MM-DDTHH:MM:SS format. Informational only; we do not compare boot times across systems.

2019-09-18T16:54:56

network_interfaces

List of facts related to network interfaces.

 
 

There are six facts for each interface: ipv6_addresses, ipv4_addresses, mac_address, mtu, state and type. The two address fields are comma-separated lists of IP addresses. The state field is either UP or DOWN. The type field is the interface type (ex: ether, loopback, bridge, etc.).

 
 

Each interface (ex: lo, em1, etc) is prefixed to the fact name. For example, em1’s mac address would be the fact named em1.mac_address.

 
 

Most network interface facts are compared to ensure they are equal across systems. However, ipv4_addresses, ipv6_addresses, and mac_address are checked to ensure they are different across systems. A subexception for lo should always have the same IP and mac address on all systems.

 

number_of_cpus

Total number of CPUs

1

number_of_sockets

Total number of sockets

1

os_kernel_version

Kernel version

4.18.0

os_release

Kernel release

8.1

running_processes

List of running processes. The fact name is the name of the process, and the value is the instance count.

crond, with a value of 1.

satellite_managed

Boolean field that indicates is a system is registered to a Satellite server.

FALSE

system_memory

Total system memory in human-readable form

3.45 GiB

8.2. Filtering System Facts

You can filter system facts in several ways: by the fact comparison state, by fact name, and/or by fact category.

Filtering by comparison state

Click the View drop-down list and select Same to show only the facts for which values are the same, Different to show only the facts that are different, or Incomplete Data to show only the facts where the information is incomplete. You can also select a combination of Same, Different, and Incomplete data states, and clear selections as necessary. When you first add systems for comparison all three options are selected by default.

filter view options

Filtering by fact name
Enter the fact name in the search box at the top to filter by specific fact name. For example, entering kernel as a filter displays all facts containing kernel in their name. Or enter installed_packages in the search box to view all packages.
Filtering by fact category
Enter a fact category in the search box to compare systems by that category. For example, installed_packages, installed_services, kernel_modules, network_interfaces, yum_repos, cpu_flags, and enabled_services.

In the example screen capture below, you can see the system comparison data filtered by facts that show a difference across systems. Some facts, such as the fqdn, are expected to be different for each system, but the installed packages are expected to be the same. Over time, some packages have been upgraded on system 1, but not on system 2 and system 3, as shown by expanding the fact category installed_packages.

drift comparison with diffs screen

8.3. Sorting System Facts

You can sort system facts alphabetically. Click the arrow next to Fact ( fact sort1 ) to switch sorting between ascending and descending order. Note that it is shown in ascending order by default. You can also sort system facts by the comparison State. Click the arrow next to State ( drift state sort ) to switch sorting by state.

Note

Sorting works in combination with any applied filters. That is, if you have filtered for installed packages or viewing facts by comparison state, the filtered data can be sorted alphabetically or by comparison state.

Chapter 9. Exporting System Comparison Output

System Comparison allows you to export the system comparison output, along with any current selections such as filters, to a comma-separated values (CSV) file.

Note

The exported CSV report preserves all your current selections on the system comparison output, including any filters applied. That is, it follows the WYSIWYG (What You See Is What You Get) paradigm. Therefore, you will need to expand any nested fact categories (installed_packages, for example) to be exported in the report.

Procedure

  1. On the system comparison output screen for two or more systems, click the options menu drift options menu located at the top.
  2. Click Export as CSV.

kebab menu options

Open the CSV file with the tool of your choice so that you can easily compare exported facts and analyze discrepancies in systems.

Chapter 10. System Baselines for Drift Analysis

Baselines are configurations (set of name / value facts) that can be defined from scratch, as a copy of an existing system configuration, or as a copy of an existing baseline. Once defined, baselines can be used in the System Comparison service to compare baselines and system configurations. You can manage system profile baseline definitions for your organization. That is, you can also edit baselines by changing values on facts, or deleting them.

Use the following three options to create a baseline:

  • Create baseline from scratch
  • Copy an existing baseline
  • Copy an existing system

drift create new baseline

10.1. Creating a New Baseline from Scratch

You can create a new baseline from scratch within the Drift AnalysisBaselines menu.

Note

When creating baselines, you can use a variety of system facts. For information about system facts, see Accessing and Using System Facts.

Procedure

  1. On the Red Hat Insights user interface, click Drift AnalysisBaselines in the left-side navigation menu.
  2. Click Create baseline.
  3. Select Create baseline from scratch.
  4. Enter a name for the new baseline you want to create from scratch in Baseline name.
  5. Click Create baseline. This will take you to the edit screen for the new baseline.
  6. Click Add fact to enter fact name and value.
  7. Enter the Fact name and Value, then click Save. If this is a new category (parent fact) under which you will add sub fact(s), then select This is a category and only enter the Fact name, and click Save.

    drift add fact

  8. To add sub facts under a category, click the more options menu drift options menu next to a category and click Add sub fact.
  9. You can also edit or delete a fact, sub fact, or category using the more options menu.
Note

Any changes performed are automatically saved. Therefore, once you have completed adding all facts, sub facts and their values, you can view and manage the baseline you have created by navigating to Drift AnalysisBaselines.

10.2. Copying an Existing Baseline to Create a New Baseline

You can copy an existing baseline to create a new baseline.

Procedure

  1. On the Red Hat Insights user interface, click Drift AnalysisBaselines in the left-side navigation menu.
  2. Click Create baseline.
  3. Select Copy an existing baseline.
  4. Enter a name for the new baseline in Baseline name.
  5. From the existing list of baselines, select the baseline you want to copy from to create a new baseline.
  6. Click Create baseline. This will take you to the newly created baseline with all populated facts.
  7. Edit or delete existing facts using the options menu next to a fact.
  8. You can also add a new fact or category by clicking Add fact. Accordingly, you can add a sub fact to a category.
Note

Based on typical expectations about fact behavior, the Insights Client alerts users to facts that may require attention. It does not flag facts that differ from each other if the differences are of no importance, or are if they are expected. Instead, it flags important differences or similarities that may be problematic. You can then address these exceptions to the typical fact behavior in a baseline.

Once you have completed adding or editing facts, sub facts and their values, you can view and manage the baseline you have created by navigating to Drift AnalysisBaselines.

Chapter 11. Exporting Baselines

You can export a list of system baselines to a CSV file and analyze them externally. Note that any search filters applied will be preserved in the downloaded CSV file.

Procedure

  1. On the Drift AnalysisBaselines screen, click the options menu located next to the Create baseline button.
  2. Click Export as CSV.

drift export baselines

Chapter 12. Comparing System Configurations to Baselines

You can compare system configurations to one or more baselines to identify discrepancies in your environment, and perform drift analysis.

Procedure

  1. On the Red Hat Insights user interface, click Drift AnalysisComparison in the left-side navigation menu.
  2. Click Add System.
  3. From the Systems tab, select one or more systems from the list. Alternatively, you can search a system by name, then select the system.
  4. From the Baselines tab, select one or more baselines from the list. Alternatively, you can search a baseline by name, then select the baseline.
  5. Click Submit.

drift choose systems

Note
  • At any time, you can add more systems and baselines by clicking the Add System button on the right-side of the systems already added for comparison.
  • Similarly, you can remove a particular system or baseline under comparison by clicking the cross sign on the upper-right corner of the individual system or baseline name. You can also remove all systems and baselines under comparison by clicking Clear all comparisons from the options menu drift options menu located at the top, and start again.

drift system baseline comparison

You can view the displayed comparison result and filter as necessary by fact name, comparison state, and category. You can export the result, along with any current selections such as filters, to a comma-separated values (CSV) file using the Export as CSV option in the menu for further analysis.

Legal Notice

Copyright © 2020 Red Hat, Inc.
The text of and illustrations in this document are licensed by Red Hat under a Creative Commons Attribution–Share Alike 3.0 Unported license ("CC-BY-SA"). An explanation of CC-BY-SA is available at http://creativecommons.org/licenses/by-sa/3.0/. In accordance with CC-BY-SA, if you distribute this document or an adaptation of it, you must provide the URL for the original version.
Red Hat, as the licensor of this document, waives the right to enforce, and agrees not to assert, Section 4d of CC-BY-SA to the fullest extent permitted by applicable law.
Red Hat, Red Hat Enterprise Linux, the Shadowman logo, the Red Hat logo, JBoss, OpenShift, Fedora, the Infinity logo, and RHCE are trademarks of Red Hat, Inc., registered in the United States and other countries.
Linux® is the registered trademark of Linus Torvalds in the United States and other countries.
Java® is a registered trademark of Oracle and/or its affiliates.
XFS® is a trademark of Silicon Graphics International Corp. or its subsidiaries in the United States and/or other countries.
MySQL® is a registered trademark of MySQL AB in the United States, the European Union and other countries.
Node.js® is an official trademark of Joyent. Red Hat is not formally related to or endorsed by the official Joyent Node.js open source or commercial project.
The OpenStack® Word Mark and OpenStack logo are either registered trademarks/service marks or trademarks/service marks of the OpenStack Foundation, in the United States and other countries and are used with the OpenStack Foundation's permission. We are not affiliated with, endorsed or sponsored by the OpenStack Foundation, or the OpenStack community.
All other trademarks are the property of their respective owners.