Menu Close

Assessing and Monitoring RHEL Resource Optimization with Insights for Red Hat Enterprise Linux

Red Hat Insights 2022

Understanding RHEL resource-usage statistics

Red Hat Customer Content Services

Abstract

Install and begin using the Insights for RHEL resource optimization service. This new service helps manage your public cloud systems.

Making open source more inclusive

Red Hat is committed to replacing problematic language in our code, documentation, and web properties. We are beginning with these four terms: master, slave, blacklist, and whitelist. Because of the enormity of this endeavor, these changes will be implemented gradually over several upcoming releases. For more details, see our CTO Chris Wright’s message.

Providing feedback on Red Hat documentation

We appreciate your feedback on our documentation. To provide feedback, highlight text in a document and add comments.

Prerequisites

  • You are logged in to the Red Hat Customer Portal.
  • In the Red Hat Customer Portal, the document is in the Multi-page HTML viewing format.

Procedure

To provide your feedback, perform the following steps:

  1. Click the Feedback button in the top-right corner of the document to see existing feedback.

    Note

    The feedback feature is enabled only in the Multi-page HTML format.

  2. Highlight the section of the document where you want to provide feedback.
  3. Click the Add Feedback pop-up that appears near the highlighted text.

    A text box appears in the feedback section on the right side of the page.

  4. Enter your feedback in the text box and click Submit.

    A documentation issue is created.

  5. To view the issue, click the issue link in the feedback view.

Chapter 1. Getting started with the resource optimization service for public cloud systems

The Insights for Red Hat Enterprise Linux resource optimization service enables RHEL customers to assess and monitor their public RHEL cloud usage and optimization. The service shows metrics for the following:

  • CPU
  • Memory
  • Disk-usage

It analyzes those metrics and compares them to resource limits recommended by your public cloud provider. Leveraging data from the past day, the resource optimization service considers each resource parameter in several distinct ways and returns actionable data. This data enables better resource allocation and helps you to save money on your public cloud investment.

Features

The service reveals the following information:

  • Utilization and optimization data for existing systems in the Insights for RHEL inventory.
  • Range of systems running in the public cloud.
  • Overview of system characteristics.
  • Highlights potential issues.
  • Formulates suggestions for issue resolution.

Chapter 2. Resource optimization service core concepts

2.1. The resource optimization service performance rules

Use the resource optimization service to view performance metrics from your managed hosts that run in the supported public cloud, Amazon Web Services (AWS). The service uses a framework called the Performance Co-Pilot (PCP) toolkit, to record performance metrics. These metrics empower you to make better business decisions.

Insights performance rules

The performance rules are sets of conditions that are applied to the data collected by PCP. They identify the following system states:

  • Undersized. The undersized state is determined by examining CPU, RAM and disk input/output (I/O) usage, and combining that with CPU idle time, over a period of 24 hours. If that results in a high score, the resource optimization service labels the system too small for its workload. A system will be reported as undersized whenever any of the dimensions are undersized.
  • Oversized. The oversized state is determined by examining CPU, RAM and disk I/O usage, and combining that with CPU idle time, over a period of 24 hours. If that results in a low score, the resource optimization service labels the system as too big for its workload. A system will be reported as oversized only if all of the dimensions are oversized.
  • Idling. The idling state is determined by examining CPU, RAM and disk I/O usage, and combining that with CPU idle time, over a period of 24 hours. If that results in very low utilization, the resource optimization service labels the system appropriate for its workload but underused. The idling condition can be viewed as a needs improvement scenario.
  • Optimized. The optimized state is determined by examining CPU, RAM and disk I/O usage, and combining that with CPU idle time, over a period of 24 hours. If that results in a middle point, the resource optimization service labels the system as optimized.
  • Under pressure. This state is only active when Kernel Pressure Stall Information (PSI) has been enabled. Systems are labeled as under pressure when they are optimized utilization-wise, but some pressure condition persists.

The resource optimization service measures the system’s state and the desired performance criteria that you have set, in order to assign a score to the system.

Additional resources

For more information about the PCP toolkit and registering PAYG vist the following links:

PCP toolkit website: PCP website

Monitoring performance with PCP toolkit: Monitoring performance with PCP

How does PCP toolkit compare with sysstat: PCP and sysstat comparison

Registering PAYG instances How to register a Red Hat Enterprise Linux system running on AWS to Insights

2.2. Data security guarantee for the resource optimization service

The resource optimization service adheres to the data and application security practices for Insights for RHEL application services. For more details see Security.

2.3. Performance metrics for resource optimization

The resource optimization service installs the pcp package on your system and runs two services, pmcd and pmlogger. Both are part of the Performance Co-Pilot (PCP) toolkit, which monitor and process specific metrics on your system. Metrics are stored in an archive, which the Insights client uploads to Insights for Red Hat Enterprise Linux.

2.4. Access usage metrics for the resource optimization service

The resource optimization service captures data from the previous day and provides system usage metrics after 24 hours. By default, the archive is uploaded to Insights for RHEL at 12:00am +/- 1 hour, local system time. However, the time that this data is uploaded can be configured in the Performance Co-Pilot (PCP) toolkit configuration.

Chapter 3. Enabling the resource optimization service

Table 3.1. Compatibility information

RHEL VersionsCloud ProviderResource Optimization Compatibility

8.x

AWS

Yes (x86_64 and ARM 64-bit)

7.7-7.9

AWS

Yes (x86_64 and ARM 64-bit)

7.0-7.6

AWS

No

6.x

AWS

No

Prerequisites

The following applications and configurations need to be installed or confirmed before the resource optimization service can be used:

  • Cloud marketplace RHEL instance is configured.
  • The Insights client is installed on the system and is operational.
  • If you want to use Ansible to install or uninstall the resource optimization service:

    • The Ansible repository is enabled and the Ansible client is installed on each system.
    • The system administrator can run Ansible Playbooks.

Chapter 4. Installing the resource optimization service

Installing resource optimization involves installing packages, configuring settings and enabling local services. This can be done manually, or with an Ansible playbook provided by Red Hat.

Note

Pay as you go (PAYG) customers need to configure the insights-client to use basic authentication. Reference the Configuring Basic Authentication for Red Hat Insights documentation for more details. If your RHEL system is using a subscription, you may use basic authentication, but it is not required: the Insights client can use the already-existing RHSM data.

4.1. Using Ansible to enable resource optimization

This procedure will install the Ansible client, and run the Ansible Playbook on your system.

Cloud marketplace images on Amazon Web Services (AWS) are configured to use repositories hosted by the cloud provider. Currently, these repositories do not contain the Ansible client, so you must perform the following steps to enable the Ansible repository on your cloud marketplace - managed RHEL system.

4.1.1. Installing Ansible

The use of Ansible is recommended to expedite the installation process.

Procedure on RHEL 8

  1. Install Ansible:

    # yum install ansible -y

Procedure on RHEL 7

  1. Enable the Subscription-Manager repository and register the system

    # subscription-manager config --rhsm.manage_repos=1
    # subscription-manager register
  2. Optionally, attach your system to a subscription pool

    # subscription-manager attach --pool xxxxxxxx
  3. Enable the required Ansible repository.

    # subscription-manager repos --enable=rhel-7-server-ansible-2.9-rpms
  4. Install Ansible:

    # yum install ansible -y
  5. If you are using RHEL PAYG and want to use RHUI update servers only, disable the Subscription-Manager repository:

    # subscription-manager config --rhsm.manage_repos=0

4.1.2. Installing the resource optimization service

Once Ansible is installed, proceed to complete the installation of the resource optimization service.

Procedure

  1. Download the Ansible Playbook with the following command:

    $ curl -O https://raw.githubusercontent.com/RedHatInsights/ros-backend/v1.0/ansible-playbooks/ros_install_and_set_up.yml
  2. Set localhost in Ansible inventory by appending the line localhost to /etc/ansible/hosts.
  3. Run the Ansible Playbook:

    # ansible-playbook -c local ros_install_and_set_up.yml

The system will show in Insights immediately in a "Waiting for data" state, and data and suggestions will be available the day after registering.

Verification step

Data files with a timestamp will appear under /var/log/pcp/pmlogger/ros and after a few minutes, you can verify metrics are being collected:

$ ls -l /var/log/pcp/pmlogger/ros
$ pmlogsummary /var/log/pcp/pmlogger/ros/

4.2. Manually installing resource optimization service

Procedure

If you choose not to utilize Ansible for installation, use the following manual installation procedure:

  1. Ensure the latest version of insights-client is installed

    $ yum update insights-client
  2. Set core_collect=True in /etc/insights-client/insights-client.conf
  3. Install the Performance Co-Pilot (PCP) toolkit.

    $ sudo yum install pcp
  4. Create the PCP configuration file /var/lib/pcp/config/pmlogger/config.ros with this content:

    log mandatory on default {
      hinv.ncpu
      mem.physmem
      mem.util.available
      disk.dev.total
      kernel.all.cpu.idle
      kernel.all.pressure.cpu.some.avg
      kernel.all.pressure.io.full.avg
      kernel.all.pressure.io.some.avg
      kernel.all.pressure.memory.full.avg
      kernel.all.pressure.memory.some.avg
    }
    [access]
    disallow .* : all;
    disallow :* : all;
    allow local:* : enquire;
  5. To configure pmlogger to gather the metrics required by resource optimization, add this line to /etc/pcp/pmlogger/control.d/local:

    LOCALHOSTNAME	n   y	PCP_LOG_DIR/pmlogger/ros	-r -T24h10m -c config.ros -v 100Mb
  6. Start and enable the required PCP services.

    $ sudo systemctl enable pmcd pmlogger
    $ sudo systemctl start pmcd pmlogger
  7. Re-register insights-client and upload the archive. The system will show in Insights immediately in a "Waiting for data" state, and data and suggestions will be available the day after registering.

    $ sudo insights-client --register

Verification step

Data files with a timestamp will appear under /var/log/pcp/pmlogger/ros and after a few minutes, you can verify metrics are being collected:

$ ls -l /var/log/pcp/pmlogger/ros
$ pmlogsummary /var/log/pcp/pmlogger/ros/

Chapter 5. Enabling Kernal Pressure Stall Information (PSI)

PSI provides a canonical way to see resource pressure increases as they develop. There are pressure metrics for three major resources—memory, CPU, and input/output (I/O). It is available on RHEL 8 and newer versions and is disabled by default.

When PSI is enabled, the resource optimization service can augment its findings and provide more details and better suggestions. Enabling PSI is strongly recommended to identify peaks.

Procedure

  1. Edit the /etc/default/grub file and append psi=1 at the end of the GRUB_CMDLINE_LINUX line (mind the quotes).
  2. Regenerate the grub configuration file.

    $ sudo grub2-mkconfig -o /boot/grub2/grub.cfg
  3. Reboot the system.
Note

Enabling PSI incurs in a slight (<1%) performance hit.

Verification step

When PSI is enabled, files for CPU, memory and IO will appear under /proc/pressure.

Chapter 6. Disabling PSI

Procedure

  1. Edit the /etc/default/grub file and remove psi=1 from the GRUB_CMDLINE_LINUX line (mind the quotes).
  2. Regenerate the grub configuration file.

    $ sudo grub2-mkconfig -o /boot/grub2/grub.cfg
  3. Reboot the system.

Verification step

When PSI is disabled, /proc/pressure does not exist.

Chapter 7. Uninstalling the resource optimization service

7.1. Using Ansible to disable resource optimization service

Perform the following steps on each system to disable and uninstall the Insights for RHEL resource optimization service.

Procedure

  1. Download the Ansible Playbook with the following command:

    $ curl -O https://raw.githubusercontent.com/RedHatInsights/ros-backend/v1.0/ansible-playbooks/ros_disable.yml
  2. Run the Ansible Playbook using command:

    # ansible-playbook -c local ros_disable_and_clean_up.yml

Uninstalling the playbook does not stop or remove the Performance Co-Pilot (PCP) toolkit. Note that PCP may support multiple applications. If you are using PCP exclusively for the resource optimization service, and desire to remove PCP as well, there are a couple options. You can stop and disable the pmlogger and pmcd services, or remove PCP completely by uninstalling the pcp package from the system.

7.2. Manually disabling resource optimization service

The use of Ansible is recommended to expedite the uninstallation process. If you choose to not use Ansible, utilize the manual procedure that follows:

Procedure

  1. Disable resource optimization service metrics collection by removing this line from /etc/pcp/pmlogger/control.d/local

    LOCALHOSTNAME	n   y	PCP_LOG_DIR/pmlogger/ros	-r -T24h10m -c config.ros -v 100Mb
  2. Restart PCP so that resource optimization service metrics collection is effectively stopped:

    $ sudo systemctl pmcd pmlogger
  3. Remove the resource optimization service configuration file

    $ sudo rm /var/lib/pcp/config/pmlogger/config.ros
  4. Remove the resource optimization data from the system

    $ sudo rm -rf /var/log/pcp/pmlogger/ros
  5. If you are not using PCP for anything else, you can remove it from your system

    $ sudo yum remove pcp

Legal Notice

Copyright © 2022 Red Hat, Inc.
The text of and illustrations in this document are licensed by Red Hat under a Creative Commons Attribution–Share Alike 3.0 Unported license ("CC-BY-SA"). An explanation of CC-BY-SA is available at http://creativecommons.org/licenses/by-sa/3.0/. In accordance with CC-BY-SA, if you distribute this document or an adaptation of it, you must provide the URL for the original version.
Red Hat, as the licensor of this document, waives the right to enforce, and agrees not to assert, Section 4d of CC-BY-SA to the fullest extent permitted by applicable law.
Red Hat, Red Hat Enterprise Linux, the Shadowman logo, the Red Hat logo, JBoss, OpenShift, Fedora, the Infinity logo, and RHCE are trademarks of Red Hat, Inc., registered in the United States and other countries.
Linux® is the registered trademark of Linus Torvalds in the United States and other countries.
Java® is a registered trademark of Oracle and/or its affiliates.
XFS® is a trademark of Silicon Graphics International Corp. or its subsidiaries in the United States and/or other countries.
MySQL® is a registered trademark of MySQL AB in the United States, the European Union and other countries.
Node.js® is an official trademark of Joyent. Red Hat is not formally related to or endorsed by the official Joyent Node.js open source or commercial project.
The OpenStack® Word Mark and OpenStack logo are either registered trademarks/service marks or trademarks/service marks of the OpenStack Foundation, in the United States and other countries and are used with the OpenStack Foundation's permission. We are not affiliated with, endorsed or sponsored by the OpenStack Foundation, or the OpenStack community.
All other trademarks are the property of their respective owners.