Chapter 1. Overview of performance monitoring options

The following are some of the performance monitoring and configuration tools available in Red Hat Enterprise Linux 8:

  • Performance Co-Pilot (pcp) is used for monitoring, visualizing, storing, and analyzing system-level performance measurements. It allows the monitoring and management of real-time data, and logging and retrieval of historical data.
  • Red Hat Enterprise Linux 8 provides several tools that can be used from the command line to monitor a system outside run level 5. The following are the built-in command line tools:

    • top is provided by the procps-ng package. It gives a dynamic view of the processes in a running system. It displays a variety of information, including a system summary and a list of tasks currently being managed by the Linux kernel.
    • ps is provided by the procps-ng package. It captures a snapshot of a select group of active processes. By default, the examined group is limited to processes that are owned by the current user and associated with the terminal where the ps command is executed.
    • Virtual memory statistics (vmstat) is provided by the procps-ng package. It provides instant reports of your system’s processes, memory, paging, block input/output, interrupts, and CPU activity.
    • System activity reporter (sar) is provided by the sysstat package. It collects and reports information about system activity that has occurred so far on the current day.
  • perf uses hardware performance counters and kernel trace-points to track the impact of other commands and applications on a system.
  • bcc-tools is used for BPF Compiler Collection (BCC). It provides over 100 eBPF scripts that monitor kernel activities. For more information on each of this tool, see the man page describing how to use it and what functions it performs.
  • turbostat is provided by the kernel-tools package. It reports on processor topology, frequency, idle power-state statistics, temperature, and power usage on the Intel 64 processors.
  • iostat is provided by the sysstat package. It monitors and reports on system input/output device loading to help administrators make decisions about how to balance input/output load between physical disks.
  • irqbalance distributes hardware interrupts across processors to improve system performance.
  • ss prints statistical information about sockets, allowing administrators to assess device performance over time. Red Hat recommends using ss over netstat in Red Hat Enterprise Linux 8.
  • numastat is provided by the numactl package. By default, numastat displays per-node NUMA hit an miss system statistics from the kernel memory allocator. Optimal performance is indicated by high numa_hit values and low numa_miss values.
  • numad is an automatic NUMA affinity management daemon. It monitors NUMA topology and resource usage within a system that dynamically improves NUMA resource allocation, management, and therefore system performance.
  • SystemTap monitors and analyzes operating system activities, especially the kernel activities.
  • OProfile monitors the system wide performance. It uses the processor’s dedicated performance monitoring hardware to retrieve information about the kernel and system executable to determine the frequency of certain events. These events can be when the memory is referenced, the number of second-level cache requests, and the number of hardware requests received.
  • valgrind analyzes applications by running it on a synthetic CPU and instrumenting existing application code as it is executed. It then prints commentary that clearly identifies each process involved in application execution to a user-specified file, file descriptor, or network socket. It is also useful for finding memory leaks.
  • pqos is provided by the the intel-cmt-cat package. It monitors and controls CPU cache and memory bandwidth on recent Intel processors.

Additional resources

  • For more information, see the man pages of pcp, top, ps, vmstat, sar, perf, iostat, irqbalance, ss, numastat, numad, valgrind, and pqos.
  • For more information on oprofile and pcp, see the documentation in the /usr/share/doc/ directory.
  • For more information on the await value and what can cause its values to be high, see the Red Hat Knowledgebase article: What exactly is the meaning of value "await" reported by iostat?.