Red Hat Training

A Red Hat training course is available for Red Hat Enterprise Linux

Chapter 23. OProfile

OProfile is a low overhead, system-wide performance monitoring tool provided by the oprofile package. It uses performance monitoring hardware on the system’s processor to retrieve information about the kernel and executables on the system, such as when memory is referenced, the number of second-level cache requests, and the number of hardware interrupts received. OProfile is also able to profile applications that run in a Java Virtual Machine (JVM).

The following is a selection of the tools provided by OProfile:

ophelp
Displays available events for the system’s processor along with a brief description of each event.
operf

The main profiling tool. The operf tool uses the Linux Performance Events subsystem, which allows OProfile to operate alongside other tools using performance monitoring hardware of your system.

Unlike the previously used opcontrol tool, no initial setup is required, and it can be used without root privileges unless the --system-wide option is used.

ocount
A tool for counting the absolute number of event occurrences. It can count events on the whole system, per process, per CPU, or per thread.
opimport
Converts sample database files from a foreign binary format to the native format for the system. Only use this option when analyzing a sample database from a different architecture.
opannotate
Creates an annotated source for an executable if the application was compiled with debugging symbols.
opreport
Reads the recorded performance data and generates a summary as specified by the profile specification. It is possible to generate different reports from the same profile data using different profile specifications.

23.1. Using OProfile

operf is the recommended tool for collecting profiling data. The tool does not require any initial configuration, and all options are passed to it on the command line. Unlike the legacy opcontrol tool, operf can run without root privileges. See the Using operf chapter in the System Administrator’s Guide for detailed instructions on how to use the operf tool.

Example 23.1. Using ocount

The following example shows counting the amount of events with ocount during execution of the sleep utility: 

$ ocount -e INST_RETIRED -- sleep 1

Events were actively counted for 1.0 seconds.
Event counts (actual) for /bin/sleep:
Event Count % time counted
INST_RETIRED 683,011 100.00
Note

The events are processor implementation specific. It might be necessary to set the option perf_event_paranoid or limit the counts to only user-space events.

Example 23.2. Basic operf usage

In the following example, the operf tool is used to collect profiling data from the ls -l ~ command.

  1. Install debugging information for the ls command: 

    # debuginfo-install -y coreutils

  2. Run the profiling: 

    $ operf ls -l ~
Profiling done.

  3. Analyze the collected data: 

    $ opreport --symbols
CPU: Intel Skylake microarchitecture, speed 3.4e+06 MHz (estimated)
Counted cpu_clk_unhalted events () with a unit mask of 0x00 (Core cycles when at least one thread on the physical core is not in halt state) count 100000
samples % image name symbol name
161 81.3131 no-vmlinux /no-vmlinux
3 1.5152 libc-2.17.so get_next_seq
3 1.5152 libc-2.17.so strcoll_l
2 1.0101 ld-2.17.so _dl_fixup
2 1.0101 ld-2.17.so _dl_lookup_symbol_x
[...]

Example 23.3. Using operf to Profile a Java Program

In the following example, the operf tool is used to collect profiling data from a Java (JIT) program, and the opreport tool is then used to output per-symbol data.

  1. Install the demonstration Java program used in this example. It is a part of the java-1.8.0-openjdk-demo package, which is included in the Optional channel. See Adding the Optional and Supplementary Repositories for instructions on how to use the Optional channel. When the Optional channel is enabled, install the package: 

    # yum install java-1.8.0-openjdk-demo

  2. Install the oprofile-jit package for OProfile to be able to collect profiling data from Java programs: 

    # yum install oprofile-jit

  3. Create a directory for OProfile data: 

    $ mkdir ~/oprofile_data

  4. Change into the directory with the demonstration program: 

    $ cd /usr/lib/jvm/java-1.8.0-openjdk/demo/applets/MoleculeViewer/

  5. Start the profiling: 

    $ operf -d ~/oprofile_data appletviewer \
-J"-agentpath:/usr/lib64/oprofile/libjvmti_oprofile.so" example2.html

  6. Change into the home directory and analyze the collected data: 

    $ cd
    $ opreport --symbols --threshold 0.5

    A sample output may look like the following: 

    $ opreport --symbols --threshold 0.5
Using /home/rkratky/oprofile_data/samples/ for samples directory.
CPU: Intel Ivy Bridge microarchitecture, speed 3600 MHz (estimated)
Counted CPU_CLK_UNHALTED events (Clock cycles when not halted) with a unit mask of 0x00 (No unit mask) count 100000
samples  %        image name               symbol name
14270    57.1257  libjvm.so                /usr/lib/jvm/java-1.8.0-openjdk-1.8.0.51-1.b16.el7_1.x86_64/jre/lib/amd64/server/libjvm.so
3537     14.1593  23719.jo                 Interpreter
690       2.7622  libc-2.17.so             fgetc
581       2.3259  libX11.so.6.3.0          /usr/lib64/libX11.so.6.3.0
364       1.4572  libpthread-2.17.so       pthread_getspecific
130       0.5204  libfreetype.so.6.10.0    /usr/lib64/libfreetype.so.6.10.0
128       0.5124  libc-2.17.so             __memset_sse2

23.2. OProfile Documentation

For more extensive information on OProfile, see the oprofile(1) manual page. Red Hat Enterprise Linux also provides two comprehensive guides to OProfile in file:///usr/share/doc/oprofile-version/:

OProfile Manual
A comprehensive manual with detailed instructions on the setup and use of OProfile is available at file:///usr/share/doc/oprofile-version/oprofile.html
OProfile Internals
Documentation on the internal workings of OProfile, useful for programmers interested in contributing to the OProfile upstream, can be found at file:///usr/share/doc/oprofile-version/internals.html