Chapter 11. SystemTap

SystemTap is a tracing and probing tool that allows users to monitor the activities of the entire system without needing to instrument, recompile, install, and reboot. It is programmable with a custom scripting language, which gives it expressiveness (to trace, filter, and analyze) and reach (to look into the running kernel and applications).
SystemTap can monitor various types of events, such as function calls within the kernel or applications, timers, tracepoints, performance counters, and so on. Some included example scripts produce output similar to netstat, ps, top, and iostat, others include pretty-printed function callgraph traces or tools for working around security bugs.
Red Hat Developer Toolset is distributed with SystemTap 2.9. This version is more recent than the version included in the previous release of Red Hat Developer Toolset and provides numerous bug fixes and enhancements.

Table 11.1. Tools Distributed with SystemTap for Red Hat Developer Toolset

Name Description
stap Translates probing instructions into C code, builds a kernel module, and loads it into a running Linux kernel.
stapdyn The Dyninst backend for SystemTap.
staprun Loads, unloads, attaches to, and detaches from kernel modules built with the stap utility.
stapsh Serves as a remote shell for SystemTap.
stap-prep Determines and—if possible—downloads the kernel information packages that are required to run SystemTap.
stap-merge Merges per-CPU files. This script is automatically executed when the stap utility is executed with the -b command line option.
stap-report Gathers important information about the system for the purpose of reporting a bug in SystemTap.
stap-server A compile server, which listens for requests from stap clients.

11.1. Installing SystemTap

In Red Hat Developer Toolset, SystemTap is provided by the devtoolset-4-systemtap package and is automatically installed with devtoolset-4-perftools as described in Section 1.5, “Installing Red Hat Developer Toolset”.

Note

The Red Hat Developer Toolset version of SystemTap is available for both Red Hat Enterprise Linux 6 and Red Hat Enterprise Linux 7, but some new features are only offered by the Red Hat Developer Toolset version of SystemTap for Red Hat Enterprise Linux 7.
In order to place instrumentation into the Linux kernel, SystemTap may also require installation of additional packages with debugging information. To determine which packages to install, run the stap-prep utility as follows: 
scl enable devtoolset-4 'stap-prep'
Note that if you execute this command as the root user, the utility automatically offers the packages for installation. For more information on how to install these packages on your system, see the Red Hat Enterprise Linux 6 SystemTap Beginners Guide or the Red Hat Enterprise Linux 7 SystemTap Beginners Guide.