Red Hat Training

A Red Hat training course is available for RHEL 8

Chapter 24. Creating uprobes with perf

24.1. Creating uprobes at the function level with perf

You can use the perf tool to create dynamic tracepoints at arbitrary points in a process or application. These tracepoints can then be used in conjunction with other perf tools such as perf stat and perf record to better understand the process or applications behavior.

Prerequisites

  • You have the perf user space tool installed as described in Installing perf.

Procedure

  1. Create the uprobe in the process or application you are interested in monitoring at a location of interest within the process or application: 

    # perf probe -x /path/to/executable -a function
Added new event:
  probe_executable:function   (on function in /path/to/executable)

You can now use it in all perf tools, such as:

        perf record -e probe_executable:function -aR sleep 1

Additional resources

24.2. Creating uprobes on lines within a function with perf

These tracepoints can then be used in conjunction with other perf tools such as perf stat and perf record to better understand the process or applications behavior.

Prerequisites

  • You have the perf user space tool installed as described in Installing perf.

  • You have gotten the debugging symbols for your executable: 

    # objdump -t ./your_executable | head
    Note

    To do this, the debuginfo package of the executable must be installed or, if the executable is a locally developed application, the application must be compiled with debugging information, the -g option in GCC.

Procedure

  1. View the function lines where you can place a uprobe: 

    $ perf probe -x ./your_executable -L main

    Output of this command looks similar to: 

    <main@/home/user/my_executable:0>
              0  int main(int argc, const char **argv)
              1  {
                        int err;
                        const char *cmd;
                        char sbuf[STRERR_BUFSIZE];

                        /* libsubcmd init */
              7         exec_cmd_init("perf", PREFIX, PERF_EXEC_PATH, EXEC_PATH_ENVIRONMENT);
              8         pager_init(PERF_PAGER_ENVIRONMENT);

  2. Create the uprobe for the desired function line: 

    # perf probe -x ./my_executable main:8
Added new event:
          probe_my_executable:main_L8   (on main:8 in /home/user/my_executable)

        You can now use it in all perf tools, such as:

                perf record -e probe_my_executable:main_L8 -aR sleep 1

24.3. Perf script output of data recorded over uprobes

A common method to analyze data collected using uprobes is using the perf script command to read a perf.data file and display a detailed trace of the recorded workload.

In the perf script example output:

  • A uprobe is added to the function isprime() in a program called my_prog
  • a is a function argument added to the uprobe. Alternatively, a could be an arbitrary variable visible in the code scope of where you add your uprobe: 
# perf script
    my_prog  1367 [007] 10802159.906593: probe_my_prog:isprime: (400551) a=2
    my_prog  1367 [007] 10802159.906623: probe_my_prog:isprime: (400551) a=3
    my_prog  1367 [007] 10802159.906625: probe_my_prog:isprime: (400551) a=4
    my_prog  1367 [007] 10802159.906627: probe_my_prog:isprime: (400551) a=5
    my_prog  1367 [007] 10802159.906629: probe_my_prog:isprime: (400551) a=6
    my_prog  1367 [007] 10802159.906631: probe_my_prog:isprime: (400551) a=7
    my_prog  1367 [007] 10802159.906633: probe_my_prog:isprime: (400551) a=13
    my_prog  1367 [007] 10802159.906635: probe_my_prog:isprime: (400551) a=17
    my_prog  1367 [007] 10802159.906637: probe_my_prog:isprime: (400551) a=19