Chapter 21. Tracing latencies with trace-cmd

The trace-cmd utility is a front end to the ftrace utility. By using trace-cmd, you can enable ftrace actions, without the need to write to the /sys/kernel/debug/tracing/ directory. trace-cmd does not add any overhead on its installation.


  • You have administrator privileges.

21.1. Installing trace-cmd

The trace-cmd utility provides a front-end to the ftrace utility.


  • You have administrator privileges.


  • Install the trace-cmd utility.

    # yum install trace-cmd

21.2. Running trace-cmd

You can use the trace-cmd utility to access all ftrace functionalities.


  • You have administrator privileges.


  • Enter trace-cmd command

    where command is an ftrace option.


    See the trace-cmd(1) man page for a complete list of commands and options. Most of the individual commands also have their own man pages, trace-cmd-command.

21.3. trace-cmd examples

This provides a number of trace-cmd examples.


  • Enable and start recording functions executing within the kernel while myapp runs.

    # trace-cmd record -p function myapp

    This records functions from all CPUs and all tasks, even those not related to myapp.

  • Display the result.

    # trace-cmd report
  • Record only functions that start with sched while myapp runs.

    # trace-cmd record -p function -l 'sched*' myapp
  • Enable all the IRQ events.

    # trace-cmd start -e irq
  • Start the wakeup_rt tracer.

    # trace-cmd start -p wakeup_rt
  • Start the preemptirqsoff tracer, while disabling function tracing.

    # trace-cmd start -p preemptirqsoff -d

    The version of trace-cmd in RHEL 8 turns off ftrace_enabled instead of using the function-trace option. You can enable ftrace again with trace-cmd start -p function.

  • Restore the state in which the system was before trace-cmd started modifying it.

    # trace-cmd start -p nop

    This is important if you want to use the debugfs file system after using trace-cmd, whether or not the system was restarted in the meantime.

  • Trace a single trace point.

    # trace-cmd record -e sched_wakeup ls /bin
  • Stop tracing.

    # trace-cmd record stop

21.4. Additional resources

  • trace-cmd(1) man page