Chapter 7. Running and interpreting system latency tests

RHEL for Real Time provides the rteval utility to test the system real-time performance under load.

7.1. Running system latency tests

With the rteval utility, you can test a system’s real-time performance under load.

Prerequisites

  • The RHEL for Real Time package group is installed.
  • You have root permissions on the system.

Procedure

  • Run the rteval utility.

    # rteval

    The rteval utility starts a heavy system load of SCHED_OTHER tasks. It then measures real-time response on each online CPU. The loads are a parallel make of the Linux kernel tree in a loop and the hackbench synthetic benchmark.

    The goal is to bring the system into a state, where each core always has a job to schedule. The jobs perform various tasks, such as memory allocation/free, disk I/O, computational tasks, memory copies, and other.

    Once the loads start, rteval starts the cyclictest measurement program. This program starts the SCHED_FIFO real-time thread on each online core. It then measures the real-time scheduling response time.

    Each measurement thread takes a timestamp, sleeps for an interval, then takes another timestamp after waking up. The latency measured is t1 - (t0 + i), which is the difference between the actual wakeup time t1, and the theoretical wakeup time of the first timestamp t0 plus the sleep interval i.

    The details of the rteval run are written to an XML file along with the boot log for the system. This report is displayed on the screen and saved to a compressed file.

    The file name is in the form rteval-<date>-N-tar.bz2, where <date> is the date the report was generated, N is a counter for the Nth run on <date>.

    The following is an example of an rteval report:

    System:
    Statistics:
    	Samples:           1440463955
    	Mean:              4.40624790712us
    	Median:            0.0us
    	Mode:              4us
    	Range:             54us
    	Min:               2us
    	Max:               56us
    	Mean Absolute Dev: 1.0776661507us
    	Std.dev:           1.81821060672us
    
    CPU core 0       Priority: 95
    Statistics:
    	Samples:           36011847
    	Mean:              5.46434910711us
    	Median:            4us
    	Mode:              4us
    	Range:             38us
    	Min:               2us
    	Max:               40us
    	Mean Absolute Dev: 2.13785341159us
    	Std.dev:           3.50155558554us

    The report includes details about the system hardware, length of the run, options used, and the timing results, both per-cpu and system-wide.

    Note

    To regenerate an rteval report from its generated file, run

    # rteval --summarize rteval-<date>-N.tar.bz2