Language and Page Formatting Options
2.13. System Partitioning
One of the key techniques for real-time tuning is partitioning the system. This means isolating a group of CPU cores for exclusive use of one or more real-time applications running on the system. For best results, partitioning should take into account the CPU topology so that related threads are placed on cores contained on the same Non-Uniform Memory Access (NUMA) node to maximize sharing of second and third-level caches. The
tunautilities are used to determine the system CPU topology. The Tuna GUI allows you to dynamically isolate CPUs and move threads and interrupts from one CPU to another so that performance impacts can be measured.
Once a partitioning strategy has been determined based on the system layout and the structure of the application, the next step is to set the system to be partitioned automatically upon boot. For that, use the utilities provided by the tuned-profiles-realtime package. This package is installed by default when the Red Hat Enterprise Linux for Real Time packages are installed. To install tuned-profiles-realtime manually, run the following command as root:
yum install tuned-profiles-realtime
The tuned-profiles-realtime package provides the
tunedreal-time profile that allows partitioning and other tunings at boot time with no additional user input required. Two configuration files control the behavior of the profile:
realtime-variables.conffile specifies the group of CPU cores to be isolated. To isolate a group of CPU cores from the system, use the
isolated_coresoption as in the following example:
# Examples: # isolated_cores=2,4-7 # isolated_cores=2-23 # isolated_cores=1-3,5,9-14
In the example above, the profile places the CPUs 1, 2, 3, 5, 9, 10, 11, 12, 13, and 14 into an isolated CPU category; the only threads on these CPUs are kernel threads specifically bound to the cores. These kernel threads are only run when a specific condition is raised, such as the migration thread or the watchdog thread.
isolated_coresvariable is set, activate the profile with the
tuned-adm profile realtime
The profile uses the
bootloaderplug-in. When activated, this plug-in adds the following boot parameters to the Linux kernel command line:
- specifies CPUs listed in the
- turns off the timer tick on an idle CPU; set to
- turns off the timer tick on a CPU when there is only one runnable task on that CPU; needs
nohzto be set to
- prevents the Intel idle driver from managing power state and CPU frequency
- prevents the kernel from detecting soft lockups in user threads
In the above example, the kernel boot command-line parameters look as follows:
isolcpus=1-3,5,9-14 nohz=on nohz_full=1-3,5,9-14 intel_pstate=disable nosoftlockup
The profile runs the
script.shshell script specified in the
tuned.conf. The script adjusts the following entries of the
sysfsvirtual file system:
workqueueentry above is set to the inverse of the isolated CPUs mask, while the second entry turns off machine check exceptions.
The script also sets the following variables in the
kernel.hung_task_timeout_secs = 600 kernel.nmi_watchdog = 0 kernel.sched_rt_runtime_us = 1000000 vm.stat_interval = 10
The script uses the
tunainterface to move any non-bound thread on the isolated CPU numbers off of the isolated CPUs.
For further tuning, copy the default
/usr/lib/tuned/realtime/script.shand modify it, then change the
tuned.confJSON file to point to the modified script.