How to use cgroup cpusets with systemd in RHEL7?
Environment
- Red Hat Enterprise Linux 7
Issue
- How to use cgroup cpuset with systemd for user processes?
In the environment we have assigned exclusive access to individual CPU cores to individual, specific user processes/threads. Under RHEL 6.5, this was done using libcgroup and the cgruoup cpuset functionality (specifically via the cgconfig service and cgclassify). A one-time configuration script generates the cgconfig.conf and this is installed by a system administrator.
They are required to launch, then classify as the application makes use of LD_LIBRARY_PATH which is not available when running a binary with setuid (such as cgexec). However, these libcgroup tools have been subsequently deprecated in RHEL 7 with the following warning:
WARNING
The deprecated cgconfig tool from the libcgroup package is available to mount and handle hierarchies for controllers not yet supported by systemd (most notably the net-prio controller). Never use libcgropup tools to modify the default hierarchies mounted by systemd since it would lead to unexpected behavior. The libcgroup library will be removed in the future versions of Red Hat Enterprise Linux. For more information on how to use cgconfig, see Chapter 3, Using libcgroup Tools.
The replacement model uses systemd as described here
However, currently the cpuset interface is not exposed through system and, so as far as we can tell, this functionality is now not available: [2][3]
Note that the number of cgroup attributes currently exposed as unit properties is limited. This will be extended later on, as their kernel interfaces are cleaned up. For example cpuset or freezer are currently not exposed at all due to the broken inheritance semantics of the kernel logic. Also, migrating units to a different slice at runtime is not supported (i.e. altering the Slice= property for running units) as the kernel currently lacks atomic cgroup subtree moves.
How is it possible to safely achieve the same result (exclusive cpu access/arbitrary cpusets for userland applications) ?
Resolution
-
As per RHEL 7 product documentation libcgroup is deprecated and systemd provides some directives which is helpful to use the cgroup feature (As of now systemd does not provide directives for all cgroup feature)
-
Specific to limit cgroup resources for those systemd does not provide directive i am sharing one dummy example which confirm how we can use systemd to configure cgroup resources.
//cc cpu.c -o cpu #include <stdio.h> #include <stdlib.h> #include <string.h> int main() { double x; while(1) { x = x + 0.0000001; } return 0; } -
Below is the service file to run this program through the systemd and configure cgroup resource limit for this program
# cat /usr/lib/systemd/system/cpus.service [Unit] Description=cpus After=syslog.target network.target auditd.service [Service] ExecStartPre=/usr/bin/mkdir -p /sys/fs/cgroup/cpuset/mygroup1 ===> Create group to manage process ExecStartPre=/bin/bash -c '/usr/bin/echo "3" > /sys/fs/cgroup/cpuset/mygroup1/cpuset.cpus' ==> Assign cpu core 3 to this process ExecStartPre=/bin/bash -c '/usr/bin/echo "0" > /sys/fs/cgroup/cpuset/mygroup1/cpuset.mems' ExecStart=/root/cpu ===> Run this process ExecStartPost=/bin/bash -c '/usr/bin/echo $MAINPID > /sys/fs/cgroup/cpuset/mygroup1/tasks' ==> Assign process id to group task file ExecStopPost=/usr/bin/rmdir /sys/fs/cgroup/cpuset/mygroup1 ==> At the time of stop remove group Restart=on-failure [Install] WantedBy=multi-user.target -
After creating the file, execute the service
# systemctl daemon-reload # systemctl start cpus.service -
It will start
cpuprocess and upon investigation should be clear it is using CPU 3.# ps -eLF | grep -e PID -e cpu UID PID PPID LWP C NLWP SZ RSS PSR STIME TTY TIME CMD root 131188 1 131188 98 1 1037 320 3 17:21 ? 00:08:59 /root/cpu -
To run in a unprivileged user we need to divide the unit file into two separate unit files.
-
The commands which needs root privileges put them into the first unit1.service file
[Service] ExecStartPre=/usr/bin/mkdir -p /sys/fs/cgroup/cpuset/mygroup1 ===> Create group to manage process ExecStartPre=/bin/bash -c '/usr/bin/echo "3" > /sys/fs/cgroup/cpuset/mygroup1/cpuset.cpus' ==> Assign cpu core 3 to this process ExecStartPre=/bin/bash -c '/usr/bin/echo "0" > /sys/fs/cgroup/cpuset/mygroup1/cpuset.mems' ExecStartPost=/bin/bash -c '/usr/bin/echo $MAINPID > /sys/fs/cgroup/cpuset/mygroup1/tasks' ==> Assign process id to group task file ExecStopPost=/usr/bin/rmdir /sys/fs/cgroup/cpuset/mygroup1 ==> At the time of stop remove group -
And then put command/binary which does not need root privileges put them into another unit2.service file
[Service] ExecStart=/root/cpu ===> Run this process Restart=on-failure
-
This solution is part of Red Hat’s fast-track publication program, providing a huge library of solutions that Red Hat engineers have created while supporting our customers. To give you the knowledge you need the instant it becomes available, these articles may be presented in a raw and unedited form.
Welcome! Check out the Getting Started with Red Hat page for quick tours and guides for common tasks.
