The cpu controller in cgroup v2 can not be used in Red Hat Enterprise Linux 8

Solution Verified - Updated -

Environment

  • Red Hat Enterprise Linux 8

Issue

  • I enabled cgroup v2 on Red Hat Enterprise Linux 8 with a boot parameter systemd.unified_cgroup_hierarchy=1, and wanted to delegate some controllers to non-privileged users.
  • It worked for all controllers except for the cpu controller.
  • It was not possible to add the cpu controller manually.

  • Enabling the CPU controller fails with the following error:

    # echo +cpu > /sys/fs/cgroup/cgroup.subtree_control
-bash: echo: write error: Invalid argument

Resolution

  • The CPU controller for CGroups V2 will not mount if Real Time, non-kernel thread processes exist on the system.
  • The application running with Real Time scheduling policies must be stopped in order to use the CPU controller.

Root Cause

  • Some applications are known to run with Real Time priority and can trigger the issue. If unnecessary, the applications may be disabled. Below is a non-exhaustive list of example applications known to trigger the issue by running in Real Time scheduling policies;

Diagnostic Steps

  • Check if any Real Time scheduled processes are running
         r8 # chrt -r 40 sleep 5000 &  # example sleep process running in real time
         r8 # ps axo pid,cls,cmd | awk '$2 ~ /(FF|RR)/' | grep -v '[[]'
           1831  RR sleep 5000

         r8 # echo +cpu > cgroup.subtree_control 
        -bash: echo: write error: Invalid argument
  • Alternatively, follow below command to find processes have scheduling class as RR (SCHED_RR).
  • Here, timekeeper was running with Real Time scheduling policies(RR). It must be stopped or killed in order to add cpu again in cpu controller manually.
        $ ps -T axo pid,ppid,user,group,lwp,nlwp,start_time,comm,cgroup,cls|grep RR
        1 895    1870 root     root    1936   15 Jan09 tk_status_file  0::/system.slice/timekeeper  RR

        $ ps -fe | grep 1895
        root        1895    1870  0 Jan09 ?        00:02:42 /opt/timekeeper/release64/timekeeperapp

This solution is part of Red Hat’s fast-track publication program, providing a huge library of solutions that Red Hat engineers have created while supporting our customers. To give you the knowledge you need the instant it becomes available, these articles may be presented in a raw and unedited form.

Comments