cpusubsystem schedules CPU access to cgroups. Access to CPU resources can be scheduled using two schedulers:
- Completely Fair Scheduler (CFS) — a proportional share scheduler which divides the CPU time (CPU bandwidth) proportionately between groups of tasks (cgroups) depending on the priority/weight of the task or shares assigned to cgroups. For more information about resource limiting using CFS, refer to Section 3.2.1, “CFS Tunable Parameters”.
- Real-Time scheduler (RT) — a task scheduler that provides a way to specify the amount of CPU time that real-time tasks can use. For more information about resource limiting of real-time tasks, refer to Section 3.2.2, “RT Tunable Parameters”.
3.2.1. CFS Tunable Parameters
In CFS, a cgroup can get more than its share of CPU if there are enough idle CPU cycles available in the system, due to the work conserving nature of the scheduler. This is usually the case for cgroups that consume CPU time based on relative shares. Ceiling enforcement can be used for cases when a hard limit on the amount of CPU that a cgroup can utilize is required (that is, tasks cannot use more than a set amount of CPU time).
The following options can be used to configure ceiling enforcement or relative sharing of CPU:
Ceiling Enforcement Tunable Parameters
- specifies a period of time in microseconds (µs, represented here as "
us") for how regularly a cgroup's access to CPU resources should be reallocated. If tasks in a cgroup should be able to access a single CPU for 0.2 seconds out of every 1 second, set
1000000. The upper limit of the
cpu.cfs_quota_usparameter is 1 second and the lower limit is 1000 microseconds.
- specifies the total amount of time in microseconds (µs, represented here as "
us") for which all tasks in a cgroup can run during one period (as defined by
cpu.cfs_period_us). As soon as tasks in a cgroup use up all the time specified by the quota, they are throttled for the remainder of the time specified by the period and not allowed to run until the next period. If tasks in a cgroup should be able to access a single CPU for 0.2 seconds out of every 1 second, set
1000000. Note that the quota and period parameters operate on a CPU basis. To allow a process to fully utilize two CPUs, for example, set
100000.Setting the value in
-1indicates that the cgroup does not adhere to any CPU time restrictions. This is also the default value for every cgroup (except the root cgroup).
- reports CPU time statistics using the following values:
nr_periods— number of period intervals (as specified in
cpu.cfs_period_us) that have elapsed.
nr_throttled— number of times tasks in a cgroup have been throttled (that is, not allowed to run because they have exhausted all of the available time as specified by their quota).
throttled_time— the total time duration (in nanoseconds) for which tasks in a cgroup have been throttled.
Relative Shares Tunable Parameters
- contains an integer value that specifies a relative share of CPU time available to the tasks in a cgroup. For example, tasks in two cgroups that have
100will receive equal CPU time, but tasks in a cgroup that has
200receive twice the CPU time of tasks in a cgroup where
cpu.sharesis set to
100. The value specified in the
cpu.sharesfile must be
2or higher.Note that shares of CPU time are distributed per all CPU cores on multi-core systems. Even if a cgroup is limited to less than 100% of CPU on a multi-core system, it may use 100% of each individual CPU core. Consider the following example: if cgroup
Ais configured to use 25% and cgroup
B75% of the CPU, starting four CPU-intensive processes (one in
Aand three in
B) on a system with four cores results in the following division of CPU shares:
Table 3.1. CPU share division
PID cgroup CPU CPU share 100 A 0 100% of CPU0 101 B 1 100% of CPU1 102 B 2 100% of CPU2 103 B 3 100% of CPU3Using relative shares to specify CPU access has two implications on resource management that should be considered:
- Because the CFS does not demand equal usage of CPU, it is hard to predict how much CPU time a cgroup will be allowed to utilize. When tasks in one cgroup are idle and are not using any CPU time, the leftover time is collected in a global pool of unused CPU cycles. Other cgroups are allowed to borrow CPU cycles from this pool.
- The actual amount of CPU time that is available to a cgroup can vary depending on the number of cgroups that exist on the system. If a cgroup has a relative share of
1000and two other cgroups have a relative share of
500, the first cgroup receives 50% of all CPU time in cases when processes in all cgroups attempt to use 100% of the CPU. However, if another cgroup is added with a relative share of
1000, the first cgroup is only allowed 33% of the CPU (the rest of the cgroups receive 16.5%, 16.5%, and 33% of CPU).