4.3. Per-group Division of CPU and Memory Resources

Procedure 4.3. Per-group CPU and memory resource management

1. In the /etc/cgconfig.conf file, configure the following subsystems to be mounted and cgroups to be created:

   mount {
       cpu     = /cgroup/cpu_and_mem;
       cpuacct = /cgroup/cpu_and_mem;
       memory  = /cgroup/cpu_and_mem;
   }
   
   group finance {
           cpu {
                   cpu.shares="250";
           }
           cpuacct {
                   cpuacct.usage="0";
           }
           memory {
                   memory.limit_in_bytes="2G";
                   memory.memsw.limit_in_bytes="3G";
           }
   }
   
   group sales {
           cpu {
                   cpu.shares="250";
           }
           cpuacct {
                   cpuacct.usage="0";
           }
           memory {
                   memory.limit_in_bytes="4G";
                   memory.memsw.limit_in_bytes="6G";
           }
   }
   
   group engineering {
           cpu {
                   cpu.shares="500";
           }
           cpuacct {
                   cpuacct.usage="0";
           }
           memory {
                   memory.limit_in_bytes="8G";
                   memory.memsw.limit_in_bytes="16G";
           }
   }
   

   When loaded, the above configuration file mounts the cpu, cpuacct, and memory subsystems to a single cpu_and_mem cgroup. For more information on these subsystems, refer to Chapter 3, Subsystems and Tunable Parameters. Next, it creates a hierarchy in cpu_and_mem which contains three cgroups: sales, finance, and engineering. In each of these cgroups, custom parameters are set for each subsystem:
   • cpu — the cpu.shares parameter determines the share of CPU resources available to each process in all cgroups. Setting the parameter to 250, 250, and 500 in the finance, sales, and engineering cgroups respectively means that processes started in these groups will split the resources with a 1:1:2 ratio. Note that when a single process is running, it consumes as much CPU as necessary no matter which cgroup it is placed in. The CPU limitation only comes into effect when two or more processes compete for CPU resources.
   • cpuacct — the cpuacct.usage="0" parameter is used to reset values stored in the cpuacct.usage and cpuacct.usage_percpu files. These files report total CPU time (in nanoseconds) consumed by all processes in a cgroup.
   • memory — the memory.limit_in_bytes parameter represents the amount of memory that is made available to all processes within a certain cgroup. In our example, processes started in the finance cgroup have 2 GB of memory available, processes in the sales group have 4 GB of memory available, and processes in the engineering group have 8 GB of memory available. The memory.memsw.limit_in_bytes parameter specifies the total amount of memory and swap space processes may use. Should a process in the finance cgroup hit the 2 GB memory limit, it is allowed to use another 1 GB of swap space, thus totaling the configured 3 GB.
2. To define the rules which the cgrulesengd daemon uses to move processes to specific cgroups, configure the /etc/cgrules.conf in the following way:

   #<user/group>         <controller(s)>         <cgroup>
   @finance              cpu,cpuacct,memory      finance
   @sales                cpu,cpuacct,memory      sales
   @engineering          cpu,cpuacct,memory      engineering
   

   The above configuration creates rules that assign a specific system group (for example, @finance) the resource controllers it may use (for example, cpu, cpuacct, memory) and a cgroup (for example, finance) which contains all processes originating from that system group.
   In our example, when the cgrulesengd daemon, started via the service cgred start command, detects a process that is started by a user that belongs to the finance system group (for example, jenn), that process is automatically moved to the /cgroup/cpu_and_mem/finance/tasks file and is subjected to the resource limitations set in the finance cgroup.
3. Start the cgconfig service to create the hierarchy of cgroups and set the needed parameters in all created cgroups:

   ~]# service cgconfig start
   Starting cgconfig service:                                 [  OK  ]
   

   Start the cgred service to let the cgrulesengd daemon detect any processes started in system groups configured in the /etc/cgrules.conf file:

   ~]# service cgred start
   Starting CGroup Rules Engine Daemon:                       [  OK  ]
   

   Note that cgred is the name of the service that starts the cgrulesengd daemon.
4. To make all of the changes above persistent across reboots, configure the cgconfig and cgred services to be started by default:

   ~]# chkconfig cgconfig on
   ~]# chkconfig cgred on

Procedure 4.4. Using a PAM module to move processes to cgroups

1. Install the libcgroup-pam package from the optional Red Hat Enterprise Linux Yum repository:

   ~]# yum install libcgroup-pam --enablerepo=rhel-6-server-optional-rpms

2. Ensure that the PAM module has been installed and exists:

   ~]# ls /lib64/security/pam_cgroup.so
   /lib64/security/pam_cgroup.so
   

   Note that on 32-bit systems, the module is placed in the /lib/security directory.
3. Add the following line to the /etc/pam.d/su file to use the pam_cgroup.so module each time the su command is executed:

   session         optional        pam_cgroup.so
   

4. Configure the /etc/cgconfig.conf and /etc/cgrules.conf files as in Procedure 4.3, “Per-group CPU and memory resource management”.
5. Log out all users that are affected by the cgroup settings in the /etc/cgrules.conf file to apply the above configuration.