Which amount of CPU and memory are recommended to reserve for the system in OCP 4 nodes?
Environment
- Red Hat OpenShift Container Platform (RHOCP)
- 4
Issue
- Recommended
systemReserved
values for OpenShift 4 clusters.
Resolution
Disclaimer: Links contained herein to external website(s) are provided for convenience only. Red Hat has not reviewed the links and is not responsible for the content or its availability. The inclusion of any link to an external website does not imply endorsement by Red Hat of the website or their entities, products or services. You agree that Red Hat is not responsible or liable for any loss or expenses that may result due to your use of (or reliance on) the external site or content.
Memory
Nodes with 1GiB of memory or less
- 255 MiB.
Nodes with more than 1GiB of memory
- 25% of the first 4 GiB of memory.
- 20% of the next 4 GiB of memory (between 4 GiB and 8 GiB).
- 10% of the next 8 GiB of memory (between 8 GiB and 16 GiB).
- 6% of the next 112GB of memory (up to 128GB)
- 2% of the remaining memory.
CPU
- 6% of the first core.
- 1% of the second core.
- 0.5% of the next 2 cores.
- 0.25% of any remaining core.
Example
Let's say that there is a node with the following hardware:
- CPU: 16 cores.
- Memory: 32 GiB.
Values recommended:
-
Recommended reserved CPU =
60m
+10m
+ (2 *5m
) + (12 *2.5m
) =110m
.- Consequent allocatable CPU =
16000m
-110m
=15890m
.
- Consequent allocatable CPU =
-
Recommended reserved memory = 25% of
4 GiB
+ 20% of4 GiB
+ 10% of8 GiB
+ 6% of16 GiB
=3.56 GiB
.- Consequent allocatable memory =
32 GiB
-3.56 GiB
=28.44 GiB
.
- Consequent allocatable memory =
Configuration
-
For OpenShift up to 4.7 the recommended values had to be set manually by following Modifying nodes.
-
Starting with OpenShift 4.8, the reservation can be configured to be calculated automatically. See Automatically allocating resources for nodes for more information.
- This feature is disabled by default.
-
The values above are the ones applied when the calculation is automatic. If you do not find them suitable for your cluster for any reason, please contact Red Hat Support for advice.
Note: It is also possible to use a script to obtain the memory and CPU values. Refer to the "Diagnostic Steps" section.
Additional information
m
stands for milicore.1000m
=1 core
.- What is the default setting value of reserved and eviction in OpenShift?.
Root Cause
The systemReserved
CPU and memory resources avoid that scheduled pods consume the whole resources of a node, at the expense of the system critical processes like cri-o
, kubelet
, sshd
, NetworkManager
. The entire list of processes using the systemReserved
resources can be shown as explained in the "Diagnostic Steps" section.
Diagnostic Steps
-
Check the entire list of processes included in the resource reservation:
$ oc debug node/[node_name] [...] sh-4.4# chroot /host bash # systemd-cgls /system.slice/
-
It is possible to test the
dynamic-node-sizing.sh
script with paramtrue
to calculate the values for thesystemReserved
the same way the automated configuration in OCP 4.8:$ oc debug node/[node_name] [...] sh-4.4# chroot /host bash # cd /tmp/ # vi /tmp/dynamic-node-sizing.sh #### paste the script code in that file while editing # chmod u+x /tmp/dynamic-node-sizing.sh # NODE_SIZES_ENV=/tmp/node-sizing.txt /tmp/dynamic-node-sizing.sh true # cat /tmp/node-sizing.txt
-
If the automatic allocation of resources is enabled, it is possible to check the values in use checking the
/etc/node-sizing.env
file within the nodes:$ oc debug node/[node_name] [...] sh-4.4# chroot /host bash # cat /etc/node-sizing.env
This solution is part of Red Hat’s fast-track publication program, providing a huge library of solutions that Red Hat engineers have created while supporting our customers. To give you the knowledge you need the instant it becomes available, these articles may be presented in a raw and unedited form.
Comments