Chapter 15. Managing Out of Memory states

Out-of-memory (OOM) is a computing state where all available memory, including swap space, has been allocated. Normally this causes the system to panic and stop functioning as expected. The provided instructions help in avoiding OOM states on your system.

Prerequisites

  • You have root permissions on the system.

15.1. Changing the Out of Memory value

The /proc/sys/vm/panic_on_oom file contains a value which is the switch that controls Out of Memory (OOM) behavior. When the file contains 1, the kernel panics on OOM and stops functioning as expected.

The default value is 0, which instructs the kernel to call the oom_killer() function when the system is in an OOM state. Usually, oom_killer() terminates unnecessary processes, which allows the system to survive.

You can change the value of /proc/sys/vm/panic_on_oom.

Procedure

  1. Display the current value of /proc/sys/vm/panic_on_oom. 

    # cat /proc/sys/vm/panic_on_oom
0

    To change the value in /proc/sys/vm/panic_on_oom:

  2. Echo the new value to /proc/sys/vm/panic_on_oom. 

    # echo 1 > /proc/sys/vm/panic_on_oom
Note

It is recommended that you make the Real-Time kernel panic on OOM (1). Otherwise, when the system encounters an OOM state, it is no longer deterministic.

Verification steps

  1. Display the value of /proc/sys/vm/panic_on_oom. 

    # cat /proc/sys/vm/panic_on_oom
1
  2. Verify that the displayed value matches the value specified.

15.2. Prioritizing processes to kill when in an Out of Memory state

You can prioritize the processes that get terminated by the oom_killer() function. This can ensure that high-priority processes keep running during an OOM state. Each process has a directory, /proc/PID. Each directory includes the following files:

  • oom_adj - Valid scores for oom_adj are in the range -16 to +15. This value is used to calculate the performance footprint of the process, using an algorithm that also takes into account how long the process has been running, among other factors.
  • oom_score - Contains the result of the algorithm calculated using the value in oom_adj.

In an Out of Memory state, the oom_killer() function terminates processes with the highest oom_score.

You can prioritize the processes to terminate by editing the oom_adj file for the process.

Prerequisites

  • Know the process ID (PID) of the process you want to prioritize.

Procedure

  1. Display the current oom_score for a process. 

    # cat /proc/12465/oom_score
79872

  2. Display the contents of oom_adj for the process. 

    # cat /proc/12465/oom_adj
13

  3. Edit the value in oom_adj. 

    # echo -5 > /proc/12465/oom_adj

Verification steps

  1. Display the current oom_score for the process. 

    # cat /proc/12465/oom_score
78
  2. Verify that the displayed value is lower than the previous value.

15.3. Disabling the Out of Memory killer for a process

You can disable the oom_killer() function for a process by setting oom_adj to the reserved value of -17. This will keep the process alive, even in an OOM state.

Procedure

  • Set the value in oom_adj to -17. 

    # echo -17 > /proc/12465/oom_adj

Verification steps

  1. Display the current oom_score for the process. 

    # cat /proc/12465/oom_score
0
  2. Verify that the displayed value is 0.