OOM may occur even if memory is limited by setting the MemoryLimit directive for a systemd unit.
Issue
- Add or modify the
MemoryLimit
setting for example.service, and reflect the setting as the following:
# EDITOR=vim systemctl edit example.service
[Service]
MemoryLimit=100G
# cat /etc/systemd/system/example.service.d/override.conf
[Service]
MemoryLimit=100G
# systemctl daemon-reload
# systemctl restart example.service
- OOM occurs even though the total RSS of all processes and threads created by example.service is less than the value set in
MemoryLimit
. The free space in the swap area is 0.
example.proc invoked oom-killer: gfp_mask=0x280da, order=0, oom_score_adj=0
example.proc cpuset=/ mems_allowed=0-1
CPU: 5 PID: 198862 Comm: example.proc Kdump: loaded Tainted: G OE ------------ 3.10.0-957.5.1.el7.x86_64 #1
Hardware name: Supermicro SYS-2029TP-OSK-NE010/X11DPT-PS, BIOS 1.0b 09/23/2017
Call Trace:
[<ffffffffb4161e41>] dump_stack+0x19/0x1b
[<ffffffffb415c86a>] dump_header+0x90/0x229
[<ffffffffb3b01092>] ? ktime_get_ts64+0x52/0xf0
[<ffffffffb3b5822f>] ? delayacct_end+0x8f/0xb0
[<ffffffffb3bba524>] oom_kill_process+0x254/0x3d0
[<ffffffffb3bb9fcd>] ? oom_unkillable_task+0xcd/0x120
[<ffffffffb3bba076>] ? find_lock_task_mm+0x56/0xc0
[<ffffffffb3bbad66>] out_of_memory+0x4b6/0x4f0
[<ffffffffb415d36e>] __alloc_pages_slowpath+0x5d6/0x724
[<ffffffffb3bc1145>] __alloc_pages_nodemask+0x405/0x420
[<ffffffffb3c11535>] alloc_pages_vma+0xb5/0x200
[<ffffffffb3be9a77>] handle_pte_fault+0x887/0xd10
[<ffffffffb3bec01d>] handle_mm_fault+0x39d/0x9b0
[<ffffffffb416f5e3>] __do_page_fault+0x203/0x500
[<ffffffffb416f915>] do_page_fault+0x35/0x90
[<ffffffffb416b758>] page_fault+0x28/0x30
Mem-Info:
[...]
Free swap = 0kB <<<<<<<< Here!!
Total swap = 41943040kB
[...]
Killed process 197501 (example.proc) total-vm:14781324kB, anon-rss:8346756kB, file-rss:0kB, shmem-rss:50712kB
[...]
Killed process 197620 (example.proc) total-vm:14785944kB, anon-rss:8950668kB, file-rss:0kB, shmem-rss:54468kB
[...]
Killed process 197713 (example.proc) total-vm:14749496kB, anon-rss:9438372kB, file-rss:0kB, shmem-rss:137588kB
Environment
- Red Hat Enterprise Linux 7
Subscriber exclusive content
A Red Hat subscription provides unlimited access to our knowledgebase, tools, and much more.