OOM may occur even if memory is limited by setting the MemoryLimit directive for a systemd unit.
Issue
- Add or modify the
MemoryLimitsetting for example.service, and reflect the setting as the following:
# EDITOR=vim systemctl edit example.service
[Service]
MemoryLimit=100G
# cat /etc/systemd/system/example.service.d/override.conf
[Service]
MemoryLimit=100G
# systemctl daemon-reload
# systemctl restart example.service
- OOM occurs even though the total RSS of all processes and threads created by example.service is less than the value set in
MemoryLimit. The free space in the swap area is 0.
example.proc invoked oom-killer: gfp_mask=0x280da, order=0, oom_score_adj=0
example.proc cpuset=/ mems_allowed=0-1
CPU: 5 PID: 198862 Comm: example.proc Kdump: loaded Tainted: G OE ------------ 3.10.0-957.5.1.el7.x86_64 #1
Hardware name: Supermicro SYS-2029TP-OSK-NE010/X11DPT-PS, BIOS 1.0b 09/23/2017
Call Trace:
[<ffffffffb4161e41>] dump_stack+0x19/0x1b
[<ffffffffb415c86a>] dump_header+0x90/0x229
[<ffffffffb3b01092>] ? ktime_get_ts64+0x52/0xf0
[<ffffffffb3b5822f>] ? delayacct_end+0x8f/0xb0
[<ffffffffb3bba524>] oom_kill_process+0x254/0x3d0
[<ffffffffb3bb9fcd>] ? oom_unkillable_task+0xcd/0x120
[<ffffffffb3bba076>] ? find_lock_task_mm+0x56/0xc0
[<ffffffffb3bbad66>] out_of_memory+0x4b6/0x4f0
[<ffffffffb415d36e>] __alloc_pages_slowpath+0x5d6/0x724
[<ffffffffb3bc1145>] __alloc_pages_nodemask+0x405/0x420
[<ffffffffb3c11535>] alloc_pages_vma+0xb5/0x200
[<ffffffffb3be9a77>] handle_pte_fault+0x887/0xd10
[<ffffffffb3bec01d>] handle_mm_fault+0x39d/0x9b0
[<ffffffffb416f5e3>] __do_page_fault+0x203/0x500
[<ffffffffb416f915>] do_page_fault+0x35/0x90
[<ffffffffb416b758>] page_fault+0x28/0x30
Mem-Info:
[...]
Free swap = 0kB <<<<<<<< Here!!
Total swap = 41943040kB
[...]
Killed process 197501 (example.proc) total-vm:14781324kB, anon-rss:8346756kB, file-rss:0kB, shmem-rss:50712kB
[...]
Killed process 197620 (example.proc) total-vm:14785944kB, anon-rss:8950668kB, file-rss:0kB, shmem-rss:54468kB
[...]
Killed process 197713 (example.proc) total-vm:14749496kB, anon-rss:9438372kB, file-rss:0kB, shmem-rss:137588kB
Environment
- Red Hat Enterprise Linux 7
Subscriber exclusive content
A Red Hat subscription provides unlimited access to our knowledgebase, tools, and much more.