The oom_score_adj for aqmon processes, which are run by conmon, should normally be set to 0. However, it is incorrectly set to -1000 after termination by the oom-killer in the context of a Memory Cgroup OOM event.
Issue
- The oom_score_adj for aqmon processes, which are run by conmon, should normally be set to 0. However, it is incorrectly set to -1000 after termination by the oom-killer in the context of a Memory Cgroup OOM event.
- When the processes are running with oom_score_adj of 0 as expected, they can be killed upon memcg OOM as expected:
...
Jun 19 09:44:20 hostname kernel: Tasks state (memory values in pages):
Jun 19 09:44:20 hostname kernel: [ pid ] uid tgid total_vm rss pgtables_bytes swapents oom_score_adj name
Jun 19 09:44:20 hostname kernel: [2005859] 2000 2005859 2929 641 61440 0 0 aqmon.sh <<-------
Jun 19 09:44:20 hostname kernel: [2005874] 2000 2005874 755653 548439 4907008 0 0 aqmon <<-------
Jun 19 09:44:20 hostname kernel: oom-kill:constraint=CONSTRAINT_MEMCG,nodemask=(null),cpuset=libpod-xxx.scope,mems_allowed=0,oom_memcg=/machine.slice/libpod-xxx.scope,task_memcg=/machine.slice/libpod-xxx.scope,task=aqmon,pid=2005874,uid=2000
Jun 19 09:44:20 hostname kernel: Memory cgroup out of memory: Killed process 2005874 (aqmon) total-vm:3022612kB, anon-rss:2089288kB, file-rss:104468kB, shmem-rss:0kB, UID:2000 pgtables:4792kB oom_score_adj:0
...
- On the other hand, when the processes are running with oom_score_adj of -1000, the OOM event occurrence ends up as "no killable processes.":
...
Jun 19 06:01:13 hostname kernel: Tasks state (memory values in pages):
Jun 19 06:01:13 hostname kernel: [ pid ] uid tgid total_vm rss pgtables_bytes swapents oom_score_adj name
Jun 19 06:01:13 hostname kernel: [ 226491] 2000 226491 2929 627 65536 0 -1000 aqmon.sh <<-------
Jun 19 06:01:13 hostname kernel: [ 226504] 2000 226504 736094 527288 4902912 0 -1000 aqmon <<-------
Jun 19 06:01:13 hostname kernel: Out of memory and no killable processes...
...
Environment
- Red Hat Enterprise Linux 8.9.z
- RHEL-shipped conmon and podman-docker packages
Subscriber exclusive content
A Red Hat subscription provides unlimited access to our knowledgebase, tools, and much more.