The Fidelis Agent is registering an enormous number of kretprobes, resulting in the enlargement of kmalloc-64 slab cache, leading to OOM (Out of Memory) issues.

Issue

• The Fidelis Agent is registering an enormous number of kretprobes, resulting in the enlargement of kmalloc-64 slab cache, leading to OOM (Out of Memory) issues.
• The memory usage report dumped during the OOM killing event suggests that a staggering 94% of the installed RAM is being used by the kmalloc-64 slab cache. There are no other processes that appear to be consuming a large amount of memory on their own.

Jun 26 12:18:53 localhost kernel: Out of memory: Killed process 1311 (slapd) total-vm:43498700kB, anon-rss:0kB, file-rss:0kB, shmem-rss:0kB, UID:55 pgtables:648kB oom_score_adj:0
Jun 26 12:18:53 localhost kernel: HangDetector invoked oom-killer: gfp_mask=0x6200ca(GFP_HIGHUSER_MOVABLE), order=0, oom_score_adj=0
Jun 26 12:18:53 localhost kernel: CPU: 1 PID: 959 Comm: HangDetector Not tainted 4.18.0-553.5.1.el8_10.x86_64 #1
    ...
Jun 26 12:18:53 localhost kernel: Mem-Info:
Jun 26 12:18:53 localhost kernel: active_anon:16 inactive_anon:23 isolated_anon:1#012 active_file:23 inactive_file:1441 isolated_file:97#012 unevictable:9 dirty:0 writeback:2#012 slab_reclaimable:23876 slab_unreclaimable:3931111#012 mapped:11273 shmem:0 pagetables:3611 bounce:0#012 free:33063 free_pcp:300 free_cma:0
Jun 26 12:18:53 localhost kernel: Node 0 active_anon:64kB inactive_anon:92kB active_file:92kB inactive_file:5764kB unevictable:36kB isolated(anon):4kB isolated(file):388kB mapped:45092kB dirty:0kB writeback:8kB shmem:0kB shmem_thp: 0kB shmem_pmdmapped: 0kB anon_thp: 0kB writeback_tmp:0kB kernel_stack:4560kB pagetables:14444kB all_unreclaimable? no
Jun 26 12:18:53 localhost kernel: Node 0 DMA free:13312kB min:60kB low:72kB high:84kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB writepending:0kB present:15992kB managed:15360kB mlocked:0kB bounce:0kB free_pcp:0kB local_pcp:0kB free_cma:0kB
Jun 26 12:18:53 localhost kernel: lowmem_reserve[]: 0 2991 16002 16002 16002
Jun 26 12:18:53 localhost kernel: Node 0 DMA32 free:64148kB min:12536kB low:15668kB high:18800kB active_anon:8kB inactive_anon:52kB active_file:0kB inactive_file:1180kB unevictable:0kB writepending:0kB present:3129216kB managed:3063680kB mlocked:0kB bounce:0kB free_pcp:912kB local_pcp:376kB free_cma:0kB
Jun 26 12:18:53 localhost kernel: lowmem_reserve[]: 0 0 13010 13010 13010
Jun 26 12:18:53 localhost kernel: Node 0 Normal free:54792kB min:54984kB low:68728kB high:82472kB active_anon:56kB inactive_anon:40kB active_file:0kB inactive_file:5468kB unevictable:36kB writepending:8kB present:13631488kB managed:13323080kB mlocked:36kB bounce:0kB free_pcp:288kB local_pcp:144kB free_cma:0kB
Jun 26 12:18:53 localhost kernel: lowmem_reserve[]: 0 0 0 0 0
Jun 26 12:18:53 localhost kernel: Node 0 DMA: 0*4kB 0*8kB 0*16kB 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 1*1024kB (U) 2*2048kB (UM) 2*4096kB (M) = 13312kB
Jun 26 12:18:53 localhost kernel: Node 0 DMA32: 297*4kB (UME) 26*8kB (ME) 33*16kB (UE) 52*32kB (UME) 22*64kB (ME) 11*128kB (UME) 3*256kB (UE) 4*512kB (ME) 4*1024kB (UM) 25*2048kB (UM) 0*4096kB = 64516kB
Jun 26 12:18:53 localhost kernel: Node 0 Normal: 6615*4kB (UME) 309*8kB (UME) 323*16kB (UME) 292*32kB (UME) 74*64kB (UME) 16*128kB (M) 7*256kB (UM) 3*512kB (UM) 2*1024kB (M) 0*2048kB 0*4096kB = 55604kB
Jun 26 12:18:53 localhost kernel: Node 0 hugepages_total=0 hugepages_free=0 hugepages_surp=0 hugepages_size=1048576kB
Jun 26 12:18:53 localhost kernel: Node 0 hugepages_total=0 hugepages_free=0 hugepages_surp=0 hugepages_size=2048kB
Jun 26 12:18:53 localhost kernel: 1597 total pagecache pages
Jun 26 12:18:53 localhost kernel: 35 pages in swap cache
Jun 26 12:18:53 localhost kernel: Swap cache stats: add 373230625, delete 373248943, find 101840328/283898171
Jun 26 12:18:53 localhost kernel: Free swap  = 3861384kB
Jun 26 12:18:53 localhost kernel: Total swap = 4169724kB
Jun 26 12:18:53 localhost kernel: 4194174 pages RAM
Jun 26 12:18:53 localhost kernel: 0 pages HighMem/MovableOnly
Jun 26 12:18:53 localhost kernel: 93644 pages reserved
Jun 26 12:18:53 localhost kernel: 0 pages hwpoisoned
Jun 26 12:18:53 localhost kernel: Unreclaimable slab info:
Jun 26 12:18:53 localhost kernel: Name                      Used          Total
Jun 26 12:18:53 localhost kernel: nf_conntrack              30KB         30KB
Jun 26 12:18:53 localhost kernel: bridge_fdb_cache           4KB          4KB
Jun 26 12:18:53 localhost kernel: AF_VSOCK                  61KB         61KB
Jun 26 12:18:53 localhost kernel: rpc_buffers               32KB         32KB
Jun 26 12:18:53 localhost kernel: rpc_tasks                  4KB          4KB
Jun 26 12:18:53 localhost kernel: xfs_bui_item               3KB          3KB
Jun 26 12:18:53 localhost kernel: xfs_bud_item               3KB          3KB
Jun 26 12:18:53 localhost kernel: xfs_cui_item               7KB          7KB
Jun 26 12:18:53 localhost kernel: xfs_cud_item               3KB          3KB
Jun 26 12:18:53 localhost kernel: xfs_efi_item             118KB        118KB
Jun 26 12:18:53 localhost kernel: xfs_efd_item              91KB         91KB
Jun 26 12:18:53 localhost kernel: xfs_buf_item              65KB         65KB
Jun 26 12:18:53 localhost kernel: xfs_trans                  7KB          7KB
Jun 26 12:18:53 localhost kernel: xfs_ifork                 71KB        203KB
Jun 26 12:18:53 localhost kernel: xfs_da_state              15KB         15KB
Jun 26 12:18:53 localhost kernel: xfs_btree_cur              7KB          7KB
Jun 26 12:18:53 localhost kernel: xfs_bmap_free_item          7KB          7KB
Jun 26 12:18:53 localhost kernel: xfs_log_ticket             7KB          7KB
Jun 26 12:18:53 localhost kernel: bio-200                    8KB          8KB
Jun 26 12:18:53 localhost kernel: bio-272                   63KB         63KB
Jun 26 12:18:53 localhost kernel: sd_ext_cdb                 4KB          4KB
Jun 26 12:18:53 localhost kernel: bio-160                   15KB         15KB
Jun 26 12:18:53 localhost kernel: scsi_sense_cache         196KB        196KB
Jun 26 12:18:53 localhost kernel: fuse_request               7KB          7KB
Jun 26 12:18:53 localhost kernel: fib6_nodes                 8KB          8KB
Jun 26 12:18:53 localhost kernel: ip6_dst_cache              7KB          7KB
Jun 26 12:18:53 localhost kernel: RAWv6                     63KB         63KB
Jun 26 12:18:53 localhost kernel: UDPv6                     63KB         63KB
Jun 26 12:18:53 localhost kernel: TCPv6                     61KB         61KB
Jun 26 12:18:53 localhost kernel: sgpool-128               224KB        288KB
Jun 26 12:18:53 localhost kernel: sgpool-64                128KB        128KB
Jun 26 12:18:53 localhost kernel: sgpool-32                 64KB         64KB
Jun 26 12:18:53 localhost kernel: sgpool-16                 40KB         40KB
Jun 26 12:18:53 localhost kernel: sgpool-8                  88KB         88KB
Jun 26 12:18:53 localhost kernel: mqueue_inode_cache         16KB         16KB
Jun 26 12:18:53 localhost kernel: kioctx                    31KB         31KB
Jun 26 12:18:53 localhost kernel: aio_kiocb                  7KB          7KB
Jun 26 12:18:53 localhost kernel: bio-248                    8KB          8KB
Jun 26 12:18:53 localhost kernel: pid_namespace              7KB          7KB
Jun 26 12:18:53 localhost kernel: posix_timers_cache          7KB          7KB
Jun 26 12:18:53 localhost kernel: UNIX                     265KB        425KB
Jun 26 12:18:53 localhost kernel: tcp_bind_bucket            8KB          8KB
Jun 26 12:18:53 localhost kernel: ip_fib_trie                7KB          7KB
Jun 26 12:18:53 localhost kernel: ip_fib_alias               7KB          7KB
Jun 26 12:18:53 localhost kernel: ip_dst_cache               8KB          8KB
Jun 26 12:18:53 localhost kernel: RAW                       47KB         47KB
Jun 26 12:18:53 localhost kernel: UDP                       45KB         45KB
Jun 26 12:18:53 localhost kernel: tw_sock_TCP                7KB          7KB
Jun 26 12:18:53 localhost kernel: request_sock_TCP          59KB         59KB
Jun 26 12:18:53 localhost kernel: TCP                      411KB        411KB
Jun 26 12:18:53 localhost kernel: hugetlbfs_inode_cache          7KB          7KB
Jun 26 12:18:53 localhost kernel: bio-280                  101KB        101KB
Jun 26 12:18:53 localhost kernel: eventpoll_pwq             55KB         55KB
Jun 26 12:18:53 localhost kernel: eventpoll_epi            100KB        100KB
Jun 26 12:18:53 localhost kernel: inotify_inode_mark         23KB         23KB
Jun 26 12:18:53 localhost kernel: request_queue            222KB        222KB
Jun 26 12:18:53 localhost kernel: blkdev_ioc                43KB         43KB
Jun 26 12:18:53 localhost kernel: bio-224                   56KB         56KB
Jun 26 12:18:53 localhost kernel: biovec-max               448KB        512KB
Jun 26 12:18:53 localhost kernel: biovec-128               128KB        128KB
Jun 26 12:18:53 localhost kernel: biovec-64                 96KB         96KB
Jun 26 12:18:53 localhost kernel: biovec-16                 56KB         56KB
Jun 26 12:18:53 localhost kernel: bio_integrity_payload          4KB          4KB
Jun 26 12:18:53 localhost kernel: khugepaged_mm_slot          7KB          7KB
Jun 26 12:18:53 localhost kernel: user_namespace            15KB         15KB
Jun 26 12:18:53 localhost kernel: uid_cache                  7KB          7KB
Jun 26 12:18:53 localhost kernel: dmaengine-unmap-256         30KB         30KB
Jun 26 12:18:53 localhost kernel: dmaengine-unmap-128         15KB         15KB
Jun 26 12:18:53 localhost kernel: dmaengine-unmap-16          3KB          3KB
Jun 26 12:18:53 localhost kernel: dmaengine-unmap-2          4KB          4KB
Jun 26 12:18:53 localhost kernel: audit_buffer               7KB          7KB
Jun 26 12:18:53 localhost kernel: skbuff_fclone_cache         86KB         86KB
Jun 26 12:18:53 localhost kernel: skbuff_head_cache        605KB        640KB
Jun 26 12:18:53 localhost kernel: configfs_dir_cache          3KB          3KB
Jun 26 12:18:53 localhost kernel: file_lock_cache           15KB         15KB
Jun 26 12:18:53 localhost kernel: file_lock_ctx             19KB         19KB
Jun 26 12:18:53 localhost kernel: fsnotify_mark_connector          7KB          7KB
Jun 26 12:18:53 localhost kernel: net_namespace             63KB         63KB
Jun 26 12:18:53 localhost kernel: task_delay_info           55KB         55KB
Jun 26 12:18:53 localhost kernel: taskstats                 15KB         15KB
Jun 26 12:18:53 localhost kernel: proc_dir_entry           260KB        260KB
Jun 26 12:18:53 localhost kernel: pde_opener                 7KB          7KB
Jun 26 12:18:53 localhost kernel: seq_file                   8KB          8KB
Jun 26 12:18:53 localhost kernel: sigqueue                   7KB          7KB
Jun 26 12:18:53 localhost kernel: shmem_inode_cache        938KB       1018KB
Jun 26 12:18:53 localhost kernel: kernfs_iattrs_cache         46KB         55KB
Jun 26 12:18:53 localhost kernel: kernfs_node_cache       5056KB       5056KB
Jun 26 12:18:53 localhost kernel: mnt_cache                346KB        409KB
Jun 26 12:18:53 localhost kernel: filp                     796KB       1136KB
Jun 26 12:18:53 localhost kernel: names_cache              128KB        128KB
Jun 26 12:18:53 localhost kernel: avc_node                   7KB          7KB
Jun 26 12:18:53 localhost kernel: lsm_inode_cache         5610KB       8208KB
Jun 26 12:18:53 localhost kernel: lsm_file_cache            90KB        128KB
Jun 26 12:18:53 localhost kernel: key_jar                   44KB         44KB
Jun 26 12:18:53 localhost kernel: nsproxy                    7KB          7KB
Jun 26 12:18:53 localhost kernel: vm_area_struct          2491KB       2499KB
Jun 26 12:18:53 localhost kernel: mm_struct                315KB        315KB
Jun 26 12:18:53 localhost kernel: fs_cache                  16KB         16KB
Jun 26 12:18:53 localhost kernel: files_cache              158KB        158KB
Jun 26 12:18:53 localhost kernel: signal_cache             281KB        345KB
Jun 26 12:18:53 localhost kernel: sighand_cache            441KB        556KB
Jun 26 12:18:53 localhost kernel: task_struct             3348KB       3417KB
Jun 26 12:18:53 localhost kernel: cred_jar                 115KB        145KB
Jun 26 12:18:53 localhost kernel: anon_vma_chain           401KB        520KB
Jun 26 12:18:53 localhost kernel: anon_vma                 354KB        484KB
Jun 26 12:18:53 localhost kernel: pid                      100KB        100KB
Jun 26 12:18:53 localhost kernel: Acpi-Operand             397KB        397KB
Jun 26 12:18:53 localhost kernel: Acpi-ParseExt              7KB          7KB
Jun 26 12:18:53 localhost kernel: Acpi-Parse                15KB         15KB
Jun 26 12:18:53 localhost kernel: Acpi-State                15KB         15KB
Jun 26 12:18:53 localhost kernel: Acpi-Namespace           179KB        179KB
Jun 26 12:18:53 localhost kernel: numa_policy                7KB          7KB
Jun 26 12:18:53 localhost kernel: perf_event               174KB        174KB
Jun 26 12:18:53 localhost kernel: trace_event_file        1051KB       1051KB
Jun 26 12:18:53 localhost kernel: ftrace_event_field       1175KB       1175KB
Jun 26 12:18:53 localhost kernel: pool_workqueue            12KB         12KB
Jun 26 12:18:53 localhost kernel: task_group                94KB         94KB
Jun 26 12:18:53 localhost kernel: vmap_area                128KB        144KB
Jun 26 12:18:53 localhost kernel: kmalloc-cg-8k             64KB         64KB
Jun 26 12:18:53 localhost kernel: kmalloc-cg-4k            128KB        128KB
Jun 26 12:18:53 localhost kernel: kmalloc-cg-2k            192KB        192KB
Jun 26 12:18:53 localhost kernel: kmalloc-cg-1k            269KB        320KB
Jun 26 12:18:53 localhost kernel: kmalloc-cg-512           176KB        176KB
Jun 26 12:18:53 localhost kernel: kmalloc-cg-256             8KB          8KB
Jun 26 12:18:53 localhost kernel: kmalloc-cg-192            27KB         27KB
Jun 26 12:18:53 localhost kernel: kmalloc-cg-128             8KB          8KB
Jun 26 12:18:53 localhost kernel: kmalloc-cg-96              7KB          7KB
Jun 26 12:18:53 localhost kernel: kmalloc-cg-64             12KB         12KB
Jun 26 12:18:53 localhost kernel: kmalloc-cg-32              8KB          8KB
Jun 26 12:18:53 localhost kernel: kmalloc-cg-16              8KB          8KB
Jun 26 12:18:53 localhost kernel: kmalloc-cg-8               8KB          8KB
Jun 26 12:18:53 localhost kernel: kmalloc-8k               336KB        480KB
Jun 26 12:18:54 localhost kernel: kmalloc-4k              3944KB       4224KB
Jun 26 12:18:54 localhost kernel: kmalloc-2k              2164KB       2656KB
Jun 26 12:18:54 localhost kernel: kmalloc-1k              1305KB       1824KB
Jun 26 12:18:54 localhost kernel: kmalloc-512             5909KB       6064KB
Jun 26 12:18:54 localhost kernel: kmalloc-256              198KB        296KB
Jun 26 12:18:54 localhost kernel: kmalloc-192             1640KB       3118KB
Jun 26 12:18:54 localhost kernel: kmalloc-128              398KB        948KB
Jun 26 12:18:54 localhost kernel: kmalloc-96               805KB        960KB
Jun 26 12:18:54 localhost kernel: kmalloc-64          15415380KB   15415430KB <<-------
Jun 26 12:18:54 localhost kernel: kmalloc-32              1898KB       2260KB
Jun 26 12:18:54 localhost kernel: kmalloc-16               278KB        484KB
Jun 26 12:18:54 localhost kernel: kmalloc-8                143KB        168KB
Jun 26 12:18:54 localhost kernel: kmem_cache_node           20KB         20KB
Jun 26 12:18:54 localhost kernel: kmem_cache               118KB        118KB
Jun 26 12:18:54 localhost kernel: Tasks state (memory values in pages):
Jun 26 12:18:54 localhost kernel: [  pid  ]   uid  tgid total_vm      rss pgtables_bytes swapents oom_score_adj name
Jun 26 12:18:54 localhost kernel: [    703]     0   703    47187        0   425984      655             0 systemd-journal
Jun 26 12:18:54 localhost kernel: [    742]     0   742    24584        0   204800      628         -1000 systemd-udevd
Jun 26 12:18:54 localhost kernel: [    851]    32   851    16832        0   167936      215             0 rpcbind
Jun 26 12:18:54 localhost kernel: [    854]     0   854    32746        0   143360      262         -1000 auditd
Jun 26 12:18:54 localhost kernel: [    856]     0   856    12181        0   143360      118             0 sedispatch
Jun 26 12:18:54 localhost kernel: [    882]     0   882    31274        0   147456      197             0 irqbalance
Jun 26 12:18:54 localhost kernel: [    883]    81   883    13944        0   147456      584          -900 dbus-daemon
       ...
Jun 26 12:18:54 localhost kernel: oom-kill:constraint=CONSTRAINT_NONE,nodemask=(null),cpuset=/,mems_allowed=0,global_oom,task_memcg=/system.slice/protect.service,task=protect,pid=456835,uid=0
Jun 26 12:18:54 localhost kernel: Out of memory: Killed process 456835 (protect) total-vm:1087652kB, anon-rss:0kB, file-rss:37864kB, shmem-rss:0kB, UID:0 pgtables:376kB oom_score_adj:0