Need explanation on RHEL kernels behaviour about cache, shared, swap and anonymous memory segments

Resolution

1. why cache memory ?

Cache memory is defined as file-backed data that is resident in RAM. For example, when an application requires a file to process a request, that file is read from disk into memory by the kernel. The application (for instance, Oracle) uses the file, completes its processing, and releases it.

At this point, the kernel may either retain the file in memory or discard it if the memory is required elsewhere. If the file is discarded to free memory and the application later requests the same file again, the kernel must fetch it from disk once more, resulting in an expensive I/O operation (as disk access is orders of magnitude slower than RAM). The file is used again, discarded again, and this cycle continues repeatedly.

This behavior leads to significant performance degradation, as frequent disk I/O operations are repeatedly performed. To avoid this inefficiency, the Linux kernel is designed to retain recently accessed file data in memory. After the application finishes using the file, the data is kept in RAM, and this retained file-backed data constitutes cache memory.

Because this cached data originates from disk, it can be discarded at any time if memory needs to be reclaimed. The kernel continuously manages cache health in the background, primarily through the kswapd thread. Memory regions such as active, inactive, and LRU lists are maintained by the kernel, and their state can be observed in /proc/meminfo. Inactive cache pages are periodically reclaimed when they are no longer required.

As a result, the kernel ensures that cache memory remains healthy and balanced. This behavior highlights the importance of cache memory and explains why it cannot and should not be disabled. A healthy amount of cache memory is an indicator of an efficiently functioning system and should not be considered a negative condition.

2. why swap memory usage when cache is available ?

Swap is treated as a separate memory segment, and no direct relationship exists between cache memory and swap. This distinction can be understood as follows.

A category of memory known as anonymous memory is defined as the logical opposite of cache memory. While cache memory is file-backed (meaning the corresponding data exists as a file on disk), anonymous memory is not backed by any file. In other words, the memory pages do not correspond to any on-disk file.

Anonymous memory is allocated through mechanisms such as malloc() and mmap() system calls. This type of memory represents the heap, stack, and dynamically allocated regions of a process. As a result, anonymous memory is accounted for under the “used” memory section in the output of the free command and primarily reflects user-space application memory consumption.

Because anonymous memory is dynamically allocated and lacks any file backing, it must be written to swap when memory reclamation becomes necessary. In practical terms, only anonymous memory and shared memory backed by normal pages (such as memory allocated using shmget(), shm_open(), or ftruncate()) are eligible for swapping. No other memory types are considered swappable.

In this specific case, shared memory is allocated using hugepages rather than normal pages. Hugepages are strictly non-swappable by design. As a result, the Oracle data stored in hugepage-backed shared memory will never be swapped out.

why is kernel using swap space when there is a GBs of cache memory is available ?

By this point, it should be clear that cache memory and swap are not directly related. The kernel is also responsible for managing anonymous memory based on its overall health and usage patterns. Similar to cache memory, anonymous memory is organized into active and inactive lists and maintained using LRU mechanisms.

Based on internal kernel heuristics, decisions are made regarding when anonymous memory should be swapped out or swapped back in. These decisions are driven by memory pressure, access patterns, and reclaim policies, rather than by any direct interaction with cache memory.

As a result, the kernel cannot be explicitly instructed to avoid swapping by favoring cache memory, because these two memory types belong to entirely different memory domains and serve different purposes. Cache memory can be reclaimed simply by dropping file-backed pages, whereas anonymous memory requires swap space when it must be evicted from RAM.

Although direct control is not possible, kernel behavior can be influenced indirectly. By lowering the 'vm.swappiness' kernel parameter, the tendency of the kernel to swap out anonymous memory can be reduced, thereby encouraging memory reclamation to be performed through other means first.

3. What if one does not want swap usage at all

As explained earlier, the kernel cannot be completely prevented from using swap, as no such direct configuration option exists. Kernel behavior can only be influenced indirectly by lowering the 'vm.swappiness' kernel parameter.

If swap usage is still required to be completely avoided, swap must be disabled altogether. This can be achieved by unmounting or turning off the swap device.

The impact of this approach should be clearly understood. When swap is disabled, anonymous memory cannot be swapped out, even if it becomes entirely inactive. Consequently, such memory remains resident in RAM and continues to be accounted for under the “used” section in the output of the free command.