Interpreting /proc/meminfo and free output for Red Hat Enterprise Linux 5, 6 and 7

Solution Verified - Updated -

Environment

  • Red Hat Enterprise Linux (RHEL) 5
  • Red Hat Enterprise Linux (RHEL) 6
  • Red Hat Enterprise Linux (RHEL) 7

Issue

  • I need an interpretation of /proc/meminfo output.
  • I want to compare the output of free -k to cat /proc/meminfo.

Resolution

Comparing the output

  • free -k output (RHEL 5 and RHEL 6):
             total       used       free     shared    buffers     cached
Mem:       7778104    2971960    4806144          0     211756    1071092
-/+ buffers/cache:    1689112    6088992
Swap:      4194296          0    4194296
  • free -k output (RHEL 7):
              total        used        free      shared  buff/cache   available
Mem:        1012952      252740      158732       11108      601480      543584
Swap:       1048572        5380     1043192
  • Relevant fields from /proc/meminfo to match them against the output of free -k:
MemTotal:        7778104 kB
MemFree:         4806144 kB
Buffers:          211756 kB
Cached:          1071092 kB
SwapTotal:       4194296 kB
SwapFree:        4194296 kB
  • For RHEL 7 there is an additional field available, which is used instead of the calculation for -/+ buffers/cache line:
MemAvailable:     543584 kB

Matching output of free -k to /proc/meminfo

The following table shows how to get the free output matched to the /proc/meminfo fields in Red Hat Enterprise Linux 5, 6 and 7.0.

free output coresponding /proc/meminfo fields
Mem: total MemTotal
Mem: used MemTotal - MemFree
Mem: free MemFree
Mem: shared (can be ignored nowadays. It has no meaning.) N/A
Mem: buffers Buffers
Mem: cached Cached
-/+ buffers/cache: used MemTotal - (MemFree + Buffers + Cached)
-/+ buffers/cache: free MemFree + Buffers + Cached
Swap: total SwapTotal
Swap: used SwapTotal - SwapFree
Swap: free SwapFree

The following table shows how to get the free output matched to the /proc/meminfo fields in Red Hat Enterprise Linux 7.1 or later (procps-ng 3.3.10).

free output coresponding /proc/meminfo fields
Mem: total MemTotal
Mem: used MemTotal - MemFree - Buffers - Cached - Slab
Mem: free MemFree
Mem: shared Shmem
Mem: buff/cache Buffers + Cached + Slab
Mem:available MemAvailable
Swap: total SwapTotal
Swap: used SwapTotal - SwapFree
Swap: free SwapFree

Root Cause

  • Analyzing memory consumption

Diagnostic Steps

Most stuff is taken from the kernel documentation (Documentation/filesystems/proc.txt and Documentation/vm/hugetlbpage.txt)

High Level statistics

RHEL 5, RHEL 6 and RHEL 7
  • MemTotal: Total usable memory
  • MemFree: The amount of physical memory not used by the system
  • Buffers: Memory in buffer cache, so relatively temporary storage for raw disk blocks. This shouldn't get very large.
  • Cached: Memory in the pagecache (Diskcache and Shared Memory)
  • SwapCached: Memory that is present within main memory, but also in the swapfile. (If memory is needed this area does not need to be swapped out AGAIN because it is already in the swapfile. This saves I/O and increases performance if machine runs short on memory.)
RHEL 7 only
  • MemAvailable: An estimate of how much memory is available for starting new applications, without swapping.

Detailed Level statistics

RHEL 5, RHEL 6 and RHEL 7
  • Active: Memory that has been used more recently and usually not swapped out or reclaimed
  • Inactive: Memory that has not been used recently and can be swapped out or reclaimed
RHEL 6 and RHEL 7 only
  • Active(anon): Anonymous memory that has been used more recently and usually not swapped out
  • Inactive(anon): Anonymous memory that has not been used recently and can be swapped out
  • Active(file): Pagecache memory that has been used more recently and usually not reclaimed until needed
  • Inactive(file): Pagecache memory that can be reclaimed without huge performance impact
  • Unevictable: Unevictable pages can't be swapped out for a variety of reasons
  • Mlocked: Pages locked to memory using the mlock() system call. Mlocked pages are also Unevictable.

Memory statistics

RHEL 5, RHEL 6 and RHEL 7
  • SwapTotal: Total swap space available
  • SwapFree: The remaining swap space available
  • Dirty: Memory waiting to be written back to disk
  • Writeback: Memory which is actively being written back to disk
  • AnonPages: Non-file backed pages mapped into userspace page tables
  • Mapped: Files which have been mmaped, such as libraries
  • Slab: In-kernel data structures cache
  • PageTables: Amount of memory dedicated to the lowest level of page tables. This can increase to a high value if a lot of processes are attached to the same shared memory segment.
  • NFS_Unstable: NFS pages sent to the server, but not yet commited to the storage
  • Bounce: Memory used for block device bounce buffers
  • CommitLimit: Based on the overcommit ratio (vm.overcommit_ratio), this is the total amount of memory currently available to be allocated on the system. This limit is only adhered to if strict overcommit accounting is enabled (mode 2 in vm.overcommit_memory).
  • Committed_AS: The amount of memory presently allocated on the system. The committed memory is a sum of all of the memory which has been allocated by processes, even if it has not been "used" by them as of yet.
  • VmallocTotal: total size of vmalloc memory area
  • VmallocUsed: amount of vmalloc area which is used
  • VmallocChunk: largest contiguous block of vmalloc area which is free
  • HugePages_Total: Number of hugepages being allocated by the kernel (Defined with vm.nr_hugepages)
  • HugePages_Free: The number of hugepages not being allocated by a process
  • HugePages_Rsvd: The number of hugepages for which a commitment to allocate from the pool has been made, but no allocation has yet been made.
  • Hugepagesize: The size of a hugepage (usually 2MB on an Intel based system)
RHEL 6 and RHEL 7 only
  • Shmem: Total used shared memory (shared between several processes, thus including RAM disks, SYS-V-IPC and BSD like SHMEM)
  • SReclaimable: The part of the Slab that might be reclaimed (such as caches)
  • SUnreclaim: The part of the Slab that can't be reclaimed under memory pressure
  • KernelStack: The memory the kernel stack uses. This is not reclaimable.
  • WritebackTmp: Memory used by FUSE for temporary writeback buffers
  • HardwareCorrupted: The amount of RAM the kernel identified as corrupted / not working
  • AnonHugePages: Non-file backed huge pages mapped into userspace page tables
  • HugePages_Surp: The number of hugepages in the pool above the value in vm.nr_hugepages. The maximum number of surplus hugepages is controlled by vm.nr_overcommit_hugepages.
  • DirectMap4k: The amount of memory being mapped to standard 4k pages
  • DirectMap2M: The amount of memory being mapped to hugepages (usually 2MB in size)
  • DirectMap1G. The amount of memory being mapped to hugepages (usually 1GB in size)
    Note for DirectMap1G: This value will only be available in /proc/meminfo provided the respective CPU hardware supports it. You can verify the same by checking for flag 'pdpe1gb' in /proc/cpuinfo.

This solution is part of Red Hat’s fast-track publication program, providing a huge library of solutions that Red Hat engineers have created while supporting our customers. To give you the knowledge you need the instant it becomes available, these articles may be presented in a raw and unedited form.

2 Comments

DirectMap1G is missing. RHEL 6.9 root#uname -a Linux server 2.6.32-754.2.1.el6.x86_64 #1 SMP Tue Jul 3 16:37:52 EDT 2018 x86_64 x86_64 x86_64 GNU/Linux root#cat /proc/meminfo MemTotal: 131816212 kB MemFree: 129272952 kB Buffers: 128064 kB Cached: 527428 kB SwapCached: 0 kB Active: 182172 kB Inactive: 498948 kB Active(anon): 25892 kB Inactive(anon): 40 kB Active(file): 156280 kB Inactive(file): 498908 kB Unevictable: 0 kB Mlocked: 0 kB SwapTotal: 33550332 kB SwapFree: 33550332 kB Dirty: 4 kB Writeback: 0 kB AnonPages: 25716 kB Mapped: 15836 kB Shmem: 216 kB Slab: 764468 kB SReclaimable: 97968 kB SUnreclaim: 666500 kB KernelStack: 28640 kB PageTables: 3760 kB NFS_Unstable: 0 kB Bounce: 0 kB WritebackTmp: 0 kB CommitLimit: 99458436 kB Committed_AS: 131480 kB VmallocTotal: 34359738367 kB VmallocUsed: 595788 kB VmallocChunk: 34291662392 kB HardwareCorrupted: 0 kB AnonHugePages: 0 kB HugePages_Total: 0 HugePages_Free: 0 HugePages_Rsvd: 0 HugePages_Surp: 0 Hugepagesize: 2048 kB DirectMap4k: 6140 kB DirectMap2M: 2869248 kB DirectMap1G: 131072000 kB

Hello Tom,

The DirectMap1G will only be seen on the systems whose CPU hardware is enabled with 'pdpe1gb' flag. You can confirm the same in the /proc/cpuinfo