System with the EDB Postgres database crashed due to an out-of-memory (OOM) condition

Issue

• The system with the EDB Postgres database crashed due to an out-of-memory (OOM) condition.

[58057.886538] Kernel panic - not syncing: Out of memory and no killable processes...               
[58057.886540] CPU: 3 PID: 16794 Comm: kworker/u41:2 Kdump: loaded Tainted: GOE  ------------ T 3.10.0-957.el7.x86_64 #1
[58057.886541] Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 12/12/2018
[58057.886542] Workqueue: events_unbound __call_usermodehelper
[58057.886543] Call Trace:
[58057.886545]  [<ffffffff83b61dc1>] dump_stack+0x19/0x1b
[58057.886547]  [<ffffffff83b5b4d0>] panic+0xe8/0x21f
[58057.886548]  [<ffffffff835baaea>] out_of_memory+0x4ea/0x4f0
[58057.886550]  [<ffffffff83b5d2ee>] __alloc_pages_slowpath+0x5d6/0x724
[58057.886552]  [<ffffffff835c0e95>] __alloc_pages_nodemask+0x405/0x420
[58057.886553]  [<ffffffff834b67a0>] ? ____call_usermodehelper+0x140/0x140
[58057.886555]  [<ffffffff83494cdd>] copy_process+0x1dd/0x1a40
[58057.886556]  [<ffffffff834966f1>] do_fork+0x91/0x320
[58057.886557]  [<ffffffff834969a6>] kernel_thread+0x26/0x30
[58057.886558]  [<ffffffff834b5e1e>] __call_usermodehelper+0x2e/0x90
[58057.886560]  [<ffffffff834b9d4f>] process_one_work+0x17f/0x440
[58057.886562]  [<ffffffff834bade6>] worker_thread+0x126/0x3c0
[58057.886563]  [<ffffffff834bacc0>] ? manage_workers.isra.25+0x2a0/0x2a0
[58057.886564]  [<ffffffff834c1c31>] kthread+0xd1/0xe0
[58057.886566]  [<ffffffff834c1b60>] ? insert_kthread_work+0x40/0x40
[58057.886567]  [<ffffffff83b74c1d>] ret_from_fork_nospec_begin+0x7/0x21
[58057.886568]  [<ffffffff834c1b60>] ? insert_kthread_work+0x40/0x40

• Unable to account for the shared memory usage and the process using the shared memory region from VMcore.

The crash built-in "kmem -i" command shows that the total memory used by the shared memory region is 81.6 GB.

crash> kmem -i
                   PAGES        TOTAL      PERCENTAGE
    TOTAL MEM  132049813     503.7 GB         ----
         FREE     290445       1.1 GB    0% of TOTAL MEM
         USED  131759368     502.6 GB   99% of TOTAL MEM
       SHARED   21392742      81.6 GB   16% of TOTAL MEM
      BUFFERS          0            0    0% of TOTAL MEM
       CACHED   21429345      81.7 GB   16% of TOTAL MEM
         SLAB     868877       3.3 GB    0% of TOTAL MEM

   TOTAL HUGE          0            0         ----
    HUGE FREE          0            0    0% of TOTAL HUGE

   TOTAL SWAP   13107199        50 GB         ----
    SWAP USED   13107199        50 GB  100% of TOTAL SWAP
    SWAP FREE          0            0    0% of TOTAL SWAP

 COMMIT LIMIT   79132105     301.9 GB         ----
    COMMITTED  143014517     545.6 GB  180% of TOTAL LIMIT

crash> kmem -V | grep NR_SHMEM:
                     NR_SHMEM: 21428076
crash> !bc -q
scale=2
21428076*4/2^20
81.74 (GiB)

The crash built-in "ipcs -m" commands show that the total memory allocated to the shared memory region is 56 bytes.

crash> ipcs -m
SHMID_KERNEL     KEY      SHMID      UID   PERMS BYTES      NATTCH STATUS
ffff9e12fb73a640 005311a1 196610     988   600   56         22   

crash> ipcs -M
SHMID_KERNEL     KEY      SHMID      UID   PERMS BYTES      NATTCH STATUS
ffff9e12fb73a640 005311a1 196610     988   600   56         22      
PAGES ALLOCATED/RESIDENT/SWAPPED: 1/1/0
INODE: ffff9e12e4b51598