Why 'podman ps' or 'podman images' command fail frequently with the error "fatal error: runtime: out of memory" on Red Hat Enterprise Linux 8 after starting Quay containers ?
Issue
- Why
podman ps
orpodman images
command fail frequently with the error "fatal error: runtime: out of memory" on Red Hat Enterprise Linux 8 after starting Quay containers ?
Scenario 1: Failure with podman images
command(just a short snip)
# podman images
WARN[0000] Error validating CNI config file /etc/cni/net.d/87-podman-bridge.conflist: [netplugin failed: "fatal error: runtime: out of memory\n\nruntime stack:\nruntime.throw(0x558fc8f70bc5, 0x16)\n\t/usr/lib/golang/src/runtime/panic.go:1116 +0x74 fp=0x7ffe99053c10 sp=0x7ffe99053be0 pc=0x558fc8e0ac34\nruntime.sysMap(0xc000000000, 0x4000000, 0x558fc931e578)\n\t/usr/lib/golang/src/runtime/mem_linux.go:169 +0xc7 fp=0x7ffe99053c50 sp=0x7ffe99053c10 pc=0x558fc8dee627\nruntime.(*mheap).sysAlloc(0x558fc9303b80, 0x400000, 0x0, 0x4)\n\t/usr/lib/golang/src/runtime/malloc.go:727 +0x1d4 fp=0x7ffe99053cf8 sp=0x7ffe99053c50 pc=0x558fc8de2954\nruntime.(*mheap).grow(0x558fc9303b80, 0x1, 0x0)\n\t/usr/lib/golang/src/runtime/mheap.go:1344 +0x85 fp=0x7ffe99053d60 sp=0x7ffe99053cf8 pc=0x558fc8dfdae5\nruntime.(*mheap).allocSpan(0x558fc9303b80, 0x1, 0x7571636f70002a00, 0x558fc931e588, 0x72682e7a62)\n
Scenario 2: Failure with podman -v
command
# podman -v
runtime/cgo: pthread_create failed: Resource temporarily unavailable
SIGABRT: abort
PC=0x7f2db18f437f m=4 sigcode=18446744073709551610
goroutine 0 [idle]:
runtime: unknown pc 0x7f2db18f437f
stack: frame={sp:0x7f2d89da87e0, fp:0x0} stack=[0x7f2d895a9288,0x7f2d89da8e88)
Environment
- Red Hat Enterprise Linux 8
- Quay and postgresql database installed on the same node for Proof of Concept
- Half of the total memory allocated for hugepages
- vm.overcommit_memory = 2
- Quay enabled with LDAP and clair
Subscriber exclusive content
A Red Hat subscription provides unlimited access to our knowledgebase, tools, and much more.