RHEL7: While operating on a Veritas filesystem, kmalloc-1024 slab caches take all resources
Issue
- When the customer does rman backups (on veritas filesystem) and some sql queries (in databases ontop of veritas filesystem), The "kmalloc-1024" buffer (part of "cat /proc/slabinfo" output) grows until the system has no memory left. So under usage the buffer constantly grows, and after some weeks memory is used up.
- The issue is seen on multiple systems.
- The kmalloc-1024 cache only seems to grow with pread and pwrite.
- Memory is not shrinking when veritas filesystems are umounted.
- Unloading the veritas modules is not possible
- Dropping caches using /proc/sys/vm/drop_caches does not influence that memory.
- "-/+ buffers/cache" in the "free" output is decreasing when the filesystem is used and kmalloc-1024 is growing.
- Simpler tools like dd, cp, tar etc. were not able to reproduce the issue.
- Normal tools like cp, tar etc. use read/write. Tools like rman use "pread" and "pwrite", these seem to cause the issue
Environment
- Red Hat Enterprise Linux (RHEL) 7.0 and 7.1
- VERITAS Storage Foundation HA 6.2.1
Subscriber exclusive content
A Red Hat subscription provides unlimited access to our knowledgebase of over 48,000 articles and solutions.
Welcome! Check out the Getting Started with Red Hat page for quick tours and guides for common tasks.
