How to trace or debug slab leaks
Environment
- Red Hat Enterprise Linux
Issue
- How do I trace allocation of slabs?
- Slab memory usage is quite high. How do I determine who is generating the slab allocations?
Resolution
Quite a few options exist to trace slab allocations.
Systemtap
Systemtap (also known as stap) provides a method for writing scripts to perform debugging and tracing activity in the kernel, compile the script into a custom kernel module, and load the module to enable the troubleshooting functionality provided by the custom script.
- For instructions on tracing slab allocations via stap, please refer to the knowledge-base article Is there a way to track slab allocations or leaks with systemtap?
- For more information in general on stap, please refer to the knowledge-base article What is SystemTap and how to use it?
- Stap is available for Red Hat Enterprise Linux 5 and above.
Perf
Perf is a userspace tool which interacts with the ftrace subsystem of the kernel to perform tracing within kernelspace execution.
- For instructions on tracing slab allocations via perf, please refer to the knowledge-base article How to track slab allocations using perf
- For more information in general on perf, please refer to the documentation Chapter 18. Getting started with perf
- Perf is available for Red Hat Enterprise Linux 6 and above.
eBPF/BCC
Extended Berkeley Packet Filter (eBPF) enables tracing and debugging within a sandboxed, in-kernel virtual machine. Tracing and debugging is written out similar to Clang programs and compiled via the BPF Compiler Collection (BCC).
- For instructions on tracing slab allocations with eBPF, please refer to the knowledge-base article How to monitor the kernel SLAB/SLUB memory allocations using eBPF/BCC script?
- For more information in general on eBPF, please refer to the blog post Introduction to eBPF in Red Hat Enterprise Linux 7
- eBPF/BCC is available for Red Hat Enterprise Linux 7 as Tech Preview and Red Hat Enterprise Linux 8.
Tooling within the kernel
The linux kernel offers the slub_debug kernel parameter (as well as sysfs files) to enable debugging options for slab.
- For instructions on tracing slab allocations via
slub_debugor sysfs files and a vmcore, please refer to the knowledge-base article How to use slub_debug to troubleshoot and debug issues with slab - For more information on debugging slab via the sysfs or
slub_debugparameter, please refer to the upstream documentation on slub. - These options are available for Red Hat Enterprise Linux 6 and above.
This solution is part of Red Hat’s fast-track publication program, providing a huge library of solutions that Red Hat engineers have created while supporting our customers. To give you the knowledge you need the instant it becomes available, these articles may be presented in a raw and unedited form.
Comments