Default pagecache reclaim policy on RHEL 5 NUMA systems can cause HPC performance degradation
Issue
High Performance Computing (HPC) applications running on Non-Uniform Memory Access (NUMA) hardware with Red Hat Enterprise Linux 5 (RHEL 5) can incur severe performance degradation when application memory allocation is forced off-node due to the local-node memory being full of clean unmapped pagecache pages. Applications with a large memory footprint may run an order of magnitude slower due to the increased non-local memory access latency and/or reduction in bandwidth across the NUMA fabric.
Environment
- Red Hat Enterprise Linux 5 (GA or later)
- NUMA capable hardware
- A latency and/or bandwith sensitive application, e.g. the STREAMS benchmark.
Subscriber exclusive content
A Red Hat subscription provides unlimited access to our knowledgebase, tools, and much more.