rhel7.7 -rhel7.9: panic with "BUG: unable to handle kernel paging request at"
Issue
- The memcg_params field of kmem_cache struct contains an old slab address that is to small for the current size of memcg_limited_groups_array_size.
- On affected kernels, we are hitting a panic in mm/memcontrol.c.
PANIC: "BUG: unable to handle kernel paging request at 000000010000024e"
crash> bt
PID: 30317 TASK: ffff9281db27c1c0 CPU: 3 COMMAND: "kworker/u12:3"
…
#9 [ffff928193a9fc00] page_fault at ffffffff8dd84778
[exception RIP: memcg_kmem_destroy_cache+0x1b]
RIP: ffffffff8d83988b RSP: ffff928193a9fcb0 RFLAGS: 00010206
RAX: 000000010000024e RBX: ffff92841d7ed800 RCX: ffff928193a9ffd8
RDX: 0000000000000000 RSI: ffff9285bfcdfd18 RDI: ffff92841d7ed800
RBP: ffff928193a9fcc0 R8: fffff420d3e7b560 R9: 0000000000000000
R10: ffff9285bfbd5fe0 R11: ffff9284a5b290c0 R12: ffff92859837b600
R13: 000000000000000f R14: 0000000000000000 R15: 0000000080000003
ORIG_RAX: ffffffffffffffff CS: 0010 SS: 0018
#10 [ffff928193a9fcc8] __kmem_cache_destroy_memcg_children at ffffffff8d83ee05
#11 [ffff928193a9fcf8] kmem_cache_destroy at ffffffff8d7e3aa4
#12 [ffff928193a9fd18] nf_conntrack_cleanup_net_list at ffffffffc0897a3b [nf_conntrack]
#13 [ffff928193a9fd60] nf_conntrack_pernet_exit at ffffffffc089886d [nf_conntrack]
#14 [ffff928193a9fd88] ops_exit_list at ffffffff8dc43779
#15 [ffff928193a9fdb8] cleanup_net at ffffffff8dc44800
#16 [ffff928193a9fe20] process_one_work at ffffffff8d6be21f
…
crash> dis -lr memcg_kmem_destroy_cache+0x1b
…
/usr/src/debug/kernel-3.10.0-1062.9.1.el7/linux-3.10.0-1062.9.1.el7.x86_64/mm/memcontrol.c: 3323
0xffffffff8d83987c <memcg_kmem_destroy_cache+0xc>: mov 0xb8(%rdi),%rax
…
/usr/src/debug/kernel-3.10.0-1062.9.1.el7/linux-3.10.0-1062.9.1.el7.x86_64/mm/slab.h: 129
…
0xffffffff8d83988b <memcg_kmem_destroy_cache+0x1b>: cmpb $0x0,(%rax)
crash> whatis memcg_kmem_destroy_cache
void memcg_kmem_destroy_cache(struct kmem_cache *);
crash> struct -o kmem_cache | grep b8
[0xb8] struct memcg_cache_params *memcg_params;
crash> struct kmem_cache.memcg_params ffff92841d7ed800
memcg_params = 0x10000024e
crash> kmem ffff92841d7ed800
CACHE OBJSIZE ALLOCATED TOTAL SLABS SSIZE NAME
ffff9280ffc01900 128 7925 10272 321 4k kmalloc-128
SLAB MEMORY NODE TOTAL ALLOCATED FREE
fffff420d275fb40 ffff92841d7ed000 0 32 22 10
FREE / [ALLOCATED]
ffff92841d7ed800
…
Environment
- Red Hat Enterprise Linux (RHEL) 7.7, 7.8 and 7.9
- kernel panic
Subscriber exclusive content
A Red Hat subscription provides unlimited access to our knowledgebase, tools, and much more.