System hung due to high load and slab cache memory usage with blocked tasks in synchronize_rcu()

Solution Unverified - Updated -

Issue

  • The system hung due to high load and slab cache memory usage with blocked tasks in synchronize_rcu().
  • The system ran out of memory (OOM) due to high slab cache memory usage.
  • Kernel panic due to Out of memory and no killable processes.
[376019.500321] Out of memory and no killable processes...
[376019.500322] Kernel panic - not syncing: System is deadlocked on memory                
[376019.502490] CPU: 9 PID: 2 Comm: kthreadd Kdump: loaded Tainted: POEL ----------- 4.18.0-305.10.2.el8_4.x86_64 #1
[376019.503477] Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 12/12/2018
[376019.504447] Call Trace:
[376019.505407]  dump_stack+0x5c/0x80
[376019.506344]  panic+0xe7/0x2a9
[376019.507260]  out_of_memory.cold.35+0x5e/0x82
[376019.508223]  __alloc_pages_slowpath+0xc02/0xd20
[376019.509096]  __alloc_pages_nodemask+0x283/0x2c0
[376019.509953]  allocate_slab+0x475/0x4b0
[376019.510789]  ___slab_alloc+0x401/0x5b0
[376019.511600]  ? copy_process+0x225/0x19d0
[376019.512411]  ? copy_process+0x225/0x19d0
[376019.513225]  __slab_alloc+0x1c/0x30
[376019.514020]  kmem_cache_alloc_node+0x1d5/0x230
[376019.514796]  ? __switch_to_asm+0x41/0x70
[376019.515552]  copy_process+0x225/0x19d0
[376019.516293]  ? __switch_to_asm+0x35/0x70
[376019.517047]  ? __switch_to_asm+0x41/0x70
[376019.517737]  ? __switch_to_asm+0x35/0x70
[376019.518405]  ? kthread_flush_work_fn+0x10/0x10
[376019.519062]  ? __switch_to_asm+0x35/0x70
[376019.519736]  _do_fork+0x8f/0x350
[376019.520385]  kernel_thread+0x25/0x30
[376019.521009]  kthreadd+0x2b4/0x300
[376019.521694]  ? kthread_create_on_cpu+0xa0/0xa0
[376019.522288]  ret_from_fork+0x1f/0x40

Environment

  • Red Hat Enterprise Linux 8
  • Proprietary (P) kernel module: [b9k_87125]

Subscriber exclusive content

A Red Hat subscription provides unlimited access to our knowledgebase, tools, and much more.

Current Customers and Partners

Log in for full access

Log In

New to Red Hat?

Learn more about Red Hat subscriptions

Using a Red Hat product through a public cloud?

How to access this content