The kernel is getting unresponsive/hung where lots of tasks are performing memory compaction
Issue
-
The kernel is getting unresponsive/hung where lots of tasks are performing memory compaction.
-
Lots of kworkers are waiting for kthreadd to create workers and getting blocked in TASK_UNINTERRUPTIBLE sleep state for quite a long time with backtraces like this:
PID: 23151 TASK: ffff9a2d3db12000 CPU: 16 COMMAND: "kworker/16:3"
#0 [ffff9a5813fc7b68] __schedule at ffffffffb3581dea
#1 [ffff9a5813fc7bf8] schedule at ffffffffb3582299
#2 [ffff9a5813fc7c08] schedule_timeout at ffffffffb357fda1
#3 [ffff9a5813fc7cb8] wait_for_completion at ffffffffb358264d
#4 [ffff9a5813fc7d18] kthread_create_on_node at ffffffffb2ec61fa
#5 [ffff9a5813fc7dd0] create_worker at ffffffffb2ebef8b
#6 [ffff9a5813fc7e20] manage_workers at ffffffffb2ebf216
#7 [ffff9a5813fc7e68] worker_thread at ffffffffb2ebf743
#8 [ffff9a5813fc7ec8] kthread at ffffffffb2ec63a1
- At that time kthreadd is trying to create a worker and thus performing direct memory compaction with a backtrace like this:
PID: 2 TASK: ffff9a4cd3981070 CPU: 14 COMMAND: "kthreadd"
#0 [ffff9a2d537d3878] __schedule at ffffffffb3581dea
#1 [ffff9a2d537d3908] __cond_resched at ffffffffb2ed7c36
#2 [ffff9a2d537d3920] _cond_resched at ffffffffb358253a
#3 [ffff9a2d537d3930] compact_checklock_irqsave at ffffffffb2fe45ad
#4 [ffff9a2d537d3960] isolate_migratepages_range at ffffffffb2fe545a
#5 [ffff9a2d537d3a20] compact_zone at ffffffffb2fe5b03
#6 [ffff9a2d537d3a78] compact_zone_order at ffffffffb2fe5d2d
#7 [ffff9a2d537d3b10] try_to_compact_pages at ffffffffb2fe60d1
#8 [ffff9a2d537d3b70] __alloc_pages_direct_compact at ffffffffb357728e
#9 [ffff9a2d537d3bd0] __alloc_pages_slowpath at ffffffffb357775e
#10 [ffff9a2d537d3cc0] __alloc_pages_nodemask at ffffffffb2fc8796
#11 [ffff9a2d537d3d68] copy_process at ffffffffb2e9914d
#12 [ffff9a2d537d3df8] do_fork at ffffffffb2e9ab71
#13 [ffff9a2d537d3e70] kernel_thread at ffffffffb2e9ae36
#14 [ffff9a2d537d3e80] kthreadd at ffffffffb2ec6d81
Environment
- Red Hat Enterprise Linux 7.7 (kernel-3.10.0-1062.37.1.el7)
- Transparent Hugepages disabled
- Hugepages enabled
Subscriber exclusive content
A Red Hat subscription provides unlimited access to our knowledgebase, tools, and much more.