System shows high load in KVM when isolcpus in use with most CPUs

Solution In Progress - Updated -

Issue

  • The CPUs not in isolcpus list had huge runqueues and the majority of tasks were handling function tdp_mmu_zap_root_work.
PID: 1165     TASK: ffff888121a6c700  CPU: 0    COMMAND: "kworker/u193:5"
 #0 [fffffe000000bd10] machine_kexec at ffffffff8106c237
 #1 [fffffe000000bd68] __crash_kexec at ffffffff811c3c9a
 #2 [fffffe000000be28] panic at ffffffff81ad7dc6
 #3 [fffffe000000beb0] unknown_nmi_error.cold at ffffffff81ad1453
 #4 [fffffe000000bec8] exc_nmi at ffffffff81b26c2d
 #5 [fffffe000000bef0] end_repeat_nmi at ffffffff81c0156a
    [exception RIP: native_queued_spin_lock_slowpath+0x42]
    RIP: ffffffff81165192  RSP: ffffc9000f4b3da0  RFLAGS: 00000202
    RAX: 0000000000c40101  RBX: ffffc9002cd21000  RCX: 0000000000000000
    RDX: ffffc9002cd21004  RSI: 0000000000000000  RDI: ffffc9002cd21004
    RBP: ffffc9002cd21004   R8: 000000000000001b   R9: 0000000008000000
    R10: 0000000000000000  R11: 0000000000000000  R12: 0000000000000000
    R13: 0000000000000001  R14: 0000000000000004  R15: ffff888121a6c700
    ORIG_RAX: ffffffffffffffff  CS: 0010  SS: 0018
--- <NMI exception stack> ---
 #6 [ffffc9000f4b3da0] native_queued_spin_lock_slowpath at ffffffff81165192
 #7 [ffffc9000f4b3da0] queued_read_lock_slowpath at ffffffff81165744
 #8 [ffffc9000f4b3db8] __cond_resched_rwlock_read at ffffffff8112c74f
 #9 [ffffc9000f4b3dc8] __tdp_mmu_zap_root at ffffffffc1213221 [kvm]
#10 [ffffc9000f4b3e70] tdp_mmu_zap_root_work at ffffffffc1213fa4 [kvm]
#11 [ffffc9000f4b3e90] process_one_work at ffffffff81118365
#12 [ffffc9000f4b3ed8] worker_thread at ffffffff81118930
#13 [ffffc9000f4b3f18] kthread at ffffffff8111f956
#14 [ffffc9000f4b3f50] ret_from_fork at ffffffff81002d9f

Environment

  • Red Hat Enterprise Linux 9.2
  • Openstack compute node 17.1

Subscriber exclusive content

A Red Hat subscription provides unlimited access to our knowledgebase, tools, and much more.

Current Customers and Partners

Log in for full access

Log In

New to Red Hat?

Learn more about Red Hat subscriptions

Using a Red Hat product through a public cloud?

How to access this content