RT task starved by repeated RT push and stopper retries on RHEL9-based kernel-rt, even with the CVE-2026-45919 fix

Solution Verified - Updated -

Issue

  • On RHEL9 kernel-rt, a latency-sensitive RT application can miss its deadlines, hit its own watchdog timeout and crash, because one of its critical RT tasks is starved of CPU time.
  • An earlier form of this problem was fixed upstream by commit 94894c9c477e (CVE-2026-45919), and our JIRAs, RHEL-240634 (9.8.z) and RHEL-244409 (9.6.z), addressed the backport of that fix to RHEL9.
  • However, a slightly different issue remains after applying 94894c9c477e. A migration-disabled RT task cannot be pushed to another CPU, but the scheduler keeps retrying the push as long as that task stays at the head of the pushable list. Each retry wakes the stopper thread, which takes the CPU away from the RT task running there, and the next retry follows right after.
  • The sequence looks like this in ftrace, with kprobes on the RT balancer (pnpt is the return of pick_next_pushable_task(), socn is stop_one_cpu_nowait(), pcs is push_cpu_stop(), flr_ent and flr_ret are the entry and return of find_lock_lowest_rq(), pullrt is pull_rt_task(), and md is the task's migration_disabled count).
         threadB-71804   [022] d..h2.. 49483.514458: pnpt: (push_rt_task.part.0+0x1e/0x3f0 <- pick_next_pushable_task) task=0xff1b7f9680228000 pid=2001078 comm="threadA" md=2 prio=49
         threadB-71804   [022] d..h2.. 49483.514461: socn: (stop_one_cpu_nowait+0x0/0x40) tgtcpu=22 pid=71804 comm="threadB" prio=49 md=0
           <...>-246     [022] d..h3.. 49483.514473: pnpt: (push_rt_task.part.0+0x1e/0x3f0 <- pick_next_pushable_task) task=0xff1b7f9680228000 pid=2001078 comm="threadA" md=2 prio=49
           <...>-246     [022] ....2.. 49483.514480: pcs: (push_cpu_stop+0x0/0x210) task=0xff1b7f99e5f44680 pid=71804 comm="threadB" prio=49 md=0 oncpu=0 onrq=1 ncpus=30
           <...>-246     [022] d...4.. 49483.514481: flr_ent: (find_lock_lowest_rq+0x0/0x150) pid=71804 comm="threadB" prio=49 md=0 oncpu=0 onrq=1 tcpu=22 ncpus=30 srccpu=22
           <...>-246     [022] d...5.. 49483.514489: pnpt: (find_lock_lowest_rq+0x98/0x150 <- pick_next_pushable_task) task=0xff1b7f9680228000 pid=2001078 comm="threadA" md=2 prio=49
           <...>-246     [022] d...4.. 49483.514489: flr_ret: (push_cpu_stop+0x118/0x210 <- find_lock_lowest_rq) ret=0x0 dest=33620255
           <...>-246     [022] d..h4.. 49483.514490: pnpt: (push_rt_task.part.0+0x1e/0x3f0 <- pick_next_pushable_task) task=0xff1b7f9680228000 pid=2001078 comm="threadA" md=2 prio=49
           <...>-246     [022] d...3.. 49483.514493: pullrt: (pull_rt_task+0x0/0x450) rqcpu=22
  • The head of the pushable list is threadA with md=2 (inside 2 nested migrate_disable() regions), so it cannot be pushed. push_rt_task() falls back to pushing the running task (rq->curr), threadB, and wakes the stopper (socn).
  • The stopper, pid 246, runs push_cpu_stop() for threadB, but after taking the locks the check picks the head again and gets threadA, not threadB, so the push is refused (flr_ret ret=0x0).
  • On the way back pull_rt_task() runs and the next retry starts. In this capture covering 89 ms, push_cpu_stop() ran 5,204 times and every one of those pushes was refused.
  • As a result, the RT task running on that CPU gets very little CPU time, and the application watchdog eventually fires.
  • The 94894c9c477e fix does not cover this remaining case.

Environment

  • Red Hat OpenShift Container Platform 4.20.z, whose RHCOS uses kernel-rt based on RHEL9.6.z, versions older than kernel-rt-5.14.0-570.146.1.el9_6
  • Red Hat Enterprise Linux 9 with kernel-rt (PREEMPT_RT).

Subscriber exclusive content

A Red Hat subscription provides unlimited access to our knowledgebase, tools, and much more.

Current Customers and Partners

Log in for full access

Log In

New to Red Hat?

Learn more about Red Hat subscriptions

Using a Red Hat product through a public cloud?

How to access this content