[RHEL 8] For isolated CPUs(with nohz_full enabled for isolated CPUs) CPU utilization statistics are not getting reflected continuously

Solution Verified - Updated -

Issue

  • For isolated CPUs(with nohz_full enabled for isolated CPUs) CPU utilization statistics (/proc/stat and/or sysstat) are not getting reflected continuously.

  • Example of kernel command line:

    isolcpus=1-3,5,7-15 nohz=on nohz_full=1-3,5,7-15

  • When process that fully consumes CPU is moved to isolated CPU 15, utilization(%usr) gets reflected intermittently on isolated CPU 15(there are time gaps):

    04:10:16 PM     CPU      %usr     %nice      %sys   %iowait    %steal      %irq     %soft    %guest    %gnice     %idle
04:10:17 PM      15      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00    100.00
04:10:18 PM      15      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00    100.00
04:10:19 PM      15      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00    100.00
04:10:20 PM      15      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00    100.00
04:10:21 PM      15      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00    100.00
04:10:22 PM      15      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00    100.00
04:10:23 PM      15      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00    100.00
04:10:24 PM      15      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00    100.00
04:10:25 PM      15     69.33      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00     30.67
04:10:26 PM      15    100.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
04:10:27 PM      15      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00    100.00
04:10:28 PM      15      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00    100.00
04:10:29 PM      15    100.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
04:10:30 PM      15      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00    100.00
04:10:31 PM      15      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00    100.00
04:10:32 PM      15      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00    100.00
04:10:33 PM      15      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00    100.00
04:10:34 PM      15      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00    100.00
04:10:35 PM      15      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00    100.00
04:10:36 PM      15      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00    100.00
04:10:37 PM      15    100.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
04:10:38 PM      15      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00    100.00
04:10:39 PM      15      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00    100.00
04:10:40 PM      15      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00    100.00
04:10:41 PM      15      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00    100.00
04:10:42 PM      15      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00    100.00
04:10:43 PM      15      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00    100.00
04:10:44 PM      15      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00    100.00
04:10:45 PM      15    100.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00

Environment

  • RHEL 8.3 and prior versions of RHEL 8. Issue is observed on z stream kernel of RHEL 8.4.
  • Kernels older than kernel-rt-4.18.0-240.15.1.rt7.69.el8_3.x86_64 on RHEL 8.
  • Kernels older than kernel-4.18.0-240.15.1.el8_3.x86_64 on RHEL 8.
  • Z stream kernels of RHEL 8.4: kernel-4.18.0-305.19.1.el8_4.x86_64 and kernel-4.18.0-305.19.1.rt7.91.el8_4.x86_64.
  • isolcpus and nohz_full parameter enabled for isolated CPUs.
  • Note: This issue is observed on regular kernel as well as kernel-rt.
  • This issue is not observed on RHEL 7 or RHEL 6.

Subscriber exclusive content

A Red Hat subscription provides unlimited access to our knowledgebase, tools, and much more.

Current Customers and Partners

Log in for full access

Log In

New to Red Hat?

Learn more about Red Hat subscriptions

Using a Red Hat product through a public cloud?

How to access this content