Uncore events show "event syntax error" in ‘perf stat’ output on several recent Intel Xeon Scalable and Xeon W processors

Solution Verified - Updated -

Environment

  • Red Hat Enterprise Linux 9.3 through 9.6.
  • Red Hat Enterprise Linux 10.0
  • Intel 4th and 5th Generation Intel® Xeon Scalable processors (Products formerly Sapphire Rapids & Emerald Rapids)
  • Intel® Xeon® W processors (Products formerly Sapphire Rapids)
  • 3th Generation Intel Xeon Scalable processors (Products formerly Ice Lake)

Issue

  • Uncore events produce an "event syntax error" in the output of the perf stat command on Intel Xeon Sapphire Rapids (SPR) and Emerald Rapids (EMR) processors.
  • Uncore events in Red Hat Enterprise Linux (RHEL) measure the functions of a processor that are outside of a process core but within the CPU package.
  • This issue is seen when running the profiler_hardware_uncore test (part of the redhat-certification-hardware certification suite) on any system with CPU(s) from these processor families.

Examples:

event syntax error: 'unc_cha_tor_inserts.ia_hit_cxl_acc_local'
                                \___ Bad event name
Unable to find event on a PMU of 'unc_cha_tor_inserts.ia_hit_cxl_acc_local'
Initial error:
event syntax error: 'unc_cha_tor_inserts.ia_hit_cxl_acc_local'
                                \___ unknown term 'ch_mask' for pmu 'uncore_cha_0'
event syntax error: '..a_tor_inserts.ia_hit_cxl_acc'
                                  \___ Bad event nameUnable to find event on a PMU of 'unc_cha_tor_inserts.ia_hit_cxl_acc'
Initial error:
event syntax error: '..a_tor_inserts.ia_hit_cxl_acc'
                                  \___ value too big for format, maximum is 4294967295
event syntax error: '..0'
                                  \___ Bad event name

Unable to find event on a PMU of 'unc_iio_comp_buf_inserts.cmpd.part0'
Initial error:
event syntax error: '..0'
                                  \___ value too big for format, maximum is 255

Resolution

  • This issue is caused by the introduction of additional uncore events in the perf package in RHEL 9.3, 9.4, and 9.6 as well as RHEL 10.0. These new events have incorrect definitions that are incompatible with the current kernel’s ability to process them. Specifically, the umask for some uncore events exceeds the 8-bit value limit supported by the kernel, causing parsing errors.
  • For the profiler_hardware_uncore test, the hardware test suite will be changed to test a smaller, more meaningful subset of uncore events, ensuring that enough valid and non-zero events are identified without the need to test all events. This will improve the efficiency of the testing process.
  • These errors in the profiler_hardware_uncore test log are not considered a blocker for hardware certification.

This solution is part of Red Hat’s fast-track publication program, providing a huge library of solutions that Red Hat engineers have created while supporting our customers. To give you the knowledge you need the instant it becomes available, these articles may be presented in a raw and unedited form.

Comments