RHEL8: trace-cmd record -e all (or -e sched) -M <HEX_MASK> does not work on machine with >128 CPUs machine
Issue
On machines with a lot of CPUs, usually more than 128..192 CPUs, trace-cmd record
cannot record events at all for specific CPU(s) and fails, for instance, like this:
[root@machine ~]# trace-cmd record -e sched:sched_switch -M 80000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000
none
trace-cmd: Invalid argument
Failed filter of /sys/kernel/tracing/events/sched/sched_switch/filter
or:
[root@machine ~]# trace-cmd record -e sched -M 80000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000
### global filter ###
# Use this to set filters for multiple events.
# Only events with the given fields will be affected.
# If no events are modified, an error message will be displayed here
trace-cmd: Invalid argument
Failed filter of /sys/kernel/tracing/events/sched/filter
or:
user@machine:~$ sudo trace-cmd record -e all -M 80000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000
(common_pid!=71477)&&(common_pid!=71476)&&(common_pid!=71475)&&(common_pid!=71474)&&(common_pid!=71473)&&(common_pid!=71472)&&(common_pid!=71471)&&(common_pid!=71470)&&(common_pid!=71469)&&(common_pid!=71468)&&(common_pid!=71467)&&(common_pid!=71466)&&(common_pid!=71465)&&(common_pid!=71464)&&(common_pid!=71463)&&(common_pid!=71462)&&(common_pid!=71461)&&(common_pid!=71460)&&(common_pid!=71459)&&(common_pid!=71458)&&(common_pid!=71457)&&(common_pid!=71456)&&(common_pid!=71455)&&(common_pid!=71454)&&(common_pid!=71453)&&(common_pid!=71452)&&(common_pid!=71451)&&(common_pid!=71450)&&(common_pid!=71449)&&(common_pid!=71448)&&(common_pid!=71447)&&(common_pid!=71446)&&(common_pid!=71445)&&(common_pid!=71444)&&(common_pid!=71443)&&(common_pid!=71442)&&(common_pid!=71441)&&(common_pid!=71440)&&(common_pid!=71439)&&(common_pid!=71438)&&(common_pid!=71437)&&(common_pid!=71436)&&(common_pid!=71435)&&(common_pid!=71434)&&(common_pid!=71433)&&(common_pid!=71432)&&(common_pid!=71431)&&(common_pid!=71430)&&(common_pid!=71429)&&(common_pid!=71428)&&(common_pid!=71427)&&(common_pid!=71426)&&(common_pid!=71425)&&(common_pid!=71424)&&(common_pid!=71423)&&(common_pid!=71422)&&(common_pid!=71421)&&(common_pid!=71420)&&(common_pid!=71419)&&(common_pid!=71418)&&(common_pid!=71417)&&(common_pid!=71416)&&(common_pid!=71415)&&(common_pid!=71414)&&(common_pid!=71413)&&(common_pid!=71412)&&(common_pid!=71411)&&(common_pid!=71410)&&(common_pid!=71409)&&(common_pid!=71408)&&(common_pid!=71407)&&(common_pid!=71406)&&(common_pid!=71405)&&(common_pid!=71404)&&(common_pid!=71403)&&(common_pid!=71402)&&(common_pid!=71401)&&(common_pid!=71400)&&(common_pid!=71399)&&(common_pid!=71398)&&(common_pid!=71397)&&(common_pid!=71396)&&(common_pid!=71395)&&(common_pid!=71394)&&(common_pid!=71393)&&(common_pid!=71392)&&(common_pid!=71391)&&(common_pid!=71390)&&(common_pid!=71389)&&(common_pid!=71388)&&(common_pid!=71387)&&(common_pid!=71386)&&(common_pid!=71385)&&(common_pid!=71384)&&(common_pid!=71383)&&(common_pid!=71382)&&(common_pid!=71381)&&(common_pid!=71380)&&(common_pid!=71379)&&(common_pid!=71378)&&(common_pid!=71377)&&(common_pid!=71376)&&(common_pid!=71375)&&(common_pid!=71374)&&(common_pid!=71373)&&(common_pid!=71372)&&(common_pid!=71371)&&(common_pid!=71370)&&(common_pid!=71369)&&(common_pid!=71368)&&(common_pid!=71367)&&(common_pid!=71366)
trace-cmd: Invalid argument
Failed filter of /sys/kernel/tracing/events/sched/sched_switch/filter
Environment
- Red Hat Enterprise Linux 8
- trace-cmd package version older than 2.7-9
- machine with more than 128 CPUs
Subscriber exclusive content
A Red Hat subscription provides unlimited access to our knowledgebase, tools, and much more.