How do I set affinty of IRQ for 4 Solarflare interfaces on RHEL 8

Posted on

Using Ansible playbook or script

I am having trouble to set the affinty of 4 solarflare cards.

ens1-0 and ens1-1 to Node 0 - highest cpu core
ens5-0 and ens5-1 to Node 1 - highest cpu core

The issue is it keep defaulting to the house keeping core, which is udnerstandable because I have /proc/cmdline set to isolate the cpu, which is a must to stop processing from using these cores.

Redhat 8 Enterprise Linux
irqbalance: disabled
Housekeeping CPU : 0 / 16
Osolated CPU
isolcpus=1-15,17-63 nohz=on nohz_full=1-15,17-63 rcu_nocbs=1-15,17-63

I read somehere

The binding is undone by the irqbalance oneshot deamon and the rcu rebind.

cat /proc/interrupts | egrep "CPU|ens1f1np1-0|ens1f0np0-0|ens5f1np1-0|ens5f0np0-0"
CPU0 CPU1 CPU2 CPU3 CPU4 CPU5 CPU6 CPU7 CPU8 CPU9 CPU10 CPU11 CPU12 CPU13 CPU14 CPU15 CPU16 CPU17 CPU18 CPU19 CPU20 CPU21 CPU22 CPU23 CPU24 CPU25 CPU26 CPU27 CPU28 CPU29 CPU30 CPU31 CPU32 CPU33 CPU34 CPU35 CPU36 CPU37 CPU38 CPU39 CPU40 CPU41 CPU42 CPU43 CPU44 CPU45 CPU46 CPU47 CPU48 CPU49 CPU50 CPU51 CPU52 CPU53 CPU54 CPU55 CPU56 CPU57 CPU58 CPU59 CPU60 CPU61 CPU62 CPU63
134: 681706 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 PCI-MSI 9961472-edge ens1f0np0-0
152: 1063860 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 PCI-MSI 9963520-edge ens1f1np1-0
170: 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 50506 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 PCI-MSI 72351744-edge ens5f0np0-0
188: 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 911 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 PCI-MSI 72353792-edge ens5f1np1-0

I have tried to use numactl to write to the file but that also keeps defaulting back e.g numactl -a -N netdev:ens5f1np1 -C 6 grep allowed /proc/self/status

[@]# numactl -a -N netdev:ens5f0np0 grep allowed /proc/self/status
Cpus_allowed: ffff0000,ffff0000
Cpus_allowed_list: 16-31,48-63
Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003
Mems_allowed_list: 0-1
[@]# numactl -a -N netdev:ens5f1np1 grep allowed /proc/self/status
Cpus_allowed: ffff0000,ffff0000
Cpus_allowed_list: 16-31,48-63
Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003
Mems_allowed_list: 0-1
[@]# numactl -a -N netdev:ens1f0np0 grep allowed /proc/self/status
Cpus_allowed: 0000ffff,0000ffff
Cpus_allowed_list: 0-15,32-47
Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003
Mems_allowed_list: 0-1
[@]# numactl -a -N netdev:ens1f1np1 grep allowed /proc/self/status
Cpus_allowed: 0000ffff,0000ffff
Cpus_allowed_list: 0-15,32-47
Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003
Mems_allowed_list: 0-1

SA

Responses