RHEL: some NIC are not recognised in Multiple NIC when X2APIC is disabled from BIOS
Issue
- Two NICs are not recognised on CPU8 or lower in Multiple NIC with following logs when X2APIC is disabled from BIOS.
[ 0.064947] IRQ remapping doesn't support X2APIC mode, disable x2apic.
[ 0.065052] Switched APIC routing to physical flat.
..............
[ 5.150243] (0000:b0:00.0): E-Switch: Total vports 9, per vport: max uc(1024) max mc(16384)
[ 5.160658] mlx5_core 0000:b0:00.0: irq 210 for MSI/MSI-X
[ 5.160670] mlx5_core 0000:b0:00.0: alloc irq vectors failed
[ 5.160691] (0000:b0:00.0): E-Switch: cleanup
[ 5.864591] mlx5_core 0000:b0:00.0: mlx5_load_one failed with error code -28
[ 5.864716] mlx5_core: probe of 0000:b0:00.0 failed with error -28
[ 5.864945] mlx5_core 0000:b0:00.1: firmware version: 14.22.1414
[ 6.768804] (0000:b0:00.1): E-Switch: Total vports 9, per vport: max uc(1024) max mc(16384)
[ 6.775205] mlx5_core 0000:b0:00.1: alloc irq vectors failed
[ 6.775211] (0000:b0:00.1): E-Switch: cleanup
[ 7.471940] mlx5_core 0000:b0:00.1: mlx5_load_one failed with error code -28
[ 7.472038] mlx5_core: probe of 0000:b0:00.1 failed with error -28
- It should be visible like this:
Interface Status:
eno1 0000:02:00.0 link=DOWN rx ring 200/2047 drv tg3 v3.137 / fw 5719-v1.46 NCSI v1.5.1.0
eno2 0000:02:00.1 link=DOWN rx ring 200/2047 drv tg3 v3.137 / fw 5719-v1.46 NCSI v1.5.1.0
eno3 0000:02:00.2 link=DOWN rx ring 200/2047 drv tg3 v3.137 / fw 5719-v1.46 NCSI v1.5.1.0
eno4 0000:02:00.3 link=DOWN rx ring 200/2047 drv tg3 v3.137 / fw 5719-v1.46 NCSI v1.5.1.0
ens1f0 0000:12:00.0 link=DOWN rx ring 512/4096 drv i40e v2.3.2-k / fw 10.4.3
ens1f1 0000:12:00.1 link=DOWN rx ring 512/4096 drv i40e v2.3.2-k / fw 10.4.3
ens3f0 0000:13:00.0 link=DOWN rx ring 1024/8192 drv mlx5_core v4.7-1.0.0 / fw 14.26.1040 (HP_2420110034)
ens3f1 0000:13:00.1 link=DOWN rx ring 1024/8192 drv mlx5_core v4.7-1.0.0 / fw 14.26.1040 (HP_2420110034)
ens4f0 0000:af:00.0 link=DOWN rx ring 512/4096 drv i40e v2.3.2-k / fw 10.4.3
ens4f1 0000:af:00.1 link=DOWN rx ring 512/4096 drv i40e v2.3.2-k / fw 10.4.3
ens6f0 0000:b0:00.0 link=DOWN rx ring 1024/8192 drv mlx5_core v4.7-1.0.0 / fw 14.26.1040 (HP_2420110034)
ens6f1 0000:b0:00.1 link=DOWN rx ring 1024/8192 drv mlx5_core v4.7-1.0.0 / fw 14.26.1040 (HP_2420110034)
ens8f0 0000:d9:00.0 link=DOWN rx ring 256/4096 drv igb v5.4.0-k / fw 1.52, 0x80000e81, 1.2028.0
ens8f1 0000:d9:00.1 link=DOWN rx ring 256/4096 drv igb v5.4.0-k / fw 1.52, 0x80000e81, 1.2028.0
ens8f2 0000:d9:00.2 link=DOWN rx ring 256/4096 drv igb v5.4.0-k / fw 1.52, 0x80000e81, 1.2028.0
ens8f3 0000:d9:00.3 link=DOWN rx ring 256/4096 drv igb v5.4.0-k / fw 1.52, 0x80000e81, 1.2028.0
- But two NIC are missing.
Interface Status:
eno1 0000:02:00.0 link=DOWN rx ring 200/2047 drv tg3 v3.137 / fw 5719-v1.46 NCSI v1.5.1.0
eno2 0000:02:00.1 link=DOWN rx ring 200/2047 drv tg3 v3.137 / fw 5719-v1.46 NCSI v1.5.1.0
eno3 0000:02:00.2 link=DOWN rx ring 200/2047 drv tg3 v3.137 / fw 5719-v1.46 NCSI v1.5.1.0
eno4 0000:02:00.3 link=DOWN rx ring 200/2047 drv tg3 v3.137 / fw 5719-v1.46 NCSI v1.5.1.0
ens1f0 0000:12:00.0 link=DOWN rx ring 512/4096 drv i40e v2.3.2-k / fw 10.4.3
ens1f1 0000:12:00.1 link=DOWN rx ring 512/4096 drv i40e v2.3.2-k / fw 10.4.3
ens3f0 0000:13:00.0 link=DOWN rx ring 1024/8192 drv mlx5_core v5.0-0 / fw 14.22.1414 (HP_2420110034)
ens3f1 0000:13:00.1 link=DOWN rx ring 1024/8192 drv mlx5_core v5.0-0 / fw 14.22.1414 (HP_2420110034)
ens4f0 0000:af:00.0 link=DOWN rx ring 512/4096 drv i40e v2.3.2-k / fw 10.4.3
ens4f1 0000:af:00.1 link=DOWN rx ring 512/4096 drv i40e v2.3.2-k / fw 10.4.3
ens8f0 0000:d9:00.0 link=DOWN rx ring 256/4096 drv igb v5.4.0-k / fw 1.52, 0x80000e81, 1.2028.0
ens8f1 0000:d9:00.1 link=DOWN rx ring 256/4096 drv igb v5.4.0-k / fw 1.52, 0x80000e81, 1.2028.0
ens8f2 0000:d9:00.2 link=DOWN rx ring 256/4096 drv igb v5.4.0-k / fw 1.52, 0x80000e81, 1.2028.0
ens8f3 0000:d9:00.3 link=DOWN rx ring 256/4096 drv igb v5.4.0-k / fw 1.52, 0x80000e81, 1.2028.0
Environment
- Red Hat Enterprise Linux
- X2APIC disabled from BIOS
- CPU 8 or lower.
- Multiple NIC
Subscriber exclusive content
A Red Hat subscription provides unlimited access to our knowledgebase, tools, and much more.