Low Throughput on XXV710 using SR-IOV with NIC partitioning, trusted VFs and promiscuous mode enabled
Issue
-
We have run some throughput tests on the Intel XXV710 NICs with SR-IOV, NIC partitioning, trusted VFs and promiscuous mode enabled on the VM (testpmd).
-
We noticed that there is a 55% drop in throughput when we use NIC partitioning with trusted VFs. Without NIC partitioning, we also see a drop in throughput of 13% when using trusted VFs. If we disable promiscuous mode, there is no change in throughput.
-
With the Mellanox MT27800 25G NIC, we don't see this difference. We can achieve close to line rate with NIC partitioning, trusted VFs and promiscuous mode on.
Throughput per port - testpmd (promisc on) - 1500 Byte IP packets:
-------------------------------------------------------------------------
Intel XXV710 NIC partitioning off, trust on: 20,8 Gbps
NIC partitioning off, trust off: 24 Gbps
NIC partitioning on, trust off: 24 Gbps
NIC partitioning on, trust on: 10,8 Gbps
-------------------------------------------------------------------------
Mellanox 25G NIC partitioning on, trust on: 24,96 Gbps
-------------------------------------------------------------------------
- Throughput per port - testpmd (promisc off) - 1500 Byte IP packets:
-------------------------------------------------------------------------
Intel XXV710 NIC partitioning off, trust on: 24 Gbps
NIC partitioning off, trust off: 24 Gbps
NIC partitioning on, trust off: 24 Gbps
NIC partitioning on, trust on: 24 Gbps
-------------------------------------------------------------------------
- With NIC partitioning, trusted VFs and promiscuos mode, we see drops on the Intel NIC when we increase the traffic:
[root@overcloud-compute-sriovmii-0 ~]# ethtool -S ens3f0 | grep drop
rx_dropped: 0
tx_dropped: 0
port.rx_dropped: 1122236
port.tx_dropped_link_down: 0
[root@overcloud-compute-sriovmii-0 ~]# ethtool -S ens3f1 | grep drop
rx_dropped: 0
tx_dropped: 0
port.rx_dropped: 11075283
port.tx_dropped_link_down: 0
- Is this behaviour expected on the Intel NICs?
Environment
- Red Hat OpenStack Platform 16.1 (RHOSP)
Subscriber exclusive content
A Red Hat subscription provides unlimited access to our knowledgebase of over 48,000 articles and solutions.