Low Throughput on XXV710 using SR-IOV with NIC partitioning, trusted VFs and promiscuous mode enabled

Solution In Progress - Updated -

Issue

  • We have run some throughput tests on the Intel XXV710 NICs with SR-IOV, NIC partitioning, trusted VFs and promiscuous mode enabled on the VM (testpmd).

  • We noticed that there is a 55% drop in throughput when we use NIC partitioning with trusted VFs. Without NIC partitioning, we also see a drop in throughput of 13% when using trusted VFs. If we disable promiscuous mode, there is no change in throughput.

  • With the Mellanox MT27800 25G NIC, we don't see this difference. We can achieve close to line rate with NIC partitioning, trusted VFs and promiscuous mode on.

Throughput per port - testpmd (promisc on) - 1500 Byte IP packets:

-------------------------------------------------------------------------
Intel XXV710       NIC partitioning off, trust on:  20,8 Gbps 
                                NIC partitioning off, trust off:  24 Gbps 
                                NIC partitioning on,  trust off:  24 Gbps 
                                NIC partitioning on,  trust on:  10,8 Gbps 
-------------------------------------------------------------------------
Mellanox 25G      NIC partitioning on,  trust on:    24,96 Gbps
-------------------------------------------------------------------------
  • Throughput per port - testpmd (promisc off) - 1500 Byte IP packets:
-------------------------------------------------------------------------
Intel XXV710       NIC partitioning off, trust on:  24 Gbps 
                                NIC partitioning off, trust off:  24 Gbps 
                                NIC partitioning on,  trust off:  24 Gbps 
                                NIC partitioning on,  trust on:  24 Gbps 
-------------------------------------------------------------------------
  • With NIC partitioning, trusted VFs and promiscuos mode, we see drops on the Intel NIC when we increase the traffic:
[root@overcloud-compute-sriovmii-0 ~]# ethtool -S ens3f0 | grep drop
     rx_dropped: 0
     tx_dropped: 0
     port.rx_dropped: 1122236
     port.tx_dropped_link_down: 0
[root@overcloud-compute-sriovmii-0 ~]# ethtool -S ens3f1 | grep drop
     rx_dropped: 0
     tx_dropped: 0
     port.rx_dropped: 11075283
     port.tx_dropped_link_down: 0
  • Is this behaviour expected on the Intel NICs?

Environment

  • Red Hat OpenStack Platform 16.1 (RHOSP)

Subscriber exclusive content

A Red Hat subscription provides unlimited access to our knowledgebase of over 48,000 articles and solutions.

Current Customers and Partners

Log in for full access

Log In