Rx Discards on switch

Latest response

I have two Dell PowerEdge r740 servers (RHEL-8 Operating System) connected to each other via SFP+ cables. I want to enable UDP-Fragmentation-Offload on the interfaces connecting these servers. When I try to do it with ethtool, like this:

$ethtool -K eno1 ufo on

I get:

unable to change UDP-Fragmentation-Offload. Unable to change any device features.

UFO is off on the interface. I am sure the NIC supports it because ethtool shows it as off, not as off [fixed]. I want to know are their any drivers etc. missing? What could be the reason?

Responses

What's the driver the NICs are using? Like ethtool -i eno1

Do you have the interfaces in a bond or team or bridge or OVS or something else like that?

No I don't have the interface in any bond, team or bridge. This is the output:

$ ethtool -i eno1

driver: i40e

version: 2.3.2-k

firmware-version: 7.10 0x800051ab 19.0.12

expansion-rom-version:

bus-info: 0000:18:00.0

supports-statistics: yes

supports-test: yes

supports-eeprom-access: yes

supports-register-dump: yes

supports-priv-flags: yes

Turns out UFO is removed as of upstream kernel v4.14:

The justification for the commit is:

Very few devices support this operation, it's usefullness is quesitonable at best, and it adds a non-trivial amount of complexity to our data paths.

I'll get this documented on the knowledgebase.

Well, I was sending UDP traffic from one machine to another, and I got around 6.5 Gbps. Currently, interfaces have GSO enabled. I was hoping to get around 9+ Gbps if I enable UFO. Is there any way to enable it?

No, the UFO code is completely gone.

Probably UFO is not the bottleneck there, but UDP's lack of congestion control.

If you need a performant stream, use a protocol which provides this such as TCP or SCTP, or you could also implement your own bandwidth moderation on top of UDP.

I've added to the knowledgebase at: Unable to change UDP-Fragmentation-Offload

Thanks. Isn't UFO implemented in the NIC? If RHEL-8 does not support it, do you think changing OS might work here?

That's the thing, hardly any NICs actually did implement UFO in hardware.

Looking back in the RHEL7 kernel tree (where UFO does exist) the only drivers which advertise UFO support are:

  • s2io
  • tilepro
  • macvlan
  • tun
  • virtio_net

Your NIC uses i40e so it never advertised any UFO support in the hardware.

Any Linux distribution with a kernel later than v4.14 will not have UFO. This isn't just a RHEL thing.

Thank you so much for the help!

One last question, It is possible to switch to RHEL 7 and change NIC driver to one of the drivers you mentioned?

If you have the hardware, sure.

s2io is the driver for "Neterion 10GbE NIC". This looks to be from 2006 and has had little work done since.

Whilst we don't favor any vendor over another, I suggest your Intel card - which is a decade newer than the Neterion, and has had significant feature and improvement work done by Intel since its introduction - is probably a better choice here.

Correction, tilepro never actually supported UFO, it just had a code comment which said that Tilera wanted to implement it one day. Turns out they never did, at least not in Linux.

The others are all virtual drivers so they're not a piece of hardware you can buy.

Like I said, UFO is almost certainly not the bottleneck here. UDP itself probably is. UDP is not made for reliable high-throughput data transfers. Don't use plain UDP for that purpose.