Language and Page Formatting Options
Chapter 34. Monitoring and tuning NIC ring buffers
Receive ring buffers are shared between the device driver and network interface controller (NIC). The card assigns a transmit (TX) and receive (RX) ring buffer. As the name implies, the ring buffer is a circular buffer where an overflow overwrites existing data. There are two ways to move data from the NIC to the kernel, hardware interrupts and software interrupts, also called SoftIRQs.
The kernel uses the RX ring buffer to store incoming packets until they can be processed by the device driver. The device driver drains the RX ring, typically using SoftIRQs, which puts the incoming packets into a kernel data structure called an
skb to begin its journey through the kernel and up to the application which owns the relevant socket.
The kernel uses the TX ring buffer to hold outgoing packets which are destined for the wire.
These ring buffers reside at the bottom of the stack and are a crucial point at which packet drop can occur, which in turn will adversely affect network performance.
34.1. Displaying the number of dropped packets
ethtool utility enables administrators to query, configure, or control network driver settings.
The exhaustion of the ring buffers causes an increment in the counters, such as "discard" or "drop" in the output of
ethtool -S interface_name. The discarded packets indicate that the available buffer is filling up faster than the kernel can process the packets.
Display the drop counters for the
$ ethtool -S enp1s0
34.2. Increasing the ring buffers to reduce a high packet drop rate
You can increase the size of an Ethernet device’s ring buffers if the packet drop rate causes applications to report:
- a loss of data
- cluster fence
- slow performance
- failed large data transfers, such as backups
Identify the number of dropped packets, and increase the TX and RX ring buffer to reduce a high packet drop rate.
Display the maximum ring buffer sizes:
# ethtool -g enp1s0 Ring parameters for enp1s0: Pre-set maximums: RX: 4096 RX Mini: 0 RX Jumbo: 16320 TX: 4096 Current hardware settings: RX: 255 RX Mini: 0 RX Jumbo: 0 TX: 255
If the values in the
Pre-set maximumssection are higher than in the
Current hardware settingssection increase the ring buffers:
To increase the RX ring buffer, enter:
# nmcli connection modify Example-Connection ethtool.ring-rx 4096
To increase the TX ring buffer, enter:
# nmcli connection modify Example-Connection ethtool.ring-tx 4096
Reload the NetworkManager connection:
# nmcli connection up Example-ConnectionImportant
Depending on the driver your network interface card uses, changing in the ring buffer can shortly interrupt the network connection.