Menu Close
Settings Close

Language and Page Formatting Options

Red Hat Training

A Red Hat training course is available for RHEL 8

Chapter 34. Monitoring and tuning NIC ring buffers

Receive ring buffers are shared between the device driver and network interface controller (NIC). The card assigns a transmit (TX) and receive (RX) ring buffer. As the name implies, the ring buffer is a circular buffer where an overflow overwrites existing data. There are two ways to move data from the NIC to the kernel, hardware interrupts and software interrupts, also called SoftIRQs.

The kernel uses the RX ring buffer to store incoming packets until they can be processed by the device driver. The device driver drains the RX ring, typically using SoftIRQs, which puts the incoming packets into a kernel data structure called an sk_buff or skb to begin its journey through the kernel and up to the application which owns the relevant socket.

The kernel uses the TX ring buffer to hold outgoing packets which are destined for the wire.

These ring buffers reside at the bottom of the stack and are a crucial point at which packet drop can occur, which in turn will adversely affect network performance.

34.1. Displaying the number of dropped packets

The ethtool utility enables administrators to query, configure, or control network driver settings.

The exhaustion of the ring buffers causes an increment in the counters, such as "discard" or "drop" in the output of ethtool -S interface_name. The discarded packets indicate that the available buffer is filling up faster than the kernel can process the packets.

Procedure

  • Display the drop counters for the enp1s0 interface:

    $ ethtool -S enp1s0

34.2. Increasing the ring buffers to reduce a high packet drop rate

You can increase the size of an Ethernet device’s ring buffers if the packet drop rate causes applications to report:

  • a loss of data
  • cluster fence
  • slow performance
  • timeouts
  • failed large data transfers, such as backups

Identify the number of dropped packets, and increase the TX and RX ring buffer to reduce a high packet drop rate.

Procedure

  1. Display the maximum ring buffer sizes:

    # ethtool -g enp1s0
     Ring parameters for enp1s0:
     Pre-set maximums:
     RX:             4096
     RX Mini:        0
     RX Jumbo:       16320
     TX:             4096
     Current hardware settings:
     RX:             255
     RX Mini:        0
     RX Jumbo:       0
     TX:             255
  2. If the values in the Pre-set maximums section are higher than in the Current hardware settings section increase the ring buffers:

    • To increase the RX ring buffer, enter:

      # nmcli connection modify Example-Connection ethtool.ring-rx 4096
    • To increase the TX ring buffer, enter:

      # nmcli connection modify Example-Connection ethtool.ring-tx 4096
  3. Reload the NetworkManager connection:

    # nmcli connection up Example-Connection
    Important

    Depending on the driver your network interface card uses, changing in the ring buffer can shortly interrupt the network connection.