Chapter 18. Network determinism tips

TCP can have a large effect on latency. TCP adds latency in order to obtain efficiency, control congestion, and to ensure reliable delivery. When tuning, consider the following points:

  • Do you need ordered delivery?

  • Do you need to guard against packet loss?

    Transmitting packets more than once can cause delays.

  • Do you need to use TCP?

    Consider disabling the Nagle buffering algorithm by using TCP_NODELAY on your socket. The Nagle algorithm collects small outgoing packets to send all at once, and can have a detrimental effect on latency.

18.1. Coalescing interrupts

In systems that transfer large amounts of data where throughput is a priority, using the default value or increasing coalescence can increase throughput and lower the number of interrupts hitting CPUs. For systems requiring a rapid network response, reducing or disabling coalescence is advised.

To reduce the number of interrupts, packets can be collected and a single interrupt generated for a collection of packets.

Prerequisites

  • You have administrator privileges.

Procedure

  • To enable coalescing interrupts, run the ethtool command with the --coalesce option. 

    # ethtool -C tun0

Verification

Verify that coalescing interrupts are enabled. 

# ethtool -c tun0
Coalesce parameters for tun0:
Adaptive RX: n/a  TX: n/a
stats-block-usecs: n/a
sample-interval: n/a
pkt-rate-low: n/a
pkt-rate-high: n/a

rx-usecs: n/a
rx-frames: 0
rx-usecs-irq: n/a
rx-frames-irq: n/a

tx-usecs: n/a
tx-frames: n/a
tx-usecs-irq: n/a
tx-frames-irq: n/a

rx-usecs-low: n/a
rx-frame-low: n/a
tx-usecs-low: n/a
tx-frame-low: n/a

rx-usecs-high: n/a
rx-frame-high: n/a
tx-usecs-high: n/a
tx-frame-high: n/a

CQE mode RX: n/a  TX: n/a

18.2. Avoiding network congestion

I/O switches can often be subject to back-pressure, where network data builds up as a result of full buffers. You can change pause parameters and avoid network congestion.

Prerequisites

  • You have administrator privileges.

Procedure

  • To change pause parameters, run the ethtool command with the -A option. 

    # ethtool -A enp0s31f6

Verification

Verify that the pause parameter changed. 

# ethtool -a enp0s31f6
Pause parameters for enp0s31f6:
Autonegotiate:	on
RX:		on
TX:		on

18.3. Monitoring network protocol statistics

The netstat command can be used to monitor network traffic.

Procedure

To monitor network traffic: 

$ netstat -s
Ip:
    Forwarding: 1
    30817508 total packets received
    2927 forwarded
    0 incoming packets discarded
    30813320 incoming packets delivered
    19184491 requests sent out
    181 outgoing packets dropped
    2628 dropped because of missing route
Icmp
    29450 ICMP messages received
    213 input ICMP message failed
    ICMP input histogram:
        destination unreachable: 29431
        echo requests: 19
    10141 ICMP messages sent
    0 ICMP messages failed
    ICMP output histogram:
        destination unreachable: 10122
        echo replies: 19
IcmpMsg:
        InType3: 29431
        InType8: 19
        OutType0: 19
        OutType3: 10122
Tcp:
    162638 active connection openings
    89 passive connection openings
    38908 failed connection attempts
    17869 connection resets received
    48 connections established
    8456952 segments received
    9323882 segments sent out
    69885 segments retransmitted
    1143 bad segments received
    56209 resets sent
Udp:
    21929780 packets received
    1319 packets to unknown port received
    712919 packet receive errors
    10134989 packets sent
    712919 receive buffer errors
    180 send buffer errors
    IgnoredMulti: 39231

18.4. Additional resources

  • ethtool(8) man page
  • netstat(8) man page