2.9. Network Determinism Tips
TCP can have a large effect on latency. TCP adds latency in order to obtain efficiency, control congestion, and to ensure reliable delivery. When tuning, consider the following points:
- Do you need ordered delivery?
- Do you need to guard against packet loss?Transmitting packets more than once can cause delays.
- If you must use TCP, consider disabling the Nagle buffering algorithm by using
TCP_NODELAYon your socket. The Nagle algorithm collects small outgoing packets to send all at once, and can have a detrimental effect on latency.
- Interrupt Coalescing
- To reduce the amount of interrupts, packets can be collected and a single interrupt generated for a collection of packets.In systems that transfer large amounts of data where throughput is a priority, using the default value or increasing coalesce can increase throughput and lower the number of interrupts hitting CPUs. For systems requiring a rapid network response, reducing or disabling coalesce is advised.Use the
--coalesce) option with the
ethtoolcommand to enable.
- Often, I/O switches can be subject to back-pressure, where network data builds up as a result of full buffers.Use the
--pause) option with the
ethtoolcommand to change pause parameters and avoid network congestion.
- Infiniband (IB)
- Infiniband is a type of communications architecture often used to increase bandwidth and provide quality of service and failover. It can also be used to improve latency through Remote Direct Memory Access (RDMA) capabilities.
- Network Protocol Statistics
- Use the
--statistics) option with the
netstatcommand to monitor network traffic.
For more information, or for further reading, the following man pages are related to the information given in this section.