4.4. TCP_NODELAY and Small Buffer Writes

Procedure 4.3. Using TCP_NODELAY and TCP_CORK to Improve Network Latency

1. Applications that require lower latency on every packet sent must be run on sockets with TCP_NODELAY enabled. It can be enabled through the setsockopt command with the sockets API:

   # int one = 1;
   
   # setsockopt(descriptor, SOL_TCP, TCP_NODELAY, &one, sizeof(one));
   

2. For this to be used effectively, applications must avoid doing small, logically related buffer writes. Because TCP_NODELAY is enabled, these small writes will make TCP send these multiple buffers as individual packets, which can result in poor overall performance.
   If applications have several buffers that are logically related, and are to be sent as one packet, it is possible to build a contiguous packet in memory and then send the logical packet to TCP on a socket configured with TCP_NODELAY.
   Alternatively, create an I/O vector and pass it to the kernel using writev on a socket configured with TCP_NODELAY.
3. Another option is to use TCP_CORK, which tells TCP to wait for the application to remove the cork before sending any packets. This command will cause the buffers it receives to be appended to the existing buffers. This allows applications to build a packet in kernel space, which can be required when using different libraries that provides abstractions for layers. To enable TCP_CORK, set it to a value of 1 using the setsockopt sockets API (this is known as "corking the socket"):

   # int one = 1;
   
   # setsockopt(descriptor, SOL_TCP, TCP_CORK, &one, sizeof(one));
   

4. When the logical packet has been built in the kernel by the various components in the application, tell TCP to remove the cork. TCP will send the accumulated logical packet right away, without waiting for any further packets from the application.

   # int zero = 0;
   
   # setsockopt(descriptor, SOL_TCP, TCP_CORK, &zero, sizeof(zero));