How to determine TCP initial window size and scaling option? Which factors affect the determination?

Solution Verified - Updated -

Environment

  • Red Hat Enterprise Linux (RHEL) 7
  • Red Hat Enterprise Linux (RHEL) 6
  • Red Hat Enterprise Linux (RHEL) 5
  • Red Hat Enterprise Linux (RHEL) 4
  • Linux kernel 2.6 and newer

Issue

  • I have two systems providing telecommunication service. Both systems connect to each other and communicate with another side frequently. TCP initial window sizes of these connections keep changing and each system seems to use different window sizes.
  • How to determine TCP initial window size? Which factors affect the determination?

Resolution

Unless problems are encountered with a particular system, or network path, the system defaults will provide good performance and should be left alone.

Determining Initial TCP window size value

There are several factors which affect the initial TCP window size:

  • The size of socket's receive buffer
  • TCP maximum segment size (MSS)
  • The window scale extension
  • TCP Timestamps
  • tcp_workaround_signed_windows parameter

  • Here is the relevant code in kernel 2.6 of Red Hat Enterprise Linux:

if (mss > (1<<*rcv_wscale)) {
     int init_cwnd = 4;
     if (mss > 1460*3)
          init_cwnd = 2;
     else if (mss > 1460)
          init_cwnd = 3;
     if (*rcv_wnd > init_cwnd*mss)
          *rcv_wnd = init_cwnd*mss;
}
  • Generally, if the MSS is 1460 bytes, the initial window size is going to be 5840 (1460 * 4), although there are still other factors that could change the size. For example, it can be 5792 when TCP Timestamp is set because there ACK timestamp overhead is 12 bytes.
net.ipv4.tcp_timestamps = 1
  • Note1 :
MSS = MTU - size of the TCP header (40 bytes)

Determining TCP window scale

  • The base window size can not exceed 65535 bytes due to limitations of the TCP header:
The TCP header uses a 16 bit field to report the receive
window size to the sender.  Therefore, the largest window
that can be used is 2**16 = 65K bytes.
                                               - RFC 1323
  • The Window Scale TCP option allows windows larger than 65K bytes by using a scale factor to multiply the window size value. This factor is set based on maximum size of receive buffer used by TCP sockets:
net.ipv4.tcp_rmem = 4096    87380    4194304
  • The code below shows how the kernel defines the scale factor:
if (wscale_ok) {
       space = max_t(u32, sysctl_tcp_rmem[2], sysctl_rmem_max);
       while (space > 65535 && (*rcv_wscale) < 14) {
               space >>= 1;
               (*rcv_wscale)++;
       }
}

So, if the maximum size of the receive buffer, tcp_rmem, is set to 4194304, the scale factor will be 7.

Note : the scale factor can be up to 14.

For more information, refer to RFC 1323.

Relevant kernel parameters

These parameters can affect the TCP window size determination:

net.ipv4.tcp_timestamps
net.core.rmem_max
net.ipv4.tcp_rmem
net.ipv4.tcp_adv_win_scale
net.ipv4.tcp_window_scaling
net.ipv4.tcp_workaround_signed_windows

This solution is part of Red Hat’s fast-track publication program, providing a huge library of solutions that Red Hat engineers have created while supporting our customers. To give you the knowledge you need the instant it becomes available, these articles may be presented in a raw and unedited form.

4 Comments

Is this also applicable for RHEL 6?

Jan, I'm not sure. I'll reach out to a support engineer and see if this needs to be updated.

Hi, Jan,

Yes, it is basically applicable for RHEL 6 since the TCP stack in Red Hat kernel follows TCP standard.

Note that in 6.2, the initial window cap was increased to 10*MSS (~15k). In 6.4, this will be tunable per route via iproute (initrwnd).