Select Your Language

Infrastructure and Management

Cloud Computing

Storage

Runtimes

Integration and Automation

  • Comments
  • TCP tuning for high bandwidth delay product network

    Posted on

    We are trying to copy some files over a very high latency connection of about 325ms round-trip. Using UDP, I'm able to get throughput speeds of about 80Mb/s, but using various TCP-based methods such as SCP or http, I am only getting about 25Mb/s. In addition to our "production" environment, we have also used a network emulator in a lab environment to replicate the bandwidth and latency, and we are getting the same results. After doing some web searching, I tried doing the following things:

    • Sending system:
      Net.core.wmem_default set to 8000000 (2x BDP)
      Net.core.wmem_max set to 16000000
      Net.ipv4.tcp_wmem set to 4096 8000000 16000000
      Rebooted, verified parameters
      Ethernet interface ring buffer sizes increased from 512 default to
      4096 maximum
      increased txqueuelen to 10000

    • Receiving system:
    Net.core.rmem_default set to 8000000
    Net.core.rmem_max set to 16000000
    Net.ipv4.tcp_rmem set to 4096 8000000 16000000
    Rebooted, verified parameters
    Ethernet interface ring buffer set to 4096 from 512

    Not only did this not fix the problem, we aren't seeing any appreciable difference in throughput. One thing we have tried is multi-threading, which gave us some moderate improvement, but I'm wondering if anyone has had success with tuning for a high latency connection, and if so, what are we missing? Thanks for your help!

    by

    points

    Responses

    Red Hat LinkedIn YouTube Facebook X, formerly Twitter

    Quick Links

    Help

    Site Info

    Related Sites

    © 2025 Red Hat