Network - high throughput requirement
Has anyone worked with adjusting the txqueuelen to something other than the default? I have 2 systems with high throughout requirements one with a qlen of 1000 and the other with 10000. The throughout performance is there with the larger one but I’m reluctant to make this change without some idea of the possible impact..
Responses
Pro
469 points
Hi,
It is good that you are testing it because the real answer is:
It depends on your applications.
The downsides to increasing queue lengths could be:
Using more memory, The queue might become longer (affects the latency experienced by packets in the queue).
For example, large FTP sessions prefer data to be held in the queue in the event of congestion.
Regards,
Dusan Baljevic (amateur radio VK2COT)
Red Hat
Guru
7257 points
This is the correct answer.
You can quantify TX queue usage by monitoring tc -s qdisc over time.
If you aren't getting drops, you don't need longer TX queues.
Our monitor.sh script makes watching this and other network performance metrics a bit easier.
Red Hat
Guru
7257 points
Yes, it could be the reason. The drops counter on the qdisc is directly related to freeing packets from memory, so it's a "packet drop" just like a CRC error or a buffer miss.
The traffic is sent from TCP but never delivered to the other end. Actually with qdisc drops, that dropped traffic is never even delivered to the device to transmit on the wire.
Depending on TCP stream parameters, a drop rate as low as 0.002% can measurably affect TCP throughput. Using SACK really helps here but you don't always have control over both ends of a TCP stream or middleboxes.
Increasing the txqueuelen is an easy and non-invasive thing to test.
Welcome! Check out the Getting Started with Red Hat page for quick tours and guides for common tasks.