Red Hat Training

A Red Hat training course is available for RHEL 8

Chapter 28. Linux traffic control

Linux offers tools for managing and manipulating the transmission of packets. The Linux Traffic Control (TC) subsystem helps in policing, classifying, shaping, and scheduling network traffic. TC also mangles the packet content during classification by using filters and actions. The TC subsystem achieves this by using queuing disciplines (qdisc), a fundamental element of the TC architecture.

The scheduling mechanism arranges or rearranges the packets before they enter or exit different queues. The most common scheduler is the First-In-First-Out (FIFO) scheduler. You can do the qdiscs operations temporarily using the tc utility or permanently using NetworkManager.

In Red Hat Enterprise Linux, you can configure default queueing disciplines in various ways to manage the traffic on a network interface.

28.1. Overview of queuing disciplines

Queuing disciplines (qdiscs) help with queuing up and, later, scheduling of traffic transmission by a network interface. A qdisc has two operations;

  • enqueue requests so that a packet can be queued up for later transmission and
  • dequeue requests so that one of the queued-up packets can be chosen for immediate transmission.

Every qdisc has a 16-bit hexadecimal identification number called a handle, with an attached colon, such as 1: or abcd:. This number is called the qdisc major number. If a qdisc has classes, then the identifiers are formed as a pair of two numbers with the major number before the minor, <major>:<minor>, for example abcd:1. The numbering scheme for the minor numbers depends on the qdisc type. Sometimes the numbering is systematic, where the first-class has the ID <major>:1, the second one <major>:2, and so on. Some qdiscs allow the user to set class minor numbers arbitrarily when creating the class.

Classful qdiscs

Different types of qdiscs exist and help in the transfer of packets to and from a networking interface. You can configure qdiscs with root, parent, or child classes. The point where children can be attached are called classes. Classes in qdisc are flexible and can always contain either multiple children classes or a single child, qdisc. There is no prohibition against a class containing a classful qdisc itself, this facilitates complex traffic control scenarios.

Classful qdiscs do not store any packets themselves. Instead, they enqueue and dequeue requests down to one of their children according to criteria specific to the qdisc. Eventually, this recursive packet passing ends up where the packets are stored (or picked up from in the case of dequeuing).

Classless qdiscs
Some qdiscs contain no child classes and they are called classless qdiscs. Classless qdiscs require less customization compared to classful qdiscs. It is usually enough to attach them to an interface.

Additional resources

  • tc(8) man page
  • tc-actions(8) man page

28.2. Inspecting qdiscs of a network interface using the tc utility

By default, Red Hat Enterprise Linux systems use fq_codel qdisc. You can inspect the qdisc counters using the tc utility.

Procedure

  1. Optional: View your current qdisc:

    # tc qdisc show dev enp0s1
  2. Inspect the current qdisc counters:

    # tc -s qdisc show dev enp0s1
    qdisc fq_codel 0: root refcnt 2 limit 10240p flows 1024 quantum 1514 target 5.0ms interval 100.0ms memory_limit 32Mb ecn
    Sent 1008193 bytes 5559 pkt (dropped 233, overlimits 55 requeues 77)
    backlog 0b 0p requeues 0
    • dropped - the number of times a packet is dropped because all queues are full
    • overlimits - the number of times the configured link capacity is filled
    • sent - the number of dequeues

28.3. Updating the default qdisc

If you observe networking packet losses with the current qdisc, you can change the qdisc based on your network-requirements.

Procedure

  1. View the current default qdisc:

    # sysctl -a | grep qdisc
    net.core.default_qdisc = fq_codel
  2. View the qdisc of current Ethernet connection:

    # tc -s qdisc show dev enp0s1
    qdisc fq_codel 0: root refcnt 2 limit 10240p flows 1024 quantum 1514 target 5.0ms interval 100.0ms memory_limit 32Mb ecn
    Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0)
    backlog 0b 0p requeues 0
    maxpacket 0 drop_overlimit 0 new_flow_count 0 ecn_mark 0
    new_flows_len 0 old_flows_len 0
  3. Update the existing qdisc:

    # sysctl -w net.core.default_qdisc=pfifo_fast
  4. To apply the changes, reload the network driver:

    # modprobe -r NETWORKDRIVERNAME
    # modprobe NETWORKDRIVERNAME
  5. Start the network interface:

    # ip link set enp0s1 up

Verification

  • View the qdisc of the Ethernet connection:

    # tc -s qdisc show dev enp0s1
    qdisc pfifo_fast 0: root refcnt 2 bands 3 priomap  1 2 2 2 1 2 0 0 1 1 1 1 1 1 1 1
     Sent 373186 bytes 5333 pkt (dropped 0, overlimits 0 requeues 0)
     backlog 0b 0p requeues 0
    ....

28.4. Temporarily setting the current qdisc of a network interface using the tc utility

You can update the current qdisc without changing the default one.

Procedure

  1. Optional: View the current qdisc:

    # tc -s qdisc show dev enp0s1
  2. Update the current qdisc:

    # tc qdisc replace dev enp0s1 root htb

Verification

  • View the updated current qdisc:

    # tc -s qdisc show dev enp0s1
    qdisc htb 8001: root refcnt 2 r2q 10 default 0 direct_packets_stat 0 direct_qlen 1000
    Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0)
    backlog 0b 0p requeues 0

28.5. Permanently setting the current qdisc of a network interface using NetworkManager

You can update the current qdisc value of a NetworkManager connection.

Procedure

  1. Optional: View the current qdisc:

    # tc qdisc show dev enp0s1
      qdisc fq_codel 0: root refcnt 2
  2. Update the current qdisc:

    # nmcli connection modify enp0s1 tc.qdiscs ‘root pfifo_fast’
  3. Optional: To add another qdisc over the existing qdisc, use the +tc.qdisc option:

    # nmcli connection modify enp0s1 +tc.qdisc ‘ingress handle ffff:’
  4. Activate the changes:

    # nmcli connection up enp0s1

Verification

  • View current qdisc the network interface:

    # tc qdisc show dev enp0s1
    qdisc pfifo_fast 8001: root refcnt 2 bands 3 priomap  1 2 2 2 1 2 0 0 1 1 1 1 1 1 1 1
    qdisc ingress ffff: parent ffff:fff1 ----------------

Additional resources

  • nm-settings(5) man page

28.6. Available qdiscs in RHEL

Each qdisc addresses unique networking-related issues. The following is the list of qdiscs available in RHEL. You can use any of the following qdisc to shape network traffic based on your networking requirements.

Table 28.1. Available schedulers in RHEL

qdisc nameIncluded inOffload support

Asynchronous Transfer Mode (ATM)

kernel-modules-extra

 

Class-Based Queueing

kernel-modules-extra

 

Credit-Based Shaper

kernel-modules-extra

Yes

CHOose and Keep for responsive flows, CHOose and Kill for unresponsive flows (CHOKE)

kernel-modules-extra

 

Controlled Delay (CoDel)

kernel-core

 

Deficit Round Robin (DRR)

kernel-modules-extra

 

Differentiated Services marker (DSMARK)

kernel-modules-extra

 

Enhanced Transmission Selection (ETS)

kernel-modules-extra

Yes

Fair Queue (FQ)

kernel-core

 

Fair Queuing Controlled Delay (FQ_CODel)

kernel-core

 

Generalized Random Early Detection (GRED)

kernel-modules-extra

 

Hierarchical Fair Service Curve (HSFC)

kernel-core

 

Heavy-Hitter Filter (HHF)

kernel-core

 

Hierarchy Token Bucket (HTB)

kernel-core

 

INGRESS

kernel-core

Yes

Multi Queue Priority (MQPRIO)

kernel-modules-extra

Yes

Multiqueue (MULTIQ)

kernel-modules-extra

Yes

Network Emulator (NETEM)

kernel-modules-extra

 

Proportional Integral-controller Enhanced (PIE)

kernel-core

 

PLUG

kernel-core

 

Quick Fair Queueing (QFQ)

kernel-modules-extra

 

Random Early Detection (RED)

kernel-modules-extra

Yes

Stochastic Fair Blue (SFB)

kernel-modules-extra

 

Stochastic Fairness Queueing (SFQ)

kernel-core

 

Token Bucket Filter (TBF)

kernel-core

Yes

Trivial Link Equalizer (TEQL)

kernel-modules-extra

 
Important

The qdisc offload requires hardware and driver support on NIC.

Additional resources

  • tc(8) man page