Chapter 7. Linux traffic control

Linux offers tools for managing and manipulating the transmission of packets. The Linux Traffic Control (TC) subsystem helps in policing, classifying, shaping, and scheduling network traffic. TC also mangles the packet content during classification by using filters and actions. The TC subsystem achieves this by using queuing disciplines (qdisc), a fundamental element of the TC architecture.

The scheduling mechanism arranges or rearranges the packets before they enter or exit different queues. The most common scheduler is the First-In-First-Out (FIFO) scheduler.

This section explains queuing disciplines and describes how to update the default qdiscs in RHEL.

7.1. Overview of queuing disciplines

Queuing disciplines (qdiscs) help with queuing up and, later, scheduling of traffic transmission by a network interface. A qdisc has two operations;

  • enqueue requests so that a packet can be queued up for later transmission and
  • dequeue requests so that one of the queued-up packets can be chosen for immediate transmission.

Every qdisc has a 16-bit hexadecimal identification number called a handle, with an attached colon, such as 1: or abcd:. This number is called the qdisc major number. If a qdisc has classes, then the identifiers are formed as a pair of two numbers with the major number before the minor, <major>:<minor>, for example abcd:1. The numbering scheme for the minor numbers depends on the qdisc type. Sometimes the numbering is systematic, where the first-class has the ID <major>:1, the second one <major>:2, and so on. Some qdiscs allow the user to set class minor numbers arbitrarily when creating the class.

Classful qdiscs

Different types of qdiscs exist and help in the transfer of packets to and from a networking interface. You can configure qdiscs with root, parent, or child classes. The point where children can be attached are called classes. Classes in qdisc are flexible and can always contain either multiple children classes or a single child, qdisc. There is no prohibition against a class containing a classful qdisc itself, this facilitates complex traffic control scenarios. Classful qdiscs do not store any packets themselves. Instead, they enqueue and dequeue requests down to one of their children according to criteria specific to the qdisc. Eventually, this recursive packet passing ends up where the packets are stored (or picked up from in the case of dequeuing).

Classless qdiscs

Some qdiscs contain no child classes and they are called classless qdiscs. Classless qdiscs require less customization compared to classful qdiscs. It is usually enough to attach them to an interface.

Additional resources

  • For detailed information about classless and classful qdiscs, refer to the tc(8) man page.
  • For detailed information about actions, refer to the actions and tc-actions.8 man pages.

7.2. Available qdiscs in RHEL

Each qdisc addresses unique networking-related issues. The following is the list of qdiscs available in RHEL. You can use any of the following qdisc to shape network traffic based on your networking requirements.

Table 7.1. Available schedulers in RHEL

qdisc nameIncluded inOffload support

Asynchronous Transfer Mode (ATM)

kernel-modules-extra

 

Class-Based Queueing

kernel-modules-extra

 

Credit-Based Shaper

kernel-modules-extra

Yes

CHOose and Keep for responsive flows, CHOose and Kill for unresponsive flows (CHOKE)

kernel-modules-extra

 

Controlled Delay (CoDel)

kernel-core

 

Deficit Round Robin (DRR)

kernel-modules-extra

 

Differentiated Services marker (DSMARK)

kernel-modules-extra

 

Enhanced Transmission Selection (ETS)

kernel-modules-extra

Yes

Fair Queue (FQ)

kernel-core

 

Fair Queuing Controlled Delay (FQ_CODel)

kernel-core

 

Generalized Random Early Detection (GRED)

kernel-modules-extra

 

Hierarchical Fair Service Curve (HSFC)

kernel-core

 

Heavy-Hitter Filter (HHF)

kernel-core

 

Hierarchy Token Bucket (HTB)

kernel-core

 

INGRESS

kernel-core

Yes

Multi Queue Priority (MQPRIO)

kernel-modules-extra

Yes

Multiqueue (MULTIQ)

kernel-modules-extra

Yes

Network Emulator (NETEM)

kernel-modules-extra

 

Proportional Integral-controller Enhanced (PIE)

kernel-core

 

PLUG

kernel-core

 

Quick Fair Queueing (QFQ)

kernel-modules-extra

 

Random Early Detection (RED)

kernel-modules-extra

Yes

Stochastic Fair Blue (SFB)

kernel-modules-extra

 

Stochastic Fairness Queueing (SFQ)

kernel-core

 

Token Bucket Filter (TBF)

kernel-core

Yes

Trivial Link Equalizer (TEQL)

kernel-modules-extra

 
Important

The qdisc offload requires hardware and driver support on NIC.

Additional resources

  • For complete information of parameters and filters used to configure the qdiscs, refer to the tc(8), cbq, cbs, choke, CoDel, drr, fq, htb, mqprio, netem, pie, sfb, pfifo, tc-red, sfq, tbf, and prio man pages.

7.3. Inspecting qdisc counters

By default, Red Hat Enterprise Linux systems use fq_codel qdisc. This procedure describes how to inspect qdisc counters.

Procedure

  1. Optional: View your current qdisc.

    # tc qdisc show dev enp0s1

  2. Inspect the current qdisc counters.

    # tc -s qdisc show dev enp0s1
    qdisc fq_codel 0: root refcnt 2 limit 10240p flows 1024 quantum 1514 target 5.0ms interval 100.0ms memory_limit 32Mb ecn
    Sent 1008193 bytes 5559 pkt (dropped 233, overlimits 55 requeues 77)
    backlog 0b 0p requeues 0
    ....
  • dropped - the number of times a packet is dropped because all queues are full
  • overlimits - the number of times the configured link capacity is filled
  • sent - the number of dequeues

7.4. Updating the default qdisc

If you observe networking packet losses with the current qdisc, you can change the qdisc based on your network-requirements. You can select the qdisc, which meets your network requirements. This procedure describes how to change the default qdisc in Red Hat Enterprise Linux.

Procedure

  1. View the current default qdisc.

    # sysctl -a | grep qdisc
    net.core.default_qdisc = fq_codel
  2. View the qdisc of current Ethernet connection.

    # tc -s qdisc show dev enp0s1
    qdisc fq_codel 0: root refcnt 2 limit 10240p flows 1024 quantum 1514 target 5.0ms interval 100.0ms memory_limit 32Mb ecn
    Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0)
    backlog 0b 0p requeues 0
    maxpacket 0 drop_overlimit 0 new_flow_count 0 ecn_mark 0
    new_flows_len 0 old_flows_len 0
  3. Update the existing qdisc.

    # sysctl -w net.core.default_qdisc=pfifo_fast

  4. To apply the changes, reload the network driver.

    # rmmod NETWORKDRIVERNAME

    # modprobe NETWORKDRIVERNAME

  5. Start the network interface.

    # ip link set enp0s1 up

Verification steps

  • View the qdisc of the Ethernet connection.

    # tc -s qdisc show dev enp0s1
    qdisc pfifo_fast 0: root refcnt 2 bands 3 priomap  1 2 2 2 1 2 0 0 1 1 1 1 1 1 1 1
     Sent 373186 bytes 5333 pkt (dropped 0, overlimits 0 requeues 0)
     backlog 0b 0p requeues 0
    ....

Additional resources

7.5. Updating the current qdisc

You can update the current qdisc without changing the default one. This procedure describes how to change the current qdisc in Red Hat Enterprise Linux.

Procedure

  1. Optional: View the current qdisc.

    # tc -s qdisc show dev enp0s1

  2. Update the current qdisc.

    # tc qdisc replace dev enp0s1 root htb

Verification step

  • View the updated current qdisc.

    # tc -s qdisc show dev enp0s1
    qdisc htb 8001: root refcnt 2 r2q 10 default 0 direct_packets_stat 0 direct_qlen 1000
    Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0)
    backlog 0b 0p requeues 0