Table of Contents
Two security flaws named SegmentSmack and FragmentSmack were recently found in the Linux kernel.
A flaw named SegmentSmack was found in the way the Linux kernel handled specially crafted TCP packets. A remote attacker could use this flaw to trigger time and calculation expensive calls to tcp_collapse_ofo_queue() and tcp_prune_ofo_queue() functions by sending specially modified packets within ongoing TCP sessions which could lead to a CPU saturation and hence a denial of service on the system with relatively small bandwidth of the incoming network traffic. In a worst case scenario, an attacker can stall an affected host or device with less than 2 kpps of an attack traffic. Maintaining the denial of service condition requires continuous two-way TCP sessions to a reachable open port, thus the attacks cannot be performed using spoofed IP addresses.
A result of the attack with 4 streams can look like a complete saturation of 4 CPU cores and delays in a network packets processing:
$ top %Cpu25 : 0.0 us, 0.0 sy, 0.0 ni, 1.4 id, 0.0 wa, 0.0 hi, 98.5 si, 0.0 st %Cpu26 : 0.0 us, 0.0 sy, 0.0 ni, 1.4 id, 0.0 wa, 0.0 hi, 98.6 si, 0.0 st %Cpu28 : 0.0 us, 0.3 sy, 0.0 ni, 0.7 id, 0.0 wa, 0.0 hi, 99.0 si, 0.0 st %Cpu30 : 0.0 us, 0.0 sy, 0.0 ni, 1.4 id, 0.0 wa, 0.0 hi, 98.6 si, 0.0 st PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND 141 root 20 0 0 0 0 R 97.3 0.0 1:16.33 ksoftirqd/26 151 root 20 0 0 0 0 R 97.3 0.0 1:16.68 ksoftirqd/28 136 root 20 0 0 0 0 R 97.0 0.0 0:39.09 ksoftirqd/25 161 root 20 0 0 0 0 R 97.0 0.0 1:16.48 ksoftirqd/30
A flaw named FragmentSmack was found in the way the Linux kernel handled reassembly of fragmented IPv4 and IPv6 packets. A remote attacker could use this flaw to trigger time and calculation expensive fragment reassembly algorithm by sending specially crafted packets which could lead to a CPU saturation and hence a denial of service on the system.
A result of the 30 kpps attack on the physical system with Intel(R) Xeon(R) Dfirstname.lastname@example.orgGHz CPUs and 32 cores in total may look like a complete saturation of a core:
top - 08:59:45 up 1:34, 2 users, load average: 0.39, 0.15, 0.08 %Cpu9 : 0.0 us, 0.0 sy, 0.0 ni, 0.0 id, 0.0 wa, 0.0 hi,100.0 si, 0.0 st PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND 54 root 20 0 0 0 0 R 99.7 0.0 0:47.53 ksoftirqd/9
An attack from a single IP host may saturate more than 1 CPU core by forging packets to be sent from different IP addresses. The Linux kernel uses complex algorithm to schedule such IP fragment reassembly among the CPU cores. So such reassembly could be distributed to the different CPU cores, but it is quite harder to achieve this compared to the SegmentSmack flaw. Such an attack from 2 forged IP addresses may look like a complete saturation of 2 cores, but it is harder for an attacker to achieve this:
top - 10:10:36 up 34 min, 2 users, load average: 0.51, 0.29, 0.15 %Cpu3 : 0.0 us, 0.0 sy, 0.0 ni, 0.0 id, 0.0 wa, 0.0 hi,100.0 si, 0.0 st %Cpu7 : 0.0 us, 0.0 sy, 0.0 ni, 0.0 id, 0.0 wa, 0.0 hi,100.0 si, 0.0 st PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND 24 root 20 0 0 0 0 R 100.0 0.0 1:50.69 ksoftirqd/3 44 root 20 0 0 0 0 R 100.0 0.0 1:07.11 ksoftirqd/7
SegmentSmack and FragmentSmack attacks are possible due to the algorithms used in the Linux kernel network stack; all the Red Hat products with moderately new Linux kernel versions are affected.
- Red Hat Enterprise Linux 6
- Red Hat Enterprise Linux 7
- Red Hat Enterprise Linux 7 for Real Time
- Red Hat Enterprise Linux 7 for ARM64
- Red Hat Enterprise Linux 7 for Power
- Red Hat Enterprise Linux Atomic Host
- Red Hat Enterprise MRG 2
- Red Hat Virtualization 4
RHEL-5 is affected by these flaws by a significantly lesser degree. Namely, in our tests only a high-speed attack of 1Mpps (packets, not bytes or bits) was able to barely saturate 1 CPU core. As such, the flaws severity for RHEL5 is considered Moderate.
No effective workaround or mitigation besides a fixed kernel is known at this time. Red Hat is tracking fixes via Bugzilla ticket 1601704. Red Hat Enterprise Linux kernel updates will be released as they become available.
Updates for Affected Products
|Red Hat Enterprise Linux 7||kernel||RHSA-2018:2384|
|Red Hat Enterprise Linux 7 for Real Time||kernel-rt||RHSA-2018:2395|
|Red Hat Enterprise Linux 6||kernel||RHSA-2018:2390|
|Red Hat Enterprise Linux 6.7 Extended Update Support**||kernel||RHSA-2018:2645|
|Red Hat Virtualization 4||rhvm-appliance||RHSA-2018:2402|
|Red Hat Virtualization 4||redhat-virtualization-host||RHSA-2018:2403|
**An active EUS subscription is required for access to this patch. Please contact Red Hat sales or your specific sales representative for more information if your account does not have an active EUS subscription. Also see What is the Red Hat Enterprise Linux Extended Update Support Subscription?.
Red Hat is tracking fixes via Bugzilla ticket 1609664. Red Hat Enterprise Linux kernel updates will be released as they become available.
Except installing a fixed kernel, one may try to change the default 4MB and 3MB values of
net.ipv4.ipfrag_low_thresh (and their IPv6 counterparts
net.ipv6.ipfrag_low_thresh) sysctl parameters to 256 kB and 192 kB (respectively) or below. The result is from some to significant CPU saturation drop during an attack, depending on a hardware and environment. For example, this mitigation applied to the 32-cores system mentioned above made a high-speed attack (~500 kpps) not noticeable.
There can be some impact on performance though, due to
ipfrag_high_thresh being set to 262144 bytes, as this way only two 64K fragments can fit in the reassembly queue at the same time. For example, there is a risk of breaking applications that rely on large UDP packets.
The following simple script can be used to quickly change to/from the default and lower settings:
#!/bin/sh if [ "x$1" == "xlow" ]; then echo Settinig limits low: sysctl -w net.ipv4.ipfrag_low_thresh=196608 sysctl -w net.ipv4.ipfrag_high_thresh=262144 sysctl -w net.ipv6.ip6frag_low_thresh=196608 sysctl -w net.ipv6.ip6frag_high_thresh=262144 echo elif [ "x$1" == "xdef" ]; then echo Settinig limits default: sysctl -w net.ipv4.ipfrag_high_thresh=4194304 sysctl -w net.ipv4.ipfrag_low_thresh=3145728 sysctl -w net.ipv6.ip6frag_high_thresh=4194304 sysctl -w net.ipv6.ip6frag_low_thresh=3145728 echo fi echo Current values: sysctl net.ipv4.ipfrag_low_thresh sysctl net.ipv4.ipfrag_high_thresh sysctl net.ipv6.ip6frag_low_thresh sysctl net.ipv6.ip6frag_high_thresh
Red Hat thanks Juha-Matti Tilli from Aalto University, Department of Communications and Networking and Nokia Bell Labs for reporting this vulnerability.
- Red Hat Enterprise Linux Atomic Host
- Red Hat Enterprise Linux for ARM 64
- Red Hat Enterprise Linux for IBM z Systems
- Red Hat Enterprise Linux for Power 9
- Red Hat Enterprise Linux for Real Time
- Red Hat Enterprise MRG Realtime
- Red Hat Virtualization
- Red Hat Enterprise Linux
- Article Type