Impacted Products
Red Hat Product Security has rated this update as having a security impact of Important.
The following Red Hat product versions are impacted:
Red Hat Enterprise Linux 5
Red Hat Enterprise Linux 6
Red Hat Enterprise Linux 7
Red Hat Atomic Host
Red Hat Enterprise MRG 2
Red Hat OpenShift Online v2
Red Hat OpenShift Online v3
Red Hat Virtualization (RHEV-H/RHV-H)
Red Hat Enterprise Linux OpenStack Platform 6.0 (Juno)
Red Hat Enterprise Linux OpenStack Platform 7.0 (Kilo) for RHEL7
Red Hat Enterprise Linux OpenStack Platform 7.0 (Kilo) director for RHEL7
Red Hat OpenStack Platform 8.0 (Liberty)
Red Hat OpenStack Platform 8.0 (Liberty) director
Red Hat OpenStack Platform 9.0 (Mitaka)
Red Hat OpenStack Platform 9.0 (Mitaka) director
Red Hat OpenStack Platform 10.0 (Newton)
Red Hat OpenStack Platform 11.0 (Ocata)
Red Hat OpenStack Platform 12.0 (Pike)
While Red Hat's Linux Containers are not directly impacted by kernel issues, their security relies upon the integrity of the host kernel environment. Red Hat recommends that you use the most recent versions of your container images. The Container Health Index, part of the Red Hat Container Catalog, can always be used to verify the security status of Red Hat containers. To protect the privacy of the containers in use, you will need to ensure that the Container host (such as Red Hat Enterprise Linux or Atomic Host) has been updated against these attacks. Red Hat has released an updated Atomic Host for this use case.
Attack Description and Impact
The attacks described in this article abuse the speculative execution capability of modern high-performance microprocessors. Modern microprocessors implement a design optimization known as “Out-of-Order” (OoO) execution, meaning the microprocessor will begin to execute independent instructions as soon as their data dependencies become available (so-called “data flow” model) rather than always executing instructions in the literal order provided by the programmer through their application binary. The illusion of a sequential execution is maintained within the processor by means of various internal reordering structures that buffer these intermediate execution states of the processor and present the in-order results. Out-of-Order (OoO) Execution was originally invented by Robert Tomasulo in 1967 for use in the early IBM mainframe systems. In the intervening decades, it has become a standard feature of nearly all microprocessors.
An extension of the Out-of-Order execution model adds highly sophisticated branch prediction algorithms that aim to predict whether a given path (branch) of software code will be executed. A branch can be thought of as changing the flow of instructions being executed by the processor in response to an “if” statement of the form “if this, then do A, otherwise do B”. The condition upon which a branch is taken or not taken often depends upon data that may not immediately be available (for example, because it requires a load from slower RAM into the internal microprocessor cache hierarchy). Since the branch condition may not be ready in a timely manner, the processor may begin to speculatively execute the most likely path, based on input from the branch predictor. Results from this execution are stored in such a manner that the entire path can be discarded if the speculated branch direction later turns out to be incorrect. Thus, speculative execution is normally completely invisible to the programmer, or to other users of the same system.
The attacks described in this article rely upon breaking open the black box that is the internal state of the microprocessor during speculative execution. In particular, the attacks rely upon a technique known as cache side-channel analysis. During speculative execution, a processor will not intermediately make results available in memory or registers visible to the programmer, to other processors, or to other running applications. Yet, in order to access memory within speculated code paths, it must bring the data in the processor cache. Side-channel analysis allows an attacker to observe speculated allocations (loads) into the system caches, even those coming from execution paths that ultimately have been discarded. A specially crafted program can then be designed to speculatively perform loads into the cache from privileged memory locations, and monitor the results which can be used to infer the content of that privileged memory.
One case that triggers CPU speculative execution is branches. An attacker can start by training the branch predictor that a particular branch in kernel code is heavily taken (or not taken). The next time the branch executes, the processor will start executing the code in the direction chosen by the attacker. If the attacker chooses a path that loads a value from memory, such load will be executed speculatively. Attacks against branch prediction can (in some affected microprocessor implementations) be extended across the kernel/hypervisor boundary, allowing unprivileged guest Operating Systems to exert influence over the execution of the hypervisor and, when combined with side-channel analysis, to extract sensitive hypervisor memory.
The effects of speculative execution however can be even more wide-ranging. Because the internal state of the microprocessor is not visible to the programmer, or to other users or applications running on the system, the processor may perform speculative data accesses even before checking whether they are permitted. Permission checks will occur in parallel and ultimately trigger an abort of the speculation prior to retiring the speculated instructions and making their execution results visible outside the processor. If the processor speculatively uses cached data from memory prior to completing the permission checks, then it becomes possible to observe that data by using it in subsequent memory accesses.
One example of such permission checks is page access checks from the memory management unit (MMU). Paging, also known as virtual memory, is a common feature of high performance microprocessors; it lets the operating system control the mapping of virtual addresses into the physical addresses of the system RAM, and also limit accesses to the virtual addresses through access control bits. For example, a page can be marked as “read-only” (so that writes cause a page fault exception) or as “kernel memory” (so that user-mode accesses cause a page fault exception).
Because the processor’s permission checks enforce that user applications cannot access kernel memory, it is standard practice in the industry for operating system kernels (including Linux) to map kernel virtual memory addresses into the same address space as user applications. Doing so creates a significant performance advantage since applications make frequent use of kernel-provided system calls, and switching address spaces during each system call would incur a significant performance overhead. Each switch would require flushing (invalidating) the content of many internal CPU structures, such as TLBs (Translation Lookaside Buffers) that cache virtual-to-physical memory translations and accelerate the use of virtual memory.
Sharing the page tables between the kernel and user applications, however, enables another kind of attack against speculative execution. In this case, the preparatory phase has the attacker trick the kernel into loading an “interesting” virtual address into the processor’s Level 1 (L1) data cache. The L1 data cache is commonly organized using a technique known as VIPT (Virtually Indexed, Physically Tagged), which lets the virtual-to-physical address translation and permission checks occur in parallel with access. In the presence of a shared virtual address space, kernel privileged virtual addresses might be accessed speculatively through the L1 cache by untrusted user code during speculative execution, and the loaded values can be used further down the speculatively-executed path. A second speculative memory access can thus fill the cache in a manner that depends on privileged memory contents, and the effects can be observed to derive those contents.
These microprocessor side-channel attacks may allow an untrusted user with access to a machine to extract sensitive information from privileged kernel or hypervisor memory, as well as from other applications or virtual machines running on the same system. Mitigation involves a number of discrete steps, some or all of which may be required depending upon the exact make and model of the microprocessor, each of which may be vulnerable to a differing extent to each variant of the attack:
- Separating the kernel and user virtual address spaces: this is performed using a design change to the Operating System kernel known as KPTI (Kernel Page Table Isolation), sometimes referred to using the older name “KAISER”.
- Disabling indirect branch prediction upon entry into the kernel or into the hypervisor: New capabilities have been added to many microprocessors across the industry through microcode, millicode, firmware, and other updates. These new capabilities are leveraged by updates to Red Hat Enterprise Linux which control their use.
- Fencing speculative loads of certain memory locations: Such loads have to be annotated through small changes to the Linux kernel, which have been integrated into Red Hat updates.
These software solutions, in combination with microcode, millicode, and firmware updates can mitigate the attacks described in this article. However, the mitigation comes at some cost to system performance. Depending upon the specific system, make, and model of the microprocessors, as well as the characteristics of the workloads, the performance impact can be significant. Red Hat is taking a proactive position that favors security over performance, while allowing users the flexibility to assess their own environment and make appropriate tradeoffs through selectively enabling and disabling the various mitigations.
The Red Hat Performance Engineering team has created a Knowledgebase article reporting observed performance impact for a variety of representative workloads, and describing options users can take to disable parts of the security fixes to regain the desired level of performance if the customer is confident their computers are physically isolated. Additional performance data will be published as this incident develops.
The Red Hat Performance Engineering and Product Engineering teams have developed a Knowledgebase article detailing the tunables available to selectively enable or disable these new security features.
Additional guidance for subscribers using AMD-based systems can be found in this Knowledgebase article.
Comments