RHOS Mitigation for MDS ("Microarchitectural Data Sampling") Security Flaws

Solution Verified - Updated -

Environment

Red Hat OpenStack Platform
x86_64 architecture

Issue

Four new microprocessor flaws, known as MDS, have been discovered which will affect
Nova Compute nodes and instances running on Intel x86_64 CPUs.

Resolution

To get mitigation for the said MDS security flaws, a new CPU flag, md-clear, needs to be exposed to the Nova instances. It can be done as follows.

  1. Update the following components to the said versions, or newer, on all relevant Compute nodes:

    • microcode_ctl-2.1-47.2.el7_6
    • kernel-4.18.0-80.1.2.el8_0
    • qemu-kvm-rhev-2.12.0-18.el7_6.5
    • libvirt-4.5.0-10.el7_6.9

  2. Live migrate all the Nova instances to another Compute node.

  3. Ensure that the CPU flag md-clear is exposed to the Nova instance.
    It can be done so in one of the three following ways, given that OpenStack Nova supports three distinct CPU modes:

    • [libvirt]/cpu_mode = host-model

      When using host-model CPU mode, the md-clear CPU flag
      will be passed through to the Nova guests automatically.

      This mode is the default, when virt_type=kvm|qemu is
      set in /etc/nova/nova/conf

    • [libvirt]/cpu_mode = host-passthrough

      When using host-passthrough CPU mode, the md-clear CPU flag will be passed through to the Nova guests automatically.

    • A specific custom CPU model — this can be enabled using the
      Nova config attributes: [libvirt]/cpu_mode = custom plus a
      particular named CPU model, e.g. [libvirt]/cpu_model =
      IvyBridge

      (The list of all valid named CPU models that are supported by
      your host, QEMU and libvirt can be found out by running the
      command virsh domcapabilities.)
      When using a custom CPU mode, you must explicitly enable the
      CPU flag md-clear to the Nova instances, in addition to the
      flags required for previous vulnerabilities, using the
      cpu_model_extra_flags. E.g.

      [libvirt]
cpu_mode = custom
cpu_model = IvyBridge
cpu_model_extra_flags = spec-ctrl,ssbd,md-clear

  4. Reboot the Compute node for the fixes to take effect.

Once the above steps have been taken on every vulnerable compute
node in the deployment, each running guest in the cluster must be
fully powered down, and cold-booted (i.e. an explicit stop followed
by a start), in order to activate the new CPU model. This can be done
by the guest administrators at a time of their choosing.

Performance Impact

Refer to the section titled "Performance Impact and Disabling MDS" in the main security notice, under the 'Resolve' tab: https://access.redhat.com/security/vulnerabilities/mds

Root Cause

Four new microprocessor flaws have been discovered, the most severe of which is rated by Red Hat Product Security as having an Important impact. These flaws, if exploited by an attacker with local shell access to a system, could allow data in the CPU’s cache to be exposed to unauthorized processes. While difficult to execute, a skilled attacker could use these flaws to read memory from a virtual or containerized instance, or the underlying host system. Red Hat has mitigations prepared for affected systems and has detailed steps customers should take as they evaluate their exposure risk and formulate their response.

Diagnostic Steps

Validate that the fixes are in effect

After applying relevant updates, administrators can check to ensure the
patches are in effect by running either of the following (on the host):

# dmesg | grep "MDS:"
[    0.162571] MDS: Vulnerable: Clear CPU buffers attempted, no microcode
[  181.862076] MDS: Mitigation: Clear CPU buffers

# cat /sys/devices/system/cpu/vulnerabilities/mds
Mitigation: Clear CPU buffers; SMT vulnerable

On the host, validate that KVM is capable of exposing the md-clear
flag to guests:

# virsh domcapabilities kvm | grep md-clear
    <feature policy='require' name='md-clear'/>

Also, refer to the 'Diagnosis' tab in the main page:
MDS.

This solution is part of Red Hat’s fast-track publication program, providing a huge library of solutions that Red Hat engineers have created while supporting our customers. To give you the knowledge you need the instant it becomes available, these articles may be presented in a raw and unedited form.

Comments