Menu Close
Settings Close

Language and Page Formatting Options

Chapter 12. Configuring AMD SEV Compute nodes to provide memory encryption for instances

As a cloud administrator, you can provide cloud users the ability to create instances that run on SEV-capable Compute nodes with memory encryption enabled.

This feature is available to use from the 2nd Gen AMD EPYC™ 7002 Series ("Rome").

To enable your cloud users to create instances that use memory encryption, you must perform the following tasks:

  1. Designate the AMD SEV Compute nodes for memory encryption.
  2. Configure the Compute nodes for memory encryption.
  3. Deploy the overcloud.
  4. Create a flavor or image for launching instances with memory encryption.
Tip

If the AMD SEV hardware is limited, you can also configure a host aggregate to optimize scheduling on the AMD SEV Compute nodes. To schedule only instances that request memory encryption on the AMD SEV Compute nodes, create a host aggregate of the Compute nodes that have the AMD SEV hardware, and configure the Compute scheduler to place only instances that request memory encryption on the host aggregate. For more information, see Creating and managing host aggregates and Filtering by isolating host aggregates.

12.1. Secure Encrypted Virtualization (SEV)

Secure Encrypted Virtualization (SEV), provided by AMD, protects the data in DRAM that a running virtual machine instance is using. SEV encrypts the memory of each instance with a unique key.

SEV increases security when you use non-volatile memory technology (NVDIMM), because an NVDIMM chip can be physically removed from a system with the data intact, similar to a hard drive. Without encryption, any stored information such as sensitive data, passwords, or secret keys can be compromised.

For more information, see the AMD Secure Encrypted Virtualization (SEV) documentation.

Limitations of instances with memory encryption

  • You cannot live migrate, or suspend and resume instances with memory encryption.
  • You cannot use PCI passthrough to directly access devices on instances with memory encryption.
  • You cannot use virtio-blk as the boot disk of instances with memory encryption with Red Hat Enterprise Linux (RHEL) kernels earlier than kernel-4.18.0-115.el8 (RHEL-8.1.0).

    Note

    You can use virtio-scsi or SATA as the boot disk, or virtio-blk for non-boot disks.

  • The operating system that runs in an encrypted instance must provide SEV support. For more information, see the Red Hat Knowledgebase solution Enabling AMD Secure Encrypted Virtualization in RHEL 8.
  • Machines that support SEV have a limited number of slots in their memory controller for storing encryption keys. Each running instance with encrypted memory consumes one of these slots. Therefore, the number of instances with memory encryption that can run concurrently is limited to the number of slots in the memory controller. For example, on 1st Gen AMD EPYC™ 7001 Series ("Naples") the limit is 16, and on 2nd Gen AMD EPYC™ 7002 Series ("Rome") the limit is 255.
  • Instances with memory encryption pin pages in RAM. The Compute service cannot swap these pages, therefore you cannot overcommit memory on a Compute node that hosts instances with memory encryption.
  • You cannot use memory encryption with instances that have multiple NUMA nodes.

12.2. Designating AMD SEV Compute nodes for memory encryption

To designate AMD SEV Compute nodes for instances that use memory encryption, you must create a new role file to configure the AMD SEV role, and configure a new overcloud flavor and AMD SEV resource class to use to tag the Compute nodes for memory encryption.

Procedure

  1. Log in to the undercloud as the stack user.
  2. Source the stackrc file:

    [stack@director ~]$ source ~/stackrc
  3. Generate a new roles data file that includes the ComputeAMDSEV role, along with any other roles that you need for the overcloud. The following example generates the roles data file roles_data_amd_sev.yaml, which includes the roles Controller and ComputeAMDSEV:

    (undercloud)$ openstack overcloud roles \
     generate -o /home/stack/templates/roles_data_amd_sev.yaml \
     Compute:ComputeAMDSEV Controller
  4. Open roles_data_amd_sev.yaml and edit or add the following parameters and sections:

    Section/ParameterCurrent valueNew value

    Role comment

    Role: Compute

    Role: ComputeAMDSEV

    Role name

    name: Compute

    name: ComputeAMDSEV

    description

    Basic Compute Node role

    AMD SEV Compute Node role

    HostnameFormatDefault

    %stackname%-novacompute-%index%

    %stackname%-novacomputeamdsev-%index%

    deprecated_nic_config_name

    compute.yaml

    compute-amd-sev.yaml

  5. Register the AMD SEV Compute nodes for the overcloud by adding them to your node definition template, node.json or node.yaml. For more information, see Registering nodes for the overcloud in the Director Installation and Usage guide.
  6. Inspect the node hardware:

    (undercloud)$ openstack overcloud node introspect \
     --all-manageable --provide

    For more information, see Creating an inventory of the bare-metal node hardware in the Director Installation and Usage guide.

  7. Create the compute-amd-sev overcloud flavor for AMD SEV Compute nodes:

    (undercloud)$ openstack flavor create --id auto \
     --ram <ram_size_mb> --disk <disk_size_gb> \
     --vcpus <no_vcpus> compute-amd-sev
    • Replace <ram_size_mb> with the RAM of the bare metal node, in MB.
    • Replace <disk_size_gb> with the size of the disk on the bare metal node, in GB.
    • Replace <no_vcpus> with the number of CPUs on the bare metal node.

      Note

      These properties are not used for scheduling instances. However, the Compute scheduler does use the disk size to determine the root partition size.

  8. Retrieve a list of your nodes to identify their UUIDs:

    (undercloud)$ openstack baremetal node list
  9. Tag each bare metal node that you want to designate for memory encryption with a custom AMD SEV resource class:

    (undercloud)$ openstack baremetal node set \
     --resource-class baremetal.AMD-SEV <node>

    Replace <node> with the ID of the bare metal node.

  10. Associate the compute-amd-sev flavor with the custom AMD SEV resource class:

    (undercloud)$ openstack flavor set \
     --property resources:CUSTOM_BAREMETAL_AMD_SEV=1 \
      compute-amd-sev

    To determine the name of a custom resource class that corresponds to a resource class of a Bare Metal service node, convert the resource class to uppercase, replace each punctuation mark with an underscore, and prefix with CUSTOM_.

    Note

    A flavor can request only one instance of a bare metal resource class.

  11. Set the following flavor properties to prevent the Compute scheduler from using the bare metal flavor properties to schedule instances:

    (undercloud)$ openstack flavor set \
     --property resources:VCPU=0 --property resources:MEMORY_MB=0 \
     --property resources:DISK_GB=0 compute-amd-sev
  12. Optional: If the network topology of the ComputeAMDSEV role is different from the network topology of your Compute role, then create a custom network interface template. For more information, see Custom network interface templates in the Advanced Overcloud Customization guide.

    If the network topology of the ComputeAMDSEV role is the same as the Compute role, then you can use the default network topology defined in compute.yaml.

  13. Register the Net::SoftwareConfig of the ComputeAMDSEV role in your network-environment.yaml file:

    resource_registry:
      OS::TripleO::Compute::Net::SoftwareConfig: /home/stack/templates/nic-configs/compute.yaml
      OS::TripleO::ComputeCPUPinning::Net::SoftwareConfig: /home/stack/templates/nic-configs/<amd_sev_net_top>.yaml
      OS::TripleO::Controller::Net::SoftwareConfig: /home/stack/templates/nic-configs/controller.yaml

    Replace <amd_sev_net_top> with the name of the file that contains the network topology of the ComputeAMDSEV role, for example, compute.yaml to use the default network topology.

  14. Add the following parameters to the node-info.yaml file to specify the number of AMD SEV Compute nodes, and the flavor that you want to use for the AMD SEV designated Compute nodes:

    parameter_defaults:
      OvercloudComputeAMDSEVFlavor: compute-amd-sev
      ComputeAMDSEVCount: 3
  15. To verify that the role was created, enter the following command:

    (undercloud)$ openstack baremetal node list --long -c "UUID" \
     -c "Instance UUID" -c "Resource Class" -c "Provisioning State" \
     -c "Power State" -c "Last Error" -c "Fault" -c "Name" -f json

    Example output:

    [
      {
        "Fault": null,
        "Instance UUID": "e8e60d37-d7c7-4210-acf7-f04b245582ea",
        "Last Error": null,
        "Name": "compute-0",
        "Power State": "power on",
        "Provisioning State": "active",
        "Resource Class": "baremetal.AMD-SEV",
        "UUID": "b5a9ac58-63a7-49ba-b4ad-33d84000ccb4"
      },
      {
        "Fault": null,
        "Instance UUID": "3ec34c0b-c4f5-4535-9bd3-8a1649d2e1bd",
        "Last Error": null,
        "Name": "compute-1",
        "Power State": "power on",
        "Provisioning State": "active",
        "Resource Class": "compute",
        "UUID": "432e7f86-8da2-44a6-9b14-dfacdf611366"
      },
      {
        "Fault": null,
        "Instance UUID": "4992c2da-adde-41b3-bef1-3a5b8e356fc0",
        "Last Error": null,
        "Name": "controller-0",
        "Power State": "power on",
        "Provisioning State": "active",
        "Resource Class": "controller",
        "UUID": "474c2fc8-b884-4377-b6d7-781082a3a9c0"
      }
    ]

12.3. Configuring AMD SEV Compute nodes for memory encryption

To enable your cloud users to create instances that use memory encryption, you must configure the Compute nodes that have the AMD SEV hardware.

Prerequisites

  • Your deployment must include a Compute node that runs on AMD hardware capable of supporting SEV, such as an AMD EPYC CPU. You can use the following command to determine if your deployment is SEV-capable:

    $ lscpu | grep sev

Procedure

  1. Open your Compute environment file.
  2. Optional: Add the following configuration to your Compute environment file to specify the maximum number of memory-encrypted instances that the AMD SEV Compute nodes can host concurrently:

    parameter_defaults:
      ComputeAMDSEVExtraConfig:
        nova::config::nova_config:
          libvirt/num_memory_encrypted_guests:
            value: 15
    Note

    The default value of the libvirt/num_memory_encrypted_guests parameter is none. If you do not set a custom value, the AMD SEV Compute nodes do not impose a limit on the number of memory-encrypted instances that the nodes can host concurrently. Instead, the hardware determines the maximum number of memory-encrypted instances that the AMD SEV Compute nodes can host concurrently, which might cause some memory-encrypted instances to fail to launch.

  3. Optional: To specify that all x86_64 images use the q35 machine type by default, add the following configuration to your Compute environment file:

    parameter_defaults:
      ComputeAMDSEVParameters:
        NovaHWMachineType: x86_64=q35

    If you specify this parameter value, you do not need to set the hw_machine_type property to q35 on every AMD SEV instance image.

  4. To ensure that the AMD SEV Compute nodes reserve enough memory for host-level services to function, add 16MB for each potential AMD SEV instance:

    parameter_defaults:
      ComputeAMDSEVParameters:
        ...
        NovaReservedHostMemory: <libvirt/num_memory_encrypted_guests * 16>
  5. Configure the kernel parameters for the AMD SEV Compute nodes:

    parameter_defaults:
      ComputeAMDSEVParameters:
        ...
        KernelArgs: "hugepagesz=1GB hugepages=32 default_hugepagesz=1GB mem_encrypt=on kvm_amd.sev=1"
    Note

    When you first add the KernelArgs parameter to the configuration of a role, the overcloud nodes are automatically rebooted. If required, you can disable the automatic rebooting of nodes and instead perform node reboots manually after each overcloud deployment. For more information, see Configuring manual node reboot to define KernelArgs.

  6. Save the updates to your Compute environment file.
  7. Add your Compute environment file to the stack with your other environment files and deploy the overcloud:

    (undercloud)$ openstack overcloud deploy --templates \
     -e [your environment files] \
     -e /home/stack/templates/<compute_environment_file>.yaml

12.4. Creating an image for memory encryption

When the overcloud contains AMD SEV Compute nodes, you can create an AMD SEV instance image that your cloud users can use to launch instances that have memory encryption.

Procedure

  1. Create a new image for memory encryption:

    (overcloud)$ openstack image create ...  \
     --property hw_firmware_type=uefi amd-sev-image
    Note

    If you use an existing image, the image must have the hw_firmware_type property set to uefi.

  2. Optional: Add the property hw_mem_encryption=True to the image to enable AMD SEV memory encryption on the image:

    (overcloud)$ openstack image set  \
     --property hw_mem_encryption=True amd-sev-image
    Tip

    You can enable memory encryption on the flavor. For more information, see Creating a flavor for memory encryption.

  3. Optional: Set the machine type to q35, if not already set in the Compute node configuration:

    (overcloud)$ openstack image set  \
     --property hw_machine_type=q35 amd-sev-image
  4. Optional: To schedule memory-encrypted instances on a SEV-capable host aggregate, add the following trait to the image extra specs:

    (overcloud)$ openstack image set  \
     --property trait:HW_CPU_X86_AMD_SEV=required amd-sev-image
    Tip

    You can also specify this trait on the flavor. For more information, see Creating a flavor for memory encryption.

12.5. Creating a flavor for memory encryption

When the overcloud contains AMD SEV Compute nodes, you can create one or more AMD SEV flavors that your cloud users can use to launch instances that have memory encryption.

Note

An AMD SEV flavor is necessary only when the hw_mem_encryption property is not set on an image.

Procedure

  1. Create a flavor for memory encryption:

    (overcloud)$ openstack flavor create --vcpus 1 --ram 512 --disk 2  \
     --property hw:mem_encryption=True m1.small-amd-sev
  2. To schedule memory-encrypted instances on a SEV-capable host aggregate, add the following trait to the flavor extra specs:

    (overcloud)$ openstack flavor set  \
     --property trait:HW_CPU_X86_AMD_SEV=required m1.small-amd-sev

12.6. Launching an instance with memory encryption

To verify that you can launch instances on an AMD SEV Compute node with memory encryption enabled, use a memory encryption flavor or image to create an instance.

Procedure

  1. Create an instance by using an AMD SEV flavor or image. The following example creates an instance by using the flavor created in Creating a flavor for memory encryption and the image created in Creating an image for memory encryption:

    (overcloud)$ openstack server create --flavor m1.small-amd-sev \
     --image amd-sev-image amd-sev-instance
  2. Log in to the instance as a cloud user.
  3. To verify that the instance uses memory encryption, enter the following command from the instance:

    $ dmesg | grep -i sev
    AMD Secure Encrypted Virtualization (SEV) active