Red Hat Training

A Red Hat training course is available for Red Hat OpenStack Platform

1.3. Configuring resource isolation on hyperconverged nodes

Colocating Ceph OSD and Compute services on hyperconverged nodes risks resource contention between Red Hat Ceph Storage and Compute services, as neither are aware of each other’s presence on the same host. Resource contention can result in degradation of service, which offsets the benefits of hyperconvergence.

You must configure resource isolation for both Ceph and Compute services to prevent contention.

Procedure

  1. Optional: Override the autogenerated Compute settings by adding the following parameters to a Compute environment file:

    parameter_defaults:
      ComputeHCIParameters:
        NovaReservedHostMemory: <ram>
        NovaCPUAllocationRatio: <ratio>
  2. To reserve memory resources for Red Hat Ceph Storage, set the parameter is_hci to true in /home/stack/templates/storage-container-config.yaml:

    parameter_defaults:
      CephAnsibleExtraConfig:
        is_hci: true

    This allows ceph-ansible to reserve memory resources for Red Hat Ceph Storage, and reduce memory growth by Ceph OSDs, by automatically adjusting the osd_memory_target parameter setting for a HCI deployment.

    警告

    Red Hat does not recommend directly overriding the ceph_osd_docker_memory_limit parameter.

    注記

    As of ceph-ansible 3.2, the ceph_osd_docker_memory_limit is set automatically to the maximum memory of the host, as discovered by Ansible, regardless of whether the FileStore or BlueStore back end is used.

  3. Optional: By default, ceph-ansible reserves one vCPU for each Ceph OSD. If you require more than one CPU per Ceph OSD, add the following configuration to /home/stack/templates/storage-container-config.yaml:

    parameter_defaults:
      CephAnsibleExtraConfig:
        ceph_osd_docker_cpu_limit: <cpu_limit>

    Replace <cpu_limit> with the number of CPUs to reserve for each Ceph OSD.

    For more information on how to tune CPU resources based on your hardware and workload, see Red Hat Ceph Storage Hardware Selection Guide.

  4. Optional: Reduce the priority of Red Hat Ceph Storage backfill and recovery operations when a Ceph OSD is removed by adding the following parameters to a Ceph environment file:

    parameter_defaults:
      CephConfigOverrides:
        osd_recovery_op_priority: <priority_value>
        osd_recovery_max_active: <no_active_recovery_requests>
        osd_max_backfills: <max_no_backfills>
    • Replace <priority_value> with the priority for recovery operations, relative to the OSD client OP priority.
    • Replace <no_active_recovery_requests> with the number of active recovery requests per OSD, at one time.
    • Replace <max_no_backfills> with the maximum number of backfills allowed to or from a single OSD.

      For more information on default Red Hat Ceph Storage backfill and recovery options, see Red Hat Ceph Storage backfill and recovery operations.

1.3.1. Process for autogenerating CPU and memory resources to reserve for the Compute service

Director provides a default plan environment file for configuring resource constraints on hyperconverged nodes during deployment. This plan environment file instructs the OpenStack Workflow to complete the following processes:

  1. Retrieve the hardware introspection data collected during inspection of the hardware nodes.
  2. Calculate optimal CPU and memory allocation workload for Compute on hyperconverged nodes based on that data.
  3. Autogenerate the parameters required to configure those constraints and reserve CPU and memory resources for Compute. These parameters are defined under the hci_profile_config section of the plan-environment-derived-params.yaml file.
注記

The average_guest_memory_size_in_mb and average_guest_cpu_utilization_percentage parameters in each workload profile are used to calculate values for the reserved_host_memory and cpu_allocation_ratio settings of Compute.

You can override the autogenerated Compute settings by adding the following parameters to your Compute environment file:

Autogenerated nova.conf parameterCompute environment file overrideDescription

reserved_host_memory

parameter_defaults:
  ComputeHCIParameters:
    NovaReservedHostMemory: 181000

Sets how much RAM should be reserved for the Ceph OSD services and per-guest instance overhead on hyperconverged nodes.

cpu_allocation_ratio

parameter_defaults:
  ComputeHCIParameters:
    NovaCPUAllocationRatio: 8.2

Sets the ratio that the Compute scheduler should use when choosing which Compute node to deploy an instance on.

These overrides are applied to all nodes that use the ComputeHCI role, namely, all hyperconverged nodes. For more information about manually determining optimal values for NovaReservedHostMemory and NovaCPUAllocationRatio, see OpenStack Workflow Compute CPU and memory calculator.

ヒント

You can use the following script to calculate suitable baseline NovaReservedHostMemory and NovaCPUAllocationRatio values for your hyperconverged nodes.

nova_mem_cpu_calc.py