Unable to create instances after a new deployment of RHOSP16

Solution Verified - Updated -

Environment

  • Red Hat OpenStack Platform 16.0
  • Red Hat OpenStack Platform 16.1
  • Red Hat OpenStack Platform 16.2

Issue

  • Unable to spawn instances as total number of available vCPU's is zero.
  • Nova lists the total number of vCPU's available on compute node as zero.
  • Introduction of NovaComputeCpuDedicatedSet and NovaComputeCpuSharedSet.

Resolution

  • Starting RHOSP16, Red Hat OpenStack Platform now supports hosting mixed workload on the same compute node. This enables nova to host both CPU pinned and Unpinned instances on the same compute host. [1]

  • In order to achieve this, the parameters mentioned below have been introduced to be specified in nova.conf at /var/lib/config-data/puppet-generated/nova_libvirt/etc/nova/.

  • These parameters can be passed through an environment file during deployment phase or even during stack update.

    NovaComputeCpuDedicatedSet:  “0-1,16-17”
NovaComputeCpuSharedSet:     “2-15,18-31”

  • Under nova.conf on the compute nodes the values of CPU's mentioned above is specified in parameter cpu_dedicated_set and cpu_shared_set.

  • Important: It is necessary to define both the CpuSet options in order to run mixed workload on the compute node. If only NovaComputeCpuDedicatedSet is defined, instances with pinned CPUs' can only be created on the specified compute node.

[1] - The official documentation for RHOSP16 covers the scope and use of the mentioned parameters.

Root Cause

  • Only the dedicated cpu set was defined using NovaComputeCpuDedicatedSet through templates which accounts for pinned CPUs on the compute node.
  • Subsequently output of nova hypervisor-show <uuid> returns the total number of available vCPUs to be zero since all of the CPUs are allocated for running pinned workload, also known as PCPU's.

  • Nova cli presently does not lists the total number of available PCPU's, although it can retrieved using placement API.

  • Note: The NovaVcpuPinSet parameter is now deprecated, in favour of NovaComputeCpuDedicatedSet for pinned and NovaComputeCpuSharedSet for the normal workload. NovaVcpuPinSet can still be used for specifying pinned CPUs' but not in conjunction with NovaComputeCpuDedicatedSet and NovaComputeCpuSharedSet.

Diagnostic Steps

  • Instance creation fails with error No valid host was found or Failed to synchronize the placement service with resource provider information supplied by the compute host.
  • Retrieving the list of resources available on hypervisor, shows the value of available vCPUs' as zero.

  • openstack hypervisor stats show returns:

    +----------------------+--------+
| Field                | Value  |
+----------------------+--------+
| count                | 4      |
| current_workload     | 0      |
| disk_available_least | 1452   |
| free_disk_gb         | 1488   |
| free_ram_mb          | 480836 |
| local_gb             | 1488   |
| local_gb_used        | 0      |
| memory_mb            | 513604 |
| memory_mb_used       | 32768  |
| running_vms          | 0      |
| vcpus                | 0      |    <<----
| vcpus_used           | 0      |    <<----
+----------------------+--------+

    Note: as of OSP 16.0 hypervisor statistics has been deprecated: Bug 1696739 - Deprecate '/os-hypervisors/statistics' API . Instead refer to Work with the placement API in Red Hat OpenStack Platform.

  • If the instance flavor include the property hw:cpu_policy='dedicated' the command below will have error.

    (overcloud) [stack@director ~]$ openstack resource provider inventory show <compute node id> PCPU
Expecting value: line 1 column 1 (char 0)

This solution is part of Red Hat’s fast-track publication program, providing a huge library of solutions that Red Hat engineers have created while supporting our customers. To give you the knowledge you need the instant it becomes available, these articles may be presented in a raw and unedited form.

Comments