Openstack cannot create more instances in a compute

Solution Verified - Updated -

Environment

  • Red Hat Openstack Platform 16.2, 17.1

Issue

Creation of more instances in a compute fail with update conflict related to CPU in nova compute log.

Resolution

  1. Edit the cpu parameters of templates to the original that was used to scale up the compute node. For example, using NovaComputeCpuDedicatedSetinstead of NovaVcpuPinSet.

  2. Run a overcloud deploy.

  3. Test the instance creation.

Root Cause

It's impossible to change the CPU core use (dedicated or shared) when there is instances actually using the resource.

If a compute where all cores were set as dedicated and some instances were created. Is possible to run a deploy setting all cores as shared, but Nova will not be able to update the resource.

Diagnostic Steps

  1. Confirm if there is enough resource in the compute to the instance.
(overcloud) [stack@director ~]$ openstack flavor show <vm_flavor>
(overcloud) [stack@director ~]$ openstack resource provider show --allocations <compute_node>
(overcloud) [stack@director ~]$ openstack resource provider inventory show <compute_node> VCPU
(overcloud) [stack@director ~]$ openstack resource provider inventory show <compute_node> PCPU

If a inventory command fail with Expecting value: line 1 column 1 (char 0), it's the initial configured but changed.

  1. Check the parameters related to CPU in nova configuration in compute node.
/var/lib/config-data/puppet-generated/nova_libvirt/etc/nova/nova.conf
...
vcpu_pin_set=5-60
cpu_shared_set=0-3
#cpu_dedicated_set=<None>
  1. Check nova compute log for the error below.
/var/log/containers/nova/nova-compute.log
...
Failed to update inventory to [{'MEMORY_MB': {'total': 2048, 'min_unit': 1, 'max_unit': 2048, 'step_size': 1, 'allocation_ratio': 1.0, 'reserved': 512}, 'VCPU': {'total': 56, 'min_unit': 1, 'max_unit': 56, 'step_size': 1, 'allocation_ratio': 1.0, 'reserved': 0}, 'DISK_GB': {'total': 9146, 'min_unit': 1, 'max_unit': 9146, 'step_size': 1, 'allocation_ratio': 1.0, 'reserved': 0}}] for resource provider with UUID 00000000-8deb-499d-0000-0ea3b1100000.  Got 409: {"errors": [{"status": 409, "title": "Conflict", "detail": "There was a conflict when trying to complete your request.\n\n update conflict: Inventory for 'PCPU' on resource provider '00000000-8deb-499d-0000-0ea3b1100000' in use.  ", "code": "placement.inventory.inuse", "request_id": "req-00000000-21e0-0000-ab22-8fedb0600000"}]}

This solution is part of Red Hat’s fast-track publication program, providing a huge library of solutions that Red Hat engineers have created while supporting our customers. To give you the knowledge you need the instant it becomes available, these articles may be presented in a raw and unedited form.

Comments