Why does instance creation fail on a large Openstack environment with "VirtualInterfaceCreateException: Virtual Interface creation failed" ?

Solution In Progress - Updated -

Issue

  • After launching large number of instances , all further instance created fails with the following error .
2014-10-31 12:30:04.892 36156 INFO nova.compute.manager [-] [instance: ace2548b-wr2w-7543-0923-qxe506g33756] During sync_power_state the instance has a pending task. Skip.
2014-10-31 12:30:20.943 36156 AUDIT nova.compute.resource_tracker [-] Auditing locally available compute resources
2014-10-31 12:30:22.511 36156 AUDIT nova.compute.resource_tracker [-] Free ram (MB): 47324
2014-10-31 12:30:22.511 36156 AUDIT nova.compute.resource_tracker [-] Free disk (GB): 430
2014-10-31 12:30:22.512 36156 AUDIT nova.compute.resource_tracker [-] Free VCPUS: -24
2014-10-31 12:30:22.529 36156 INFO nova.compute.resource_tracker [-] Compute_service record updated for :
2014-10-31 12:30:27.065 36156 WARNING nova.virt.libvirt.driver [req-2c4d46c5-28b6-4403-b637-9194db2156a8 d6160437f4b8489e8f80cc2ed855d257 3127399184f24cfbadcc0c2660350bee] Timeout waiting for vif plugging callback for instance ace2548b-wr2w-7543-0923-qxe506g33756
2014-10-31 12:30:27.278 36156 INFO nova.compute.manager [-] Lifecycle event 1 on VM ace2548b-wr2w-7543-0923-qxe506g33756
2014-10-31 12:30:27.372 36156 INFO nova.compute.manager [-] [instance: ace2548b-wr2w-7543-0923-qxe506g33756] During sync_power_state the instance has a pending task. Skip.
2014-10-31 12:30:27.731 36156 INFO nova.virt.libvirt.driver [req-2c4d46c5-28b6-4403-b637-9194db2156a8 d6160437f4b8489e8f80cc2ed855d257 3127399184f24cfbadcc0c2660350bee] [instance: ace2548b-wr2w-7543-0923-qxe506g33756] Deleting instance files /var/lib/nova/instances/ace2548b-wr2w-7543-0923-qxe506g33756
2014-10-31 12:30:27.731 36156 INFO nova.virt.libvirt.driver [req-2c4d46c5-28b6-4403-b637-9194db2156a8 d6160437f4b8489e8f80cc2ed855d257 3127399184f24cfbadcc0c2660350bee] [instance: ace2548b-wr2w-7543-0923-qxe506g33756] Deletion of /var/lib/nova/instances/ace2548b-wr2w-7543-0923-qxe506g33756 complete
2014-10-31 12:30:27.791 36156 ERROR nova.compute.manager [req-2c4d46c5-28b6-4403-b637-9194db2156a8 d6160437f4b8489e8f80cc2ed855d257 3127399184f24cfbadcc0c2660350bee] [instance: ace2548b-wr2w-7543-0923-qxe506g33756] Instance failed to spawn
2014-10-31 12:30:27.791 36156 TRACE nova.compute.manager [instance: ace2548b-wr2w-7543-0923-qxe506g33756] Traceback (most recent call last):
2014-10-31 12:30:27.791 36156 TRACE nova.compute.manager [instance: ace2548b-wr2w-7543-0923-qxe506g33756]   File "/usr/lib/python2.7/site-packages/nova/compute/", line 1714, in _spawn
2014-10-31 12:30:27.791 36156 TRACE nova.compute.manager [instance: ace2548b-wr2w-7543-0923-qxe506g33756]     block_device_info)
2014-10-31 12:30:27.791 36156 TRACE nova.compute.manager [instance: ace2548b-wr2w-7543-0923-qxe506g33756]   File "/usr/lib/python2.7/site-packages/nova/virt/libvirt/", line 2289, in spawn
2014-10-31 12:30:27.791 36156 TRACE nova.compute.manager [instance: ace2548b-wr2w-7543-0923-qxe506g33756]     block_device_info)
2014-10-31 12:30:27.791 36156 TRACE nova.compute.manager [instance: ace2548b-wr2w-7543-0923-qxe506g33756]   File "/usr/lib/python2.7/site-packages/nova/virt/libvirt/", line 3701, in _create_domain_and_network
2014-10-31 12:30:27.791 36156 TRACE nova.compute.manager [instance: ace2548b-wr2w-7543-0923-qxe506g33756]     raise exception.VirtualInterfaceCreateException()
2014-10-31 12:30:27.791 36156 TRACE nova.compute.manager [instance: ace2548b-wr2w-7543-0923-qxe506g33756] VirtualInterfaceCreateException: Virtual Interface creation failed
  • After deploying large number of instance for a specific project, instance creation fails with VirtualInterfaceCreateException: Virtual Interface creation failed .

Environment

  • Red Hat OpenStack 5.0 on RHEL 7

Subscriber exclusive content

A Red Hat subscription provides unlimited access to our knowledgebase of over 48,000 articles and solutions.

Current Customers and Partners

Log in for full access

Log In