Instances fail to spawn on a Compute node

Solution In Progress - Updated -

Issue

  • It looks like for some reason the Compute node tries to connect to Glance using its own IP address.
  • Instance fail to launch
2015-11-30 16:36:36.821 75614 ERROR nova.image.glance [req-ee0be8a3-f981-4ad3-b466-72921c4b24b3 09d542b5311340eb87138e00b5bb13f1 da9c5226a3e34126ae77117f54234507 - - -] Error contacting glance server '10.225.21.29:9292' for 'data', done trying.
2015-11-30 16:36:36.821 75614 TRACE nova.image.glance Traceback (most recent call last):
2015-11-30 16:36:36.821 75614 TRACE nova.image.glance   File "/usr/lib/python2.7/site-packages/nova/image/glance.py", line 219, in call
2015-11-30 16:36:36.821 75614 TRACE nova.image.glance     return getattr(client.images, method)(*args, **kwargs)
2015-11-30 16:36:36.821 75614 TRACE nova.image.glance   File "/usr/lib/python2.7/site-packages/glanceclient/v1/images.py", line 143, in data
2015-11-30 16:36:36.821 75614 TRACE nova.image.glance     % urlparse.quote(str(image_id)))
2015-11-30 16:36:36.821 75614 TRACE nova.image.glance   File "/usr/lib/python2.7/site-packages/glanceclient/common/http.py", line 262, in get
2015-11-30 16:36:36.821 75614 TRACE nova.image.glance     return self._request('GET', url, **kwargs)
2015-11-30 16:36:36.821 75614 TRACE nova.image.glance   File "/usr/lib/python2.7/site-packages/glanceclient/common/http.py", line 217, in _request
2015-11-30 16:36:36.821 75614 TRACE nova.image.glance     raise exc.CommunicationError(message=message)
2015-11-30 16:36:36.821 75614 TRACE nova.image.glance CommunicationError: Error finding address for http://10.225.21.29:9292/v1/images/098eb7e9-b2e4-40ad-9462-6ec96932977e: ('Connection aborted.', error(111, 'ECONNREFUSED'))
2015-11-30 16:36:36.821 75614 TRACE nova.image.glance 
2015-11-30 16:36:36.821 75614 ERROR nova.compute.manager [req-ee0be8a3-f981-4ad3-b466-72921c4b24b3 09d542b5311340eb87138e00b5bb13f1 da9c5226a3e34126ae77117f54234507 - - -] [instance: 11a0f94f-c290-41d1-b46e-564e0243632e] Instance failed to spawn
2015-11-30 16:36:36.821 75614 TRACE nova.compute.manager [instance: 11a0f94f-c290-41d1-b46e-564e0243632e] Traceback (most recent call last):
2015-11-30 16:36:36.821 75614 TRACE nova.compute.manager [instance: 11a0f94f-c290-41d1-b46e-564e0243632e]   File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 2480, in _build_resources
2015-11-30 16:36:36.821 75614 TRACE nova.compute.manager [instance: 11a0f94f-c290-41d1-b46e-564e0243632e]     yield resources
2015-11-30 16:36:36.821 75614 TRACE nova.compute.manager [instance: 11a0f94f-c290-41d1-b46e-564e0243632e]   File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 2352, in _build_and_run_instance
2015-11-30 16:36:36.821 75614 TRACE nova.compute.manager [instance: 11a0f94f-c290-41d1-b46e-564e0243632e]     block_device_info=block_device_info)
2015-11-30 16:36:36.821 75614 TRACE nova.compute.manager [instance: 11a0f94f-c290-41d1-b46e-564e0243632e]   File "/usr/lib/python2.7/site-packages/nova/virt/libvirt/driver.py", line 2456, in spawn
2015-11-30 16:36:36.821 75614 TRACE nova.compute.manager [instance: 11a0f94f-c290-41d1-b46e-564e0243632e]     admin_pass=admin_password)
2015-11-30 16:36:36.821 75614 TRACE nova.compute.manager [instance: 11a0f94f-c290-41d1-b46e-564e0243632e]   File "/usr/lib/python2.7/site-packages/nova/virt/libvirt/driver.py", line 2864, in _create_image
2015-11-30 16:36:36.821 75614 TRACE nova.compute.manager [instance: 11a0f94f-c290-41d1-b46e-564e0243632e]     instance, size, fallback_from_host)
2015-11-30 16:36:36.821 75614 TRACE nova.compute.manager [instance: 11a0f94f-c290-41d1-b46e-564e0243632e]   File "/usr/lib/python2.7/site-packages/nova/virt/libvirt/driver.py", line 6040, in _try_fetch_image_cache
2015-11-30 16:36:36.821 75614 TRACE nova.compute.manager [instance: 11a0f94f-c290-41d1-b46e-564e0243632e]     size=size)
2015-11-30 16:36:36.821 75614 TRACE nova.compute.manager [instance: 11a0f94f-c290-41d1-b46e-564e0243632e]   File "/usr/lib/python2.7/site-packages/nova/virt/libvirt/imagebackend.py", line 231, in cache
2015-11-30 16:36:36.821 75614 TRACE nova.compute.manager [instance: 11a0f94f-c290-41d1-b46e-564e0243632e]     *args, **kwargs)
2015-11-30 16:36:36.821 75614 TRACE nova.compute.manager [instance: 11a0f94f-c290-41d1-b46e-564e0243632e]   File "/usr/lib/python2.7/site-packages/nova/virt/libvirt/imagebackend.py", line 507, in create_image
2015-11-30 16:36:36.821 75614 TRACE nova.compute.manager [instance: 11a0f94f-c290-41d1-b46e-564e0243632e]     prepare_template(target=base, max_size=size, *args, **kwargs)
2015-11-30 16:36:36.821 75614 TRACE nova.compute.manager [instance: 11a0f94f-c290-41d1-b46e-564e0243632e]   File "/usr/lib/python2.7/site-packages/oslo_concurrency/lockutils.py", line 445, in inner
2015-11-30 16:36:36.821 75614 TRACE nova.compute.manager [instance: 11a0f94f-c290-41d1-b46e-564e0243632e]     return f(*args, **kwargs)
2015-11-30 16:36:36.821 75614 TRACE nova.compute.manager [instance: 11a0f94f-c290-41d1-b46e-564e0243632e]   File "/usr/lib/python2.7/site-packages/nova/virt/libvirt/imagebackend.py", line 221, in fetch_func_sync
2015-11-30 16:36:36.821 75614 TRACE nova.compute.manager [instance: 11a0f94f-c290-41d1-b46e-564e0243632e]     fetch_func(target=target, *args, **kwargs)
2015-11-30 16:36:36.821 75614 TRACE nova.compute.manager [instance: 11a0f94f-c290-41d1-b46e-564e0243632e]   File "/usr/lib/python2.7/site-packages/nova/virt/libvirt/utils.py", line 507, in fetch_image
2015-11-30 16:36:36.821 75614 TRACE nova.compute.manager [instance: 11a0f94f-c290-41d1-b46e-564e0243632e]     max_size=max_size)
2015-11-30 16:36:36.821 75614 TRACE nova.compute.manager [instance: 11a0f94f-c290-41d1-b46e-564e0243632e]   File "/usr/lib/python2.7/site-packages/nova/virt/images.py", line 87, in fetch_to_raw
2015-11-30 16:36:36.821 75614 TRACE nova.compute.manager [instance: 11a0f94f-c290-41d1-b46e-564e0243632e]     max_size=max_size)
2015-11-30 16:36:36.821 75614 TRACE nova.compute.manager [instance: 11a0f94f-c290-41d1-b46e-564e0243632e]   File "/usr/lib/python2.7/site-packages/nova/virt/images.py", line 77, in fetch
2015-11-30 16:36:36.821 75614 TRACE nova.compute.manager [instance: 11a0f94f-c290-41d1-b46e-564e0243632e]     IMAGE_API.download(context, image_href, dest_path=path)
2015-11-30 16:36:36.821 75614 TRACE nova.compute.manager [instance: 11a0f94f-c290-41d1-b46e-564e0243632e]   File "/usr/lib/python2.7/site-packages/nova/image/api.py", line 182, in download
2015-11-30 16:36:36.821 75614 TRACE nova.compute.manager [instance: 11a0f94f-c290-41d1-b46e-564e0243632e]     dst_path=dest_path)
2015-11-30 16:36:36.821 75614 TRACE nova.compute.manager [instance: 11a0f94f-c290-41d1-b46e-564e0243632e]   File "/usr/lib/python2.7/site-packages/nova/image/glance.py", line 352, in download
2015-11-30 16:36:36.821 75614 TRACE nova.compute.manager [instance: 11a0f94f-c290-41d1-b46e-564e0243632e]     _reraise_translated_image_exception(image_id)
2015-11-30 16:36:36.821 75614 TRACE nova.compute.manager [instance: 11a0f94f-c290-41d1-b46e-564e0243632e]   File "/usr/lib/python2.7/site-packages/nova/image/glance.py", line 350, in download
2015-11-30 16:36:36.821 75614 TRACE nova.compute.manager [instance: 11a0f94f-c290-41d1-b46e-564e0243632e]     image_chunks = self._client.call(context, 1, 'data', image_id)
2015-11-30 16:36:36.821 75614 TRACE nova.compute.manager [instance: 11a0f94f-c290-41d1-b46e-564e0243632e]   File "/usr/lib/python2.7/site-packages/nova/image/glance.py", line 237, in call
2015-11-30 16:36:36.821 75614 TRACE nova.compute.manager [instance: 11a0f94f-c290-41d1-b46e-564e0243632e]     host=host, port=port, reason=six.text_type(e))
2015-11-30 16:36:36.821 75614 TRACE nova.compute.manager [instance: 11a0f94f-c290-41d1-b46e-564e0243632e] GlanceConnectionFailed: Connection to glance host 10.225.21.29:9292 failed: Error finding address for http://10.225.21.29:9292/v1/images/098eb7e9-b2e4-40ad-9462-6ec96932977e: ('Connection aborted.', error(111, 'ECONNREFUSED'))
2015-11-30 16:36:36.821 75614 TRACE nova.compute.manager [instance: 11a0f94f-c290-41d1-b46e-564e0243632e]

Subscriber exclusive content

A Red Hat subscription provides unlimited access to our knowledgebase, tools, and much more.

Current Customers and Partners

Log in for full access

Log In

New to Red Hat?

Learn more about Red Hat subscriptions

Using a Red Hat product through a public cloud?

How to access this content