An instance is stuck in a building state due to qpid connection errors.
Issue
- An instance is stuck in a building state due to qpid connection errors.
- Following seen in
/var/log/nova/scheduler.log
2014-06-10 12:12:14.709 30225 TRACE root
2014-06-10 12:13:15.955 30225 ERROR root [-] Unexpected exception occurred 61 time(s)... retrying.
2014-06-10 12:13:15.955 30225 TRACE root Traceback (most recent call last):
2014-06-10 12:13:15.955 30225 TRACE root File "/usr/lib/python2.6/site-packages/nova/openstack/common/excutils.py", line 78, in inner_func
2014-06-10 12:13:15.955 30225 TRACE root return infunc(*args, **kwargs)
2014-06-10 12:13:15.955 30225 TRACE root File "/usr/lib/python2.6/site-packages/nova/openstack/common/rpc/impl_qpid.py", line 698, in _consumer_thread
2014-06-10 12:13:15.955 30225 TRACE root self.consume()
2014-06-10 12:13:15.955 30225 TRACE root File "/usr/lib/python2.6/site-packages/nova/openstack/common/rpc/impl_qpid.py", line 689, in consume
2014-06-10 12:13:15.955 30225 TRACE root it.next()
2014-06-10 12:13:15.955 30225 TRACE root File "/usr/lib/python2.6/site-packages/nova/openstack/common/rpc/impl_qpid.py", line 606, in iterconsume
2014-06-10 12:13:15.955 30225 TRACE root yield self.ensure(_error_callback, _consume)
2014-06-10 12:13:15.955 30225 TRACE root File "/usr/lib/python2.6/site-packages/nova/openstack/common/rpc/impl_qpid.py", line 540, in ensure
2014-06-10 12:13:15.955 30225 TRACE root return method(*args, **kwargs)
2014-06-10 12:13:15.955 30225 TRACE root File "/usr/lib/python2.6/site-packages/nova/openstack/common/rpc/impl_qpid.py", line 597, in _consume
2014-06-10 12:13:15.955 30225 TRACE root nxt_receiver = self.session.next_receiver(timeout=timeout)
2014-06-10 12:13:15.955 30225 TRACE root File "<string>", line 6, in next_receiver
2014-06-10 12:13:15.955 30225 TRACE root File "/usr/lib/python2.6/site-packages/qpid/messaging/endpoints.py", line 660, in next_receiver
2014-06-10 12:13:15.955 30225 TRACE root if self._ecwait(lambda: self.incoming, timeout):
2014-06-10 12:13:15.955 30225 TRACE root File "/usr/lib/python2.6/site-packages/qpid/messaging/endpoints.py", line 50, in _ecwait
2014-06-10 12:13:15.955 30225 TRACE root result = self._ewait(lambda: self.closed or predicate(), timeout)
2014-06-10 12:13:15.955 30225 TRACE root File "/usr/lib/python2.6/site-packages/qpid/messaging/endpoints.py", line 566, in _ewait
2014-06-10 12:13:15.955 30225 TRACE root result = self.connection._ewait(lambda: self.error or predicate(), timeout)
2014-06-10 12:13:15.955 30225 TRACE root File "/usr/lib/python2.6/site-packages/qpid/messaging/endpoints.py", line 209, in _ewait
2014-06-10 12:13:15.955 30225 TRACE root self.check_error()
2014-06-10 12:13:15.955 30225 TRACE root File "/usr/lib/python2.6/site-packages/qpid/messaging/endpoints.py", line 202, in check_error
2014-06-10 12:13:15.955 30225 TRACE root raise self.error
2014-06-10 12:13:15.955 30225 TRACE root InternalError: Traceback (most recent call last):
2014-06-10 12:13:15.955 30225 TRACE root File "/usr/lib/python2.6/site-packages/qpid/messaging/driver.py", line 651, in write
2014-06-10 12:13:15.955 30225 TRACE root self._op_dec.write(*self._seg_dec.read())
2014-06-10 12:13:15.955 30225 TRACE root File "/usr/lib/python2.6/site-packages/qpid/framing.py", line 269, in write
2014-06-10 12:13:15.955 30225 TRACE root if self.op.headers is None:
2014-06-10 12:13:15.955 30225 TRACE root AttributeError: 'NoneType' object has no attribute 'headers'
Environment
- Red Hat Enterprise Linux Openstack Platform 4.
Subscriber exclusive content
A Red Hat subscription provides unlimited access to our knowledgebase of over 48,000 articles and solutions.
Welcome! Check out the Getting Started with Red Hat page for quick tours and guides for common tasks.
