Unable to delete or show stack information

Solution In Progress - Updated -

Issue

  • End user is reporting stack deploy failed. We can see in openstack stack list the stack that failed: stack-vnf (96b65910-8935-4e22-b7d3-2062b4cd2698). However, when we use the command openstack stack show 96b65910-8935-4e22-b7d3-2062b4cd2698, the request times out with the error message: ERROR: Timed out waiting for a reply to message ID cd3cb5cfff94454bacb474181da1dcb3

  • We can confirm that other OSP commands work fine. (user list, flavor list, server list, subnet list, etc.) We did see one occurrance of openstack stack list producing the error:

[stack@undercloud-0-rhosp10 ~]$ openstack stack list
ERROR: Timed out waiting for a reply to message ID c484741172bd48998e220b96a39b4ee9
  • The following errors are seen in /var/log/heat/heat-api.log:
2019-10-04 13:15:08.395 4783 ERROR heat.common.wsgi [req-981d59c3-7aba-4b9c-9ea4-da075f5ba040 bd0b103077624f24b966e623716d321d 87e55c90ba20400e866480f5a989d52a - default default] Unexpected error occurred serving API: Timed out waiting for a reply to message ID f3456c0dd42c4c0c823aade12ee0465d
2019-10-04 13:15:08.395 4783 ERROR heat.common.wsgi Traceback (most recent call last):
2019-10-04 13:15:08.395 4783 ERROR heat.common.wsgi   File "/usr/lib/python2.7/site-packages/heat/common/wsgi.py", line 842, in __call__
2019-10-04 13:15:08.395 4783 ERROR heat.common.wsgi     request, **action_args)
2019-10-04 13:15:08.395 4783 ERROR heat.common.wsgi   File "/usr/lib/python2.7/site-packages/heat/common/wsgi.py", line 916, in dispatch
2019-10-04 13:15:08.395 4783 ERROR heat.common.wsgi     return method(*args, **kwargs)
2019-10-04 13:15:08.395 4783 ERROR heat.common.wsgi   File "/usr/lib/python2.7/site-packages/heat/api/openstack/v1/util.py", line 38, in handle_stack_method
2019-10-04 13:15:08.395 4783 ERROR heat.common.wsgi     return handler(controller, req, **kwargs)
2019-10-04 13:15:08.395 4783 ERROR heat.common.wsgi   File "/usr/lib/python2.7/site-packages/heat/api/openstack/v1/stacks.py", line 416, in lookup
2019-10-04 13:15:08.395 4783 ERROR heat.common.wsgi     stack_name)
2019-10-04 13:15:08.395 4783 ERROR heat.common.wsgi   File "/usr/lib/python2.7/site-packages/heat/rpc/client.py", line 119, in identify_stack
2019-10-04 13:15:08.395 4783 ERROR heat.common.wsgi     stack_name=stack_name))
2019-10-04 13:15:08.395 4783 ERROR heat.common.wsgi   File "/usr/lib/python2.7/site-packages/heat/rpc/client.py", line 84, in call
2019-10-04 13:15:08.395 4783 ERROR heat.common.wsgi     return client.call(ctxt, method, **kwargs)
2019-10-04 13:15:08.395 4783 ERROR heat.common.wsgi   File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/client.py", line 428, in call
2019-10-04 13:15:08.395 4783 ERROR heat.common.wsgi     return self.prepare().call(ctxt, method, **kwargs)
2019-10-04 13:15:08.395 4783 ERROR heat.common.wsgi   File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/client.py", line 169, in call
2019-10-04 13:15:08.395 4783 ERROR heat.common.wsgi     retry=self.retry)
2019-10-04 13:15:08.395 4783 ERROR heat.common.wsgi   File "/usr/lib/python2.7/site-packages/oslo_messaging/transport.py", line 97, in _send
2019-10-04 13:15:08.395 4783 ERROR heat.common.wsgi     timeout=timeout, retry=retry)
2019-10-04 13:15:08.395 4783 ERROR heat.common.wsgi   File "/usr/lib/python2.7/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 584, in send
2019-10-04 13:15:08.395 4783 ERROR heat.common.wsgi     retry=retry)
2019-10-04 13:15:08.395 4783 ERROR heat.common.wsgi   File "/usr/lib/python2.7/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 573, in _send
2019-10-04 13:15:08.395 4783 ERROR heat.common.wsgi     result = self._waiter.wait(msg_id, timeout)
2019-10-04 13:15:08.395 4783 ERROR heat.common.wsgi   File "/usr/lib/python2.7/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 456, in wait
2019-10-04 13:15:08.395 4783 ERROR heat.common.wsgi     message = self.waiters.get(msg_id, timeout=timeout)
2019-10-04 13:15:08.395 4783 ERROR heat.common.wsgi   File "/usr/lib/python2.7/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 347, in get
2019-10-04 13:15:08.395 4783 ERROR heat.common.wsgi     'to message ID %s' % msg_id)
2019-10-04 13:15:08.395 4783 ERROR heat.common.wsgi MessagingTimeout: Timed out waiting for a reply to message ID f3456c0dd42c4c0c823aade12ee0465d
2019-10-04 13:15:08.395 4783 ERROR heat.common.wsgi 

Environment

  • Red Hat OpenStack Platform 10.0 (RHOSP)

Subscriber exclusive content

A Red Hat subscription provides unlimited access to our knowledgebase, tools, and much more.

Current Customers and Partners

Log in for full access

Log In

New to Red Hat?

Learn more about Red Hat subscriptions

Using a Red Hat product through a public cloud?

How to access this content