MessagingTimeout error in heat-api-log during large stack update even though stack gets completed eventually

Solution Unverified - Updated -

Issue

  • MessagingTimeout error in heat-api-log during large stack update even though stack gets completed eventually

  • heat-api.log:

2018-12-11 11:34:19.521 4214 ERROR heat.common.wsgi [req-f2e13a6a-b554-4fb0-8658-f095bc60aacd 46b1c2d662a543f0b40556a495784714 19f23c45d9cb4d2f863c5f8f454348aa - default default] Unexpected error occurred serving API: Timed out waiting for a reply to message ID d0eb2fcd98f4460cb8b0e98b89f4b533
2018-12-11 11:34:19.521 4214 ERROR heat.common.wsgi Traceback (most recent call last):
2018-12-11 11:34:19.521 4214 ERROR heat.common.wsgi   File "/usr/lib/python2.7/site-packages/heat/common/wsgi.py", line 842, in __call__
2018-12-11 11:34:19.521 4214 ERROR heat.common.wsgi     request, **action_args)
2018-12-11 11:34:19.521 4214 ERROR heat.common.wsgi   File "/usr/lib/python2.7/site-packages/heat/common/wsgi.py", line 916, in dispatch
2018-12-11 11:34:19.521 4214 ERROR heat.common.wsgi     return method(*args, **kwargs)
2018-12-11 11:34:19.521 4214 ERROR heat.common.wsgi   File "/usr/lib/python2.7/site-packages/heat/api/openstack/v1/util.py", line 38, in handle_stack_method
2018-12-11 11:34:19.521 4214 ERROR heat.common.wsgi     return handler(controller, req, **kwargs)
2018-12-11 11:34:19.521 4214 ERROR heat.common.wsgi   File "/usr/lib/python2.7/site-packages/heat/api/openstack/v1/util.py", line 54, in handle_stack_method
2018-12-11 11:34:19.521 4214 ERROR heat.common.wsgi     return handler(controller, req, dict(stack_identity), **kwargs)
2018-12-11 11:34:19.521 4214 ERROR heat.common.wsgi   File "/usr/lib/python2.7/site-packages/heat/api/openstack/v1/stacks.py", line 484, in update
2018-12-11 11:34:19.521 4214 ERROR heat.common.wsgi     environment_files=data.environment_files())
2018-12-11 11:34:19.521 4214 ERROR heat.common.wsgi   File "/usr/lib/python2.7/site-packages/heat/rpc/client.py", line 302, in update_stack
2018-12-11 11:34:19.521 4214 ERROR heat.common.wsgi     environment_files=environment_files)
2018-12-11 11:34:19.521 4214 ERROR heat.common.wsgi   File "/usr/lib/python2.7/site-packages/heat/rpc/client.py", line 323, in _update_stack
2018-12-11 11:34:19.521 4214 ERROR heat.common.wsgi     version='1.29')
2018-12-11 11:34:19.521 4214 ERROR heat.common.wsgi   File "/usr/lib/python2.7/site-packages/heat/rpc/client.py", line 84, in call
2018-12-11 11:34:19.521 4214 ERROR heat.common.wsgi     return client.call(ctxt, method, **kwargs)
2018-12-11 11:34:19.521 4214 ERROR heat.common.wsgi   File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/client.py", line 169, in call
2018-12-11 11:34:19.521 4214 ERROR heat.common.wsgi     retry=self.retry)
2018-12-11 11:34:19.521 4214 ERROR heat.common.wsgi   File "/usr/lib/python2.7/site-packages/oslo_messaging/transport.py", line 97, in _send
2018-12-11 11:34:19.521 4214 ERROR heat.common.wsgi     timeout=timeout, retry=retry)
2018-12-11 11:34:19.521 4214 ERROR heat.common.wsgi   File "/usr/lib/python2.7/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 584, in send
2018-12-11 11:34:19.521 4214 ERROR heat.common.wsgi     retry=retry)
2018-12-11 11:34:19.521 4214 ERROR heat.common.wsgi   File "/usr/lib/python2.7/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 573, in _send
2018-12-11 11:34:19.521 4214 ERROR heat.common.wsgi     result = self._waiter.wait(msg_id, timeout)
2018-12-11 11:34:19.521 4214 ERROR heat.common.wsgi   File "/usr/lib/python2.7/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 456, in wait
2018-12-11 11:34:19.521 4214 ERROR heat.common.wsgi     message = self.waiters.get(msg_id, timeout=timeout)
2018-12-11 11:34:19.521 4214 ERROR heat.common.wsgi   File "/usr/lib/python2.7/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 347, in get
2018-12-11 11:34:19.521 4214 ERROR heat.common.wsgi     'to message ID %s' % msg_id)
2018-12-11 11:34:19.521 4214 ERROR heat.common.wsgi MessagingTimeout: Timed out waiting for a reply to message ID d0eb2fcd98f4460cb8b0e98b89f4b533
2018-12-11 11:34:19.521 4214 ERROR heat.common.wsgi
2018-12-11 11:34:19.534 4214 INFO eventlet.wsgi.server [req-f2e13a6a-b554-4fb0-8658-f095bc60aacd 46b1c2d662a543f0b40556a495784714 19f23c45d9cb4d2f863c5f8f454348aa - default default] <IP ADDR> - - [11/Dec/2018 11:34:19] "PUT /v1/e7b047587c464d02bfa830421d91bf90/stacks/<stack-name>/c4d62649-7ad8-4775-b2b9-613777b0a373 HTTP/1.1" 500 506 60.508204

Environment

  • Red Hat OpenStack Platform 10

Subscriber exclusive content

A Red Hat subscription provides unlimited access to our knowledgebase, tools, and much more.

Current Customers and Partners

Log in for full access

Log In

New to Red Hat?

Learn more about Red Hat subscriptions

Using a Red Hat product through a public cloud?

How to access this content