MessagingTimeout error in heat-api-log during large stack update even though stack gets completed eventually
Issue
-
MessagingTimeout
error in heat-api-log during large stack update even though stack gets completed eventually -
heat-api.log:
2018-12-11 11:34:19.521 4214 ERROR heat.common.wsgi [req-f2e13a6a-b554-4fb0-8658-f095bc60aacd 46b1c2d662a543f0b40556a495784714 19f23c45d9cb4d2f863c5f8f454348aa - default default] Unexpected error occurred serving API: Timed out waiting for a reply to message ID d0eb2fcd98f4460cb8b0e98b89f4b533
2018-12-11 11:34:19.521 4214 ERROR heat.common.wsgi Traceback (most recent call last):
2018-12-11 11:34:19.521 4214 ERROR heat.common.wsgi File "/usr/lib/python2.7/site-packages/heat/common/wsgi.py", line 842, in __call__
2018-12-11 11:34:19.521 4214 ERROR heat.common.wsgi request, **action_args)
2018-12-11 11:34:19.521 4214 ERROR heat.common.wsgi File "/usr/lib/python2.7/site-packages/heat/common/wsgi.py", line 916, in dispatch
2018-12-11 11:34:19.521 4214 ERROR heat.common.wsgi return method(*args, **kwargs)
2018-12-11 11:34:19.521 4214 ERROR heat.common.wsgi File "/usr/lib/python2.7/site-packages/heat/api/openstack/v1/util.py", line 38, in handle_stack_method
2018-12-11 11:34:19.521 4214 ERROR heat.common.wsgi return handler(controller, req, **kwargs)
2018-12-11 11:34:19.521 4214 ERROR heat.common.wsgi File "/usr/lib/python2.7/site-packages/heat/api/openstack/v1/util.py", line 54, in handle_stack_method
2018-12-11 11:34:19.521 4214 ERROR heat.common.wsgi return handler(controller, req, dict(stack_identity), **kwargs)
2018-12-11 11:34:19.521 4214 ERROR heat.common.wsgi File "/usr/lib/python2.7/site-packages/heat/api/openstack/v1/stacks.py", line 484, in update
2018-12-11 11:34:19.521 4214 ERROR heat.common.wsgi environment_files=data.environment_files())
2018-12-11 11:34:19.521 4214 ERROR heat.common.wsgi File "/usr/lib/python2.7/site-packages/heat/rpc/client.py", line 302, in update_stack
2018-12-11 11:34:19.521 4214 ERROR heat.common.wsgi environment_files=environment_files)
2018-12-11 11:34:19.521 4214 ERROR heat.common.wsgi File "/usr/lib/python2.7/site-packages/heat/rpc/client.py", line 323, in _update_stack
2018-12-11 11:34:19.521 4214 ERROR heat.common.wsgi version='1.29')
2018-12-11 11:34:19.521 4214 ERROR heat.common.wsgi File "/usr/lib/python2.7/site-packages/heat/rpc/client.py", line 84, in call
2018-12-11 11:34:19.521 4214 ERROR heat.common.wsgi return client.call(ctxt, method, **kwargs)
2018-12-11 11:34:19.521 4214 ERROR heat.common.wsgi File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/client.py", line 169, in call
2018-12-11 11:34:19.521 4214 ERROR heat.common.wsgi retry=self.retry)
2018-12-11 11:34:19.521 4214 ERROR heat.common.wsgi File "/usr/lib/python2.7/site-packages/oslo_messaging/transport.py", line 97, in _send
2018-12-11 11:34:19.521 4214 ERROR heat.common.wsgi timeout=timeout, retry=retry)
2018-12-11 11:34:19.521 4214 ERROR heat.common.wsgi File "/usr/lib/python2.7/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 584, in send
2018-12-11 11:34:19.521 4214 ERROR heat.common.wsgi retry=retry)
2018-12-11 11:34:19.521 4214 ERROR heat.common.wsgi File "/usr/lib/python2.7/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 573, in _send
2018-12-11 11:34:19.521 4214 ERROR heat.common.wsgi result = self._waiter.wait(msg_id, timeout)
2018-12-11 11:34:19.521 4214 ERROR heat.common.wsgi File "/usr/lib/python2.7/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 456, in wait
2018-12-11 11:34:19.521 4214 ERROR heat.common.wsgi message = self.waiters.get(msg_id, timeout=timeout)
2018-12-11 11:34:19.521 4214 ERROR heat.common.wsgi File "/usr/lib/python2.7/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 347, in get
2018-12-11 11:34:19.521 4214 ERROR heat.common.wsgi 'to message ID %s' % msg_id)
2018-12-11 11:34:19.521 4214 ERROR heat.common.wsgi MessagingTimeout: Timed out waiting for a reply to message ID d0eb2fcd98f4460cb8b0e98b89f4b533
2018-12-11 11:34:19.521 4214 ERROR heat.common.wsgi
2018-12-11 11:34:19.534 4214 INFO eventlet.wsgi.server [req-f2e13a6a-b554-4fb0-8658-f095bc60aacd 46b1c2d662a543f0b40556a495784714 19f23c45d9cb4d2f863c5f8f454348aa - default default] <IP ADDR> - - [11/Dec/2018 11:34:19] "PUT /v1/e7b047587c464d02bfa830421d91bf90/stacks/<stack-name>/c4d62649-7ad8-4775-b2b9-613777b0a373 HTTP/1.1" 500 506 60.508204
Environment
- Red Hat OpenStack Platform 10
Subscriber exclusive content
A Red Hat subscription provides unlimited access to our knowledgebase, tools, and much more.