Calamari dashboard shows an HTTP 500 error, after upgrading `salt` packages.

Solution In Progress - Updated -

Issue

  • Upgrading salt packages on the Calamari node ends up showing an HTTP 500 error on the Calamari dashboard.

  • The packages were upgraded to:

salt-2015.5.10-2.el7.noarch
salt-minion-2015.5.10-2.el7.noarch
salt-master-2015.5.10-2.el7.noarch
  • Calamari logs at /var/log/calamari/calamari.log shows a python traceback.
2017-02-23 07:28:33,488 - metric_access - django.request.profile Slow RPC 'server_by_service' (203.300952911ms)
2017-02-23 07:28:33,844 - metric_access - django.request.profile Slow RPC 'list' (201.82800293ms)
2017-02-23 20:51:30,050 - ERROR - django.request Internal Server Error: /dashboard/
Traceback (most recent call last):
  File "/opt/calamari/venv/lib/python2.7/site-packages/django/core/handlers/base.py", line 115, in get_response
    response = callback(request, *callback_args, **callback_kwargs)
  File "/opt/calamari/venv/lib/python2.7/site-packages/django/contrib/auth/decorators.py", line 25, in _wrapped_view
    return view_func(request, *args, **kwargs)
  File "/opt/calamari/venv/lib/python2.7/site-packages/calamari_web-0.1-py2.7.egg/calamari_web/views.py", line 38, in dashboard
    clusters = client.list_clusters()
  File "/opt/calamari/venv/lib/python2.7/site-packages/zerorpc/core.py", line 260, in <lambda>
    return lambda *args, **kargs: self(method, *args, **kargs)
  File "/opt/calamari/venv/lib/python2.7/site-packages/zerorpc/core.py", line 245, in __call__
    return self._process_response(request_event, bufchan, timeout)
  File "/opt/calamari/venv/lib/python2.7/site-packages/zerorpc/core.py", line 217, in _process_response
    reply_event = bufchan.recv(timeout)
  File "/opt/calamari/venv/lib/python2.7/site-packages/zerorpc/channel.py", line 267, in recv
    event = self._input_queue.get(timeout=timeout)
  File "/opt/calamari/venv/lib/python2.7/site-packages/gevent/queue.py", line 200, in get
    result = waiter.get()
  File "/opt/calamari/venv/lib/python2.7/site-packages/gevent/hub.py", line 568, in get
    return self.hub.switch()
  File "/opt/calamari/venv/lib/python2.7/site-packages/gevent/hub.py", line 331, in switch
    return greenlet.switch(self)
LostRemote: Lost remote after 10s heartbeat

Environment

  • Red Hat Ceph Storage 1.3.x

Subscriber exclusive content

A Red Hat subscription provides unlimited access to our knowledgebase of over 48,000 articles and solutions.

Current Customers and Partners

Log in for full access

Log In
Close

Welcome! Check out the Getting Started with Red Hat page for quick tours and guides for common tasks.