Calamari upgrade from RHCS1.2.3 to RHCS1.3 fails with the message "LostRemote: Lost remote after 10s heartbeat".

Solution In Progress - Updated -

Issue

  • The Calamari node upgrade failed which was part of the upgrade process from RHCS1.2.3 to RHCS1.3.

  • The upgrade process is detailed in RHCS1.3 Installation Guide

  • The log file /var/log/calamari/calamari.log shows the following python trace-back.

2015-07-02 20:26:25,865 - ERROR - django.request Internal Server Error: /dashboard/
Traceback (most recent call last):
  File "/opt/calamari/venv/lib/python2.7/site-packages/django/core/handlers/base.py", line 115, in get_response
    response = callback(request, *callback_args, **callback_kwargs)
  File "/opt/calamari/venv/lib/python2.7/site-packages/django/contrib/auth/decorators.py", line 25, in _wrapped_view
    return view_func(request, *args, **kwargs)
  File "/opt/calamari/venv/lib/python2.7/site-packages/calamari_web-0.1-py2.7.egg/calamari_web/views.py", line 38, in dashboard
    clusters = client.list_clusters()
  File "/opt/calamari/venv/lib/python2.7/site-packages/zerorpc/core.py", line 260, in <lambda>
    return lambda *args, **kargs: self(method, *args, **kargs)
  File "/opt/calamari/venv/lib/python2.7/site-packages/zerorpc/core.py", line 245, in __call__
    return self._process_response(request_event, bufchan, timeout)
  File "/opt/calamari/venv/lib/python2.7/site-packages/zerorpc/core.py", line 217, in _process_response
    reply_event = bufchan.recv(timeout)
  File "/opt/calamari/venv/lib/python2.7/site-packages/zerorpc/channel.py", line 267, in recv
    event = self._input_queue.get(timeout=timeout)
  File "/opt/calamari/venv/lib/python2.7/site-packages/gevent/queue.py", line 200, in get
    result = waiter.get()
  File "/opt/calamari/venv/lib/python2.7/site-packages/gevent/hub.py", line 568, in get
    return self.hub.switch()
  File "/opt/calamari/venv/lib/python2.7/site-packages/gevent/hub.py", line 331, in switch
    return greenlet.switch(self)
LostRemote: Lost remote after 10s heartbeat
  • Cleaning up Clamari instances and initializing it again, didn't help.
# calamari-ctl clean

# calamari-ctl initialize

Environment

  • Red Hat Ceph Storage 1.2.3

Subscriber exclusive content

A Red Hat subscription provides unlimited access to our knowledgebase of over 48,000 articles and solutions.

Current Customers and Partners

Log in for full access

Log In
Close

Welcome! Check out the Getting Started with Red Hat page for quick tours and guides for common tasks.