Calamari upgrade from RHCS1.2.3 to RHCS1.3 fails with the message "LostRemote: Lost remote after 10s heartbeat".
Issue
-
The Calamari node upgrade failed which was part of the upgrade process from RHCS1.2.3 to RHCS1.3.
-
The upgrade process is detailed in RHCS1.3 Installation Guide
-
The log file /var/log/calamari/calamari.log shows the following python trace-back.
2015-07-02 20:26:25,865 - ERROR - django.request Internal Server Error: /dashboard/
Traceback (most recent call last):
File "/opt/calamari/venv/lib/python2.7/site-packages/django/core/handlers/base.py", line 115, in get_response
response = callback(request, *callback_args, **callback_kwargs)
File "/opt/calamari/venv/lib/python2.7/site-packages/django/contrib/auth/decorators.py", line 25, in _wrapped_view
return view_func(request, *args, **kwargs)
File "/opt/calamari/venv/lib/python2.7/site-packages/calamari_web-0.1-py2.7.egg/calamari_web/views.py", line 38, in dashboard
clusters = client.list_clusters()
File "/opt/calamari/venv/lib/python2.7/site-packages/zerorpc/core.py", line 260, in <lambda>
return lambda *args, **kargs: self(method, *args, **kargs)
File "/opt/calamari/venv/lib/python2.7/site-packages/zerorpc/core.py", line 245, in __call__
return self._process_response(request_event, bufchan, timeout)
File "/opt/calamari/venv/lib/python2.7/site-packages/zerorpc/core.py", line 217, in _process_response
reply_event = bufchan.recv(timeout)
File "/opt/calamari/venv/lib/python2.7/site-packages/zerorpc/channel.py", line 267, in recv
event = self._input_queue.get(timeout=timeout)
File "/opt/calamari/venv/lib/python2.7/site-packages/gevent/queue.py", line 200, in get
result = waiter.get()
File "/opt/calamari/venv/lib/python2.7/site-packages/gevent/hub.py", line 568, in get
return self.hub.switch()
File "/opt/calamari/venv/lib/python2.7/site-packages/gevent/hub.py", line 331, in switch
return greenlet.switch(self)
LostRemote: Lost remote after 10s heartbeat
- Cleaning up Clamari instances and initializing it again, didn't help.
# calamari-ctl clean
# calamari-ctl initialize
Environment
- Red Hat Ceph Storage 1.2.3
Subscriber exclusive content
A Red Hat subscription provides unlimited access to our knowledgebase of over 48,000 articles and solutions.
Welcome! Check out the Getting Started with Red Hat page for quick tours and guides for common tasks.
