Ceph dashboard throws "500 Internal Server Error" while collecting metrics.

Solution Verified - Updated -

Issue

  • Ceph dashboard throws 500 Internal Server Error while collecting metrics.
  • Manager logs shows ConnectionError: ('Connection aborted.', BadStatusLine(XXX))
2021-11-11 10:11:20.497 7fd1c673a700  0 mgr[dashboard] [11/Nov/2021:10:11:20] HTTP Traceback (most recent call last):
  File "/usr/lib/python2.7/site-packages/cherrypy/_cprequest.py", line 656, in respond
    response.body = self.handler()
  File "/usr/lib/python2.7/site-packages/cherrypy/lib/encoding.py", line 188, in __call__
    self.body = self.oldhandler(*args, **kwargs)
  File "/usr/lib/python2.7/site-packages/cherrypy/_cptools.py", line 221, in wrap
    return self.newhandler(innerfunc, *args, **kwargs)
  File "/usr/share/ceph/mgr/dashboard/services/exception.py", line 88, in dashboard_exception_handler
    return handler(*args, **kwargs)
  File "/usr/lib/python2.7/site-packages/cherrypy/_cpdispatch.py", line 34, in __call__
    return self.callable(*self.args, **self.kwargs)
  File "/usr/share/ceph/mgr/dashboard/controllers/__init__.py", line 662, in inner
    ret = func(*args, **kwargs)
  File "/usr/share/ceph/mgr/dashboard/controllers/grafana.py", line 28, in validation
    response = grafana.url_validation(method, url)
  File "/usr/share/ceph/mgr/dashboard/grafana.py", line 21, in url_validation
    verify=Settings.GRAFANA_API_SSL_VERIFY)
  File "/usr/lib/python2.7/site-packages/requests/api.py", line 50, in request
    response = session.request(method=method, url=url, **kwargs)
  File "/usr/lib/python2.7/site-packages/requests/sessions.py", line 486, in request
    resp = self.send(prep, **send_kwargs)
  File "/usr/lib/python2.7/site-packages/requests/sessions.py", line 598, in send
    r = adapter.send(request, **kwargs)
  File "/usr/lib/python2.7/site-packages/requests/adapters.py", line 415, in send
    raise ConnectionError(err, request=request)
ConnectionError: ('Connection aborted.', BadStatusLine('\x15\x03\x01\x00\x02\x02\n',))

2021-11-11 10:11:20.497 7fd1c673a700  0 mgr[dashboard] [192.168.5.20:50302] [GET] [500] [0.009s] [admin] [513B] /api/grafana/validation/z99hzWtmk
2021-11-11 10:11:20.497 7fd1c673a700  0 mgr[dashboard] ['{"status": "500 Internal Server Error", "detail": "The server encountered an unexpected condition which prevented it from fulfilling the request.", "request_id": "209ce1e0-33fa-4b88-93ce-3e428525aeb8"}                                                                                                                                                                                                                                                                                                                        ']
2021-11-11 10:11:20.504 7fd1ec50c700  0 log_channel(cluster) log [DBG] : pgmap v2067814: 2944 pgs: 2944 active+clean; 3.8 TiB data, 29 TiB used, 404 TiB / 434 TiB avail; 1.5 MiB/s rd, 84 B/s wr, 1.58k op/s
2021-11-11 10:11:20.728 7fd1c4f37700  0 mgr[dashboard] frontend error (https://node2.example.com:8443/#/pool): Http failure response for https://node2.example.com:8443/api/grafana/validation/z99hzWtmk: 500 Internal Server Error
None

Environment

  • Red Hat Ceph Storage 4.2

Subscriber exclusive content

A Red Hat subscription provides unlimited access to our knowledgebase, tools, and much more.

Current Customers and Partners

Log in for full access

Log In

New to Red Hat?

Learn more about Red Hat subscriptions

Using a Red Hat product through a public cloud?

How to access this content