[Ceph] MON store busy with leveldb, running into cpu_tp timeouts, mgr set_store mon returned -27

Solution Verified - Updated -

Issue

We're running our MONs still on leveldb, as we were upgrading from an older version where rocksdb was not yet available as MON store. Recently we did run into MON cpu_tp thread timeouts. A lot of leveldb activity can be seen in the logs. Also we do see mgr set_store mon returned -27 error messages.

Looking at the ceph report output, we do see that a lot of osdmaps (>70k) are being kept:

ceph_report:    "osdmap_first_committed": 410264,
ceph_report:    "osdmap_last_committed": 484941,

Environment

  • Red Hat Ceph Storage 4
  • MON store still on leveldb from older Ceph version

Subscriber exclusive content

A Red Hat subscription provides unlimited access to our knowledgebase, tools, and much more.

Current Customers and Partners

Log in for full access

Log In

New to Red Hat?

Learn more about Red Hat subscriptions

Using a Red Hat product through a public cloud?

How to access this content