Ceph/ODF: ceph-mgr daemon missing from ceph status (no active mgr)

Solution Verified - Updated -

Issue

The ceph-mgr daemon missing from ceph status

The Ceph MGR daemon can go missing from ceph status. However, the service shows as up and running from the perspective of System Control (systemctl status) and Process Listing (ps -ef | grep -i mgr).

ODF example where only one MGR is configured:

ceph_status 
  cluster:
    id:     7e49xxxx-Redacted-Cluster-ID-yyyybb9cf3c1
    health: HEALTH_WARN
            no active mgr     <-- Note

  services:
    mon: 3 daemons, quorum j,l,o (age 12d)
    mgr: no daemons active (since 2d)
    mds: 1/1 daemons up, 1 hot standby
    osd: 72 osds: 72 up (since 3d), 72 in (since 3d)
    rgw: 1 daemon active (1 hosts, 1 zones)

  data:
    volumes: 1/1 healthy
    pools:   11 pools, 1361 pgs
    objects: 3.24M objects, 1.5 TiB
    usage:   4.7 TiB used, 388 TiB / 393 TiB avail
    pgs:     1361 active+clean

  io:
    client:   113 MiB/s rd, 135 MiB/s wr, 48 op/s rd, 458 op/s wr

Standalone Ceph example where 3 MGRs are configured:

  cluster:
    id:     2daxxxx-Redacted-Cluster-ID-yyyy57e8dd5f
    health: OK

  services:
    mon: 3 daemons, quorum edon-01,edon-03,edon-05 (age 2d)
    mgr: edon-05(active, since 5h), standbys: edon-01     <-- Note that edon-03 is missing
    mds: cephfs:1 {0=edon-02=up:active} 2 up:standby
    osd: 234 osds: 234 up (since 8d), 234 in (since 4w)
    rgw: 2 daemons active (edon-06.rgw0, edon-07.rgw1)

  data:
    pools:   13 pools, 1616 pgs
    objects: 1.31G objects, 126 TiB
    usage:   608 TiB used, 941 TiB / 1.5 PiB avail
    pgs:     1589 active+clean

  io:
    client:   34 MiB/s rd, 258 MiB/s wr, 8.14k op/s rd, 5.58k op/s wr

Environment

Red Hat OpenShift Container Platform (OCP) 4.x
Red Hat OpenShift Container Storage (OCS) 4.x
Red Hat OpenShift Data Foundation (ODF) 4.x
Red Hat Ceph Storage (RHCS) 5.x
Red Hat Ceph Storage (RHCS) 6.x
Red Hat Ceph Storage (RHCS) 7.x

Subscriber exclusive content

A Red Hat subscription provides unlimited access to our knowledgebase, tools, and much more.

Current Customers and Partners

Log in for full access

Log In

New to Red Hat?

Learn more about Red Hat subscriptions

Using a Red Hat product through a public cloud?

How to access this content