Ceph/ODF: All MDS are up:standby, "mds: 0/1 daemons up (1 failed), 2 standby".

Solution Verified - Updated -

Issue

All MDS are up:standby, "mds: 0/1 daemons up (1 failed), 2 standby".

This example shows a system where there should be 1 Active MDS and 1 Standby MDS.

sh-5.1$ ceph status
  cluster:
    id:     [REDACTED]
    health: HEALTH_ERR
            1 filesystem is degraded
            1 filesystem has a failed mds daemon
            1 filesystem is offline

  services:
    mon: 3 daemons, quorum b,c,d (age 14h)
    mgr: a(active, since 11h), standbys: b
    mds: 0/1 daemons up (1 failed), 2 standby
    osd: 3 osds: 3 up (since 14h), 3 in (since 36h)
    rgw: 1 daemon active (1 hosts, 1 zones)

  data:
    volumes: 0/1 healthy, 1 failed
    pools:   12 pools, 169 pgs
    objects: 583 objects, 484 MiB
    usage:   2.0 GiB used, 30 TiB / 30 TiB avail
    pgs:     169 active+clean

For this solution to be applicable, these 3 conditions must exist.
See example in Diagnostic Steps section.

  • Both MDS Servers are "up:standby".
  • The MDS Filesystem is not damaged.
  • The MDS logs show all MDS servers transitioning to "up:standby" without cause.

Environment

Red Hat OpenShift Container Platform (OCP) 4.x
Red Hat OpenShift Container Storage (OCS) 4.x
Red Hat OpenShift Data Foundation (ODF) 4.x
Red Hat Ceph Storage (RHCS) 6.x
Red Hat Ceph Storage (RHCS) 7.x
Ceph File System (CephFS)

Subscriber exclusive content

A Red Hat subscription provides unlimited access to our knowledgebase, tools, and much more.

Current Customers and Partners

Log in for full access

Log In

New to Red Hat?

Learn more about Red Hat subscriptions

Using a Red Hat product through a public cloud?

How to access this content