Machine Config Pool (MCP) Rollout/OCP Upgrade Fails to Apply on the OpenShift Data Foundation (ODF) Nodes - Ceph not Healthy w/Placement Groups (PGs) not in an "active+clean" State

Issue

• MachineConfig fails to apply on ODF nodes
• Degraded Machines
• Unable to evict OSD pods from ODF nodes
• Unable to evict monitor pod from ODF nodes

    health: HEALTH_WARN
            6 pgs not deep-scrubbed in time
            4 pgs not scrubbed in time

  services:
    mon: 3 daemons, quorum a,b,c (age 2d)
    mgr: a(active, since 39h), standbys: b
    mds: 1/1 daemons up, 1 hot standby
    osd: 3 osds: 3 up (since 2d), 3 in (since 2d)
    rgw: 1 daemon active (1 hosts, 1 zones)

  data:
    volumes: 1/1 healthy
    pools:   12 pools, 169 pgs
    objects: 1.45k objects, 2.6 GiB
    usage:   10 GiB used, 1.5 TiB / 1.5 TiB avail
    pgs:     240616/6029865 objects misplaced (3.990%)
             358 active+clean <------------------------------- ALL PGs need to look like this
             39  active+clean+scrubbing+deep <---------------- NOT active+clean
             16  active+remapped+backfill_wait <-------------- NOT active+clean
             3   active+remapped+backfilling <---------------- NOT active+clean
             1   active+clean+scrubbing <--------------------- NOT active+clean