Red Hat Ceph Storage 5.x: Why are Ceph upgrades stuck while updating manager services?

Issue

• During a Ceph upgrade from version 5.x to 5.y or from version 5.x to 6.0, this process gets in a loop while upgrading the manager ( mgr ) services. Sample output:

  # ceph orch upgrade status
  {
      "target_image": "registry.example.com/rhceph/rhceph-5-rhel8@sha256:7dc93a9627bf75b2fbfdde6b93d886d41f2f25f2026136e9a93d92de8c8913b9",
      "in_progress": true,
      "services_complete": [
          "mgr"
      ],
      "progress": "3/59 daemons upgraded",
      "message": ""
  }
  

• A key factor is that this is seen in Ceph deployments with more than two manager services.

• Why is this occurring?
• Are there any workarounds to allow the upgrade to continue?