Ceph: Throttling down or up backfill and recovery and rebalance.

When one adds or removes OSD's (disks) in a Ceph Cluster, the CRUSH algorithm will rebalance the cluster by moving placement groups to or from Ceph OSD's to restore balance. The process of migrating placement groups and the objects they contain can reduce the cluster’s operational performance considerably. To maintain operational performance, Ceph performs this migration with backfilli...