Resolution

ceph osd reweight (Temporary)

Syntax: ceph osd reweight{osd-num} {weight}

e.g:
~$ sudo ceph osd reweight 5 .8

Note:
The ceph osd reweight is not a persistent setting. When an OSD gets marked out, the osd weight will be set to 0. When it gets marked in again, the weight will be changed to 1.

Because of the ceph osd reweight is a temporary solution. You should only use it to keep your cluster running while you’re ordering more hardware.

ceph osd crush reweight (Non-Temporary)

Syntax: ceph osd crush reweight {name} {weight}

e.g:
~$ sudo ceph osd crush reweight osd.5 .8

Note:
The ceph osd crush reweight sets the CRUSH weight of the OSD. This weight is an arbitrary value (generally the size of the disk in TB or something) and controls how much data the system tries to allocate to the OSD.

The ceph osd crush reweight sets an override weight on the OSD. This value is in the range 0 to 1, and forces CRUSH to re-place (1-weight) of the data that would otherwise live on this drive. It does not change the weights assigned to the buckets above the OSD, and is a corrective measure in case the normal CRUSH distribution isn’t working out quite right. For instanse, if one of your OSD is at 90% and the others are at 50%, you could reduce this weight to try and compensate for it.

ceph osd reweight-by-utilization

Syntax: ceph osd reweight-by-utilization threshold(> 100)

e.g:
~$ sudo ceph osd reweight-by-utilization 120

Note:
Reweights all the OSDs by reducing the weight of OSDs which are heavily overused. By default it will adjust the weights downward on OSDs which have 120% of the average utilization, but if you include threshold it will use that percentage instead.