Appendix I. Ceph scrubbing options

Ceph ensures data integrity by scrubbing placement groups. The following are the Ceph scrubbing options that you can adjust to increase or decrease scrubbing operations.

You can set these configuration options with the ceph config set global CONFIGURATION_OPTION VALUE command.

mds_max_scrub_ops_in_progress
Description
The maximum number of scrub operations performed in parallel. You can set this value with ceph config set mds_max_scrub_ops_in_progress VALUE command.
Type
integer
Default
5
osd_max_scrubs
Description
The maximum number of simultaneous scrub operations for a Ceph OSD Daemon.
Type
integer
Default
1
osd_scrub_begin_hour
Description
The specific hour at which the scrubbing begins. Along with osd_scrub_end_hour, you can define a time window in which the scrubs can happen. Use osd_scrub_begin_hour = 0 and osd_scrub_end_hour = 0 to allow scrubbing the entire day.
Type
integer
Default
0
Allowed range
[0, 23]
osd_scrub_end_hour
Description
The specific hour at which the scrubbing ends. Along with osd_scrub_begin_hour, you can define a time window, in which the scrubs can happen. Use osd_scrub_begin_hour = 0 and osd_scrub_end_hour = 0 to allow scrubbing for the entire day.
Type
integer
Default
0
Allowed range
[0, 23]
osd_scrub_begin_week_day
Description
The specific day on which the scrubbing begins. 0 = Sunday, 1 = Monday, etc. Along with “osd_scrub_end_week_day”, you can define a time window in which scrubs can happen. Use osd_scrub_begin_week_day = 0 and osd_scrub_end_week_day = 0 to allow scrubbing for the entire week.
Type
integer
Default
0
Allowed range
[0, 6]
osd_scrub_end_week_day
Description
This defines the day on which the scrubbing ends. 0 = Sunday, 1 = Monday, etc. Along with osd_scrub_begin_week_day, they define a time window, in which the scrubs can happen. Use osd_scrub_begin_week_day = 0 and osd_scrub_end_week_day = 0 to allow scrubbing for the entire week.
Type
integer
Default
0
Allowed range
[0, 6]
osd_scrub_during_recovery
Description
Allow scrub during recovery. Setting this to false disables scheduling new scrub, and deep-scrub, while there is an active recovery. The already running scrubs continue which is useful to reduce load on busy storage clusters.
Type
boolean
Default
false
osd_scrub_load_threshold
Description
The normalized maximum load. Scrubbing does not happen when the system load, as defined by getloadavg() / number of online CPUs, is higher than this defined number.
Type
float
Default
0.5
osd_scrub_min_interval
Description
The minimal interval in seconds for scrubbing the Ceph OSD daemon when the Ceph storage Cluster load is low.
Type
float
Default
1 day
osd_scrub_max_interval
Description
The maximum interval in seconds for scrubbing the Ceph OSD daemon irrespective of cluster load.
Type
float
Default
7 days
osd_scrub_chunk_min
Description
The minimal number of object store chunks to scrub during a single operation. Ceph blocks writes to a single chunk during scrub.
type
integer
Default
5
osd_scrub_chunk_max
Description
The maximum number of object store chunks to scrub during a single operation.
type
integer
Default
25
osd_scrub_sleep
Description
Time to sleep before scrubbing the next group of chunks. Increasing this value slows down the overall rate of scrubbing, so that client operations are less impacted.
type
float
Default
0.0
osd_scrub_extended_sleep
Description
Duration to inject a delay during scrubbing out of scrubbing hours or seconds.
type
float
Default
0.0
osd_scrub_backoff_ratio
Description
Backoff ratio for scheduling scrubs. This is the percentage of ticks that do NOT schedule scrubs, 66% means that 1 out of 3 ticks schedules scrubs.
type
float
Default
0.66
osd_deep_scrub_interval
Description
The interval for deep scrubbing, fully reading all data. The osd_scrub_load_threshold does not affect this setting.
type
float
Default
7 days
osd_debug_deep_scrub_sleep
Description
Inject an expensive sleep during deep scrub IO to make it easier to induce preemption.
type
float
Default
0
osd_scrub_interval_randomize_ratio
Description
Add a random delay to osd_scrub_min_interval when scheduling the next scrub job for a placement group. The delay is a random value less than osd_scrub_min_interval * osd_scrub_interval_randomized_ratio. The default setting spreads scrubs throughout the allowed time window of [1, 1.5] * osd_scrub_min_interval.
type
float
Default
0.5
osd_deep_scrub_stride
Description
Read size when doing a deep scrub.
type
size
Default
512 KB
osd_scrub_auto_repair_num_errors
Description
Auto repair does not occur if more than these many errors are found.
type
integer
Default
5
osd_scrub_auto_repair
Description
Setting this to true enables automatic Placement Group (PG) repair when errors are found by scrubs or deep-scrubs. However, if more than osd_scrub_auto_repair_num_errors errors are found, a repair is NOT performed. This option is for periodic scrubs and not for operator-initiated scrubs.
type
boolean
Default
false
osd_scrub_max_preemptions
Description
Set the maximum number of times you need to preempt a deep scrub due to a client operation before blocking client IO to complete the scrub.
type
integer
Default
5
osd_deep_scrub_keys
Description
Number of keys to read from an object at a time during deep scrub.
type
integer
Default
1024