Red Hat Training

A Red Hat training course is available for Red Hat Ceph Storage

Appendix A. Configuration Reference

A.1. MDS Configuration Reference

mon force standby active
Description
If set to true, monitors force MDS in standby replay mode to be active. Set under the [mon] or [global] section in the Ceph configuration file. See Standby Replay for details.
Type
Boolean
Default
true
max mds
Description
The number of active MDS daemons during cluster creation. Set under the [mon] or [global] section in the Ceph configuration file.
Type
32-bit Integer
Default
1
mds max file size
Description
The maximum allowed file size to set when creating a new file system.
Type
64-bit Integer Unsigned
Default
1ULL << 40
mds cache size
Description
The number of inodes to cache.
Type
32-bit Integer
Default
100000
mds cache mid
Description
The insertion point for new items in the cache LRU (from the top).
Type
Float
Default
0.7
mds dir commit ratio
Description
The fraction of directory contains erroneous information before Ceph commits using a full update (instead of partial update).
Type
Float
Default
0.5
mds dir max commit size
Description
The maximum size of a directory update before Ceph breaks the directory into smaller transactions (in MB).
Type
32-bit Integer
Default
90
mds decay halflife
Description
The half-life of MDS cache temperature.
Type
Float
Default
5
mds beacon interval
Description
The frequency (in seconds) of beacon messages sent to the monitor.
Type
Float
Default
4
mds beacon grace
Description
The interval without beacons before Ceph declares an MDS laggy (and possibly replace it).
Type
Float
Default
15
mds blacklist interval
Description
The blacklist duration for failed MDS daemons in the OSD map.
Type
Float
Default
24.0*60.0
mds session timeout
Description
The interval (in seconds) of client inactivity before Ceph times out capabilities and leases.
Type
Float
Default
60
mds session autoclose
Description
The interval (in seconds) before Ceph closes a laggy client’s session.
Type
Float
Default
300
mds reconnect timeout
Description
The interval (in seconds) to wait for clients to reconnect during MDS restart.
Type
Float
Default
45
mds tick interval
Description
How frequently the MDS performs internal periodic tasks.
Type
Float
Default
5
mds dirstat min interval
Description
The minimum interval (in seconds) to try to avoid propagating recursive statistics up the tree.
Type
Float
Default
1
mds scatter nudge interval
Description
How quickly changes in directory statistics propagate up.
Type
Float
Default
5
mds client prealloc inos
Description
The number of inode numbers to preallocate per client session.
Type
32-bit Integer
Default
1000
mds early reply
Description
Determines whether the MDS allows clients to see request results before they commit to the journal.
Type
Boolean
Default
true
mds use tmap
Description
Use trivialmap for directory updates.
Type
Boolean
Default
true
mds default dir hash
Description
The function to use for hashing files across directory fragments.
Type
32-bit Integer
Default
2 (that is, rjenkins)
mds log
Description
Set to true if the MDS should journal metadata updates (disabled for benchmarking only).
Type
Boolean
Default
true
mds log skip corrupt events
Description
Determines whether the MDS tries to skip corrupt journal events during journal replay.
Type
Boolean
Default
false
mds log max events
Description
The maximum events in the journal before Ceph initiates trimming. Set to -1 to disable limits.
Type
32-bit Integer
Default
-1
mds log max segments
Description
The maximum number of segments (objects) in the journal before Ceph initiates trimming. Set to -1 to disable limits.
Type
32-bit Integer
Default
30
mds log max expiring
Description
The maximum number of segments to expire in parallels.
Type
32-bit Integer
Default
20
mds log eopen size
Description
The maximum number of inodes in an EOpen event.
Type
32-bit Integer
Default
100
mds bal sample interval
Description
Determines how frequently to sample directory temperature (for fragmentation decisions).
Type
Float
Default
3
mds bal replicate threshold
Description
The maximum temperature before Ceph attempts to replicate metadata to other nodes.
Type
Float
Default
8000
mds bal unreplicate threshold
Description
The minimum temperature before Ceph stops replicating metadata to other nodes.
Type
Float
Default
0
mds bal frag
Description
Determines whether the MDS will fragment directories.
Type
Boolean
Default
false
mds bal split size
Description
The maximum directory size before the MDS will split a directory fragment into smaller bits.
Type
32-bit Integer
Default
10000
mds bal split rd
Description
The maximum directory read temperature before Ceph splits a directory fragment.
Type
Float
Default
25000
mds bal split wr
Description
The maximum directory write temperature before Ceph splits a directory fragment.
Type
Float
Default
10000
mds bal split bits
Description
The number of bits by which to split a directory fragment.
Type
32-bit Integer
Default
3
mds bal merge size
Description
The minimum directory size before Ceph tries to merge adjacent directory fragments.
Type
32-bit Integer
Default
50
mds bal merge rd
Description
The minimum read temperature before Ceph merges adjacent directory fragments.
Type
Float
Default
1000
mds bal merge wr
Description
The minimum write temperature before Ceph merges adjacent directory fragments.
Type
Float
Default
1000
mds bal interval
Description
The frequency (in seconds) of workload exchanges between MDS nodes.
Type
32-bit Integer
Default
10
mds bal fragment interval
Description
The frequency (in seconds) of adjusting directory fragmentation.
Type
32-bit Integer
Default
5
mds bal idle threshold
Description
The minimum temperature before Ceph migrates a subtree back to its parent.
Type
Float
Default
0
mds bal max
Description
The number of iterations to run balancer before Ceph stops. Used for testing purposes only.
Type
32-bit Integer
Default
-1
mds bal max until
Description
The number of seconds to run balancer before Ceph stops. Used for testing purposes only.
Type
32-bit Integer
Default
-1
mds bal mode
Description

The method for calculating MDS load:

  • 1 = Hybrid.
  • 2 = Request rate and latency.
  • 3 = CPU load.
Type
32-bit Integer
Default
0
mds bal min rebalance
Description
The minimum subtree temperature before Ceph migrates.
Type
Float
Default
0.1
mds bal min start
Description
The minimum subtree temperature before Ceph searches a subtree.
Type
Float
Default
0.2
mds bal need min
Description
The minimum fraction of target subtree size to accept.
Type
Float
Default
0.8
mds bal need max
Description
The maximum fraction of target subtree size to accept.
Type
Float
Default
1.2
mds bal midchunk
Description
Ceph will migrate any subtree that is larger than this fraction of the target subtree size.
Type
Float
Default
0.3
mds bal minchunk
Description
Ceph will ignore any subtree that is smaller than this fraction of the target subtree size.
Type
Float
Default
0.001
mds bal target removal min
Description
The minimum number of balancer iterations before Ceph removes an old MDS target from the MDS map.
Type
32-bit Integer
Default
5
mds bal target removal max
Description
The maximum number of balancer iterations before Ceph removes an old MDS target from the MDS map.
Type
32-bit Integer
Default
10
mds replay interval
Description
The journal poll interval when in standby-replay mode (hot standby).
Type
Float
Default
1
mds shutdown check
Description
The interval for polling the cache during MDS shutdown.
Type
32-bit Integer
Default
0
mds thrash exports
Description
Ceph will randomly export subtrees between nodes (testing only).
Type
32-bit Integer
Default
0
mds thrash fragments
Description
Ceph will randomly fragment or merge directories.
Type
32-bit Integer
Default
0
mds dump cache on map
Description
Ceph will dump the MDS cache contents to a file on each MDS map.
Type
Boolean
Default
false
mds dump cache after rejoin
Description
Ceph will dump MDS cache contents to a file after rejoining the cache during recovery.
Type
Boolean
Default
false
mds verify scatter
Description
Ceph will assert that various scatter/gather invariants are true (for developers only).
Type
Boolean
Default
false
mds debug scatterstat
Description
Ceph will assert that various recursive statistics invariants are true (for developers only).
Type
Boolean
Default
false
mds debug frag
Description
Ceph will verify directory fragmentation invariants when convenient (for developers only).
Type
Boolean
Default
false
mds debug auth pins
Description
The debug authentication pin invariants (for developers only).
Type
Boolean
Default
false
mds debug subtrees
Description
The debug subtree invariants (for developers only).
Type
Boolean
Default
false
mds kill mdstable at
Description
Ceph will inject MDS failure in MDS Table code (for developers only).
Type
32-bit Integer
Default
0
mds kill export at
Description
Ceph will inject MDS failure in the subtree export code (for developers only).
Type
32-bit Integer
Default
0
mds kill import at
Description
Ceph will inject MDS failure in the subtree import code (for developers only).
Type
32-bit Integer
Default
0
mds kill link at
Description
Ceph will inject MDS failure in hard link code (for developers only).
Type
32-bit Integer
Default
0
mds kill rename at
Description
Ceph will inject MDS failure in the rename code (for developers only).
Type
32-bit Integer
Default
0
mds wipe sessions
Description
Ceph will delete all client sessions on startup (for testing only).
Type
Boolean
Default
0
mds wipe ino prealloc
Description
Ceph will delete inode preallocation metadata on startup (for testing only).
Type
Boolean
Default
0
mds skip ino
Description
The number of inode numbers to skip on startup (for testing only).
Type
32-bit Integer
Default
0
mds standby for name
Description
The MDS daemon will standby for another MDS daemon of the name specified in this setting.
Type
String
Default
N/A
mds standby for rank
Description
An instance of the MDS daemon will be standby for another MDS daemon instance of this rank.
Type
32-bit Integer
Default
-1
mds standby replay
Description
Determines whether the MDS daemon polls and replays the log of an active MDS (hot standby).
Type
Boolean
Default
false

A.2. Journaler Configuration Reference

journaler allow split entries
Description
Allow an entry to span a stripe boundary.
Type
Boolean
Required
No
Default
true
journaler write head interval
Description
How frequently to update the journal head object.
Type
Integer
Required
No
Default
15
journaler prefetch periods
Description
How many stripe periods to read ahead on journal replay.
Type
Integer
Required
No
Default
10
journal prezero periods
Description
How many stripe periods to zero ahead of write position.
Type
Integer
Required
No
Default
10
journaler batch interval
Description
Maximum additional latency in seconds to incur artificially.
Type
Double
Required
No
Default
.001
journaler batch max
Description
Maximum bytes that will be delayed flushing.
Type
64-bit Unsigned Integer
Required
No
Default
0

A.3. FUSE Client Configuration Reference

This section lists configuration options for CephFS FUSE clients. Set them in the Ceph configuration file under the [client] section.

client_acl_type
Description
Set the ACL type. Currently, only possible value is posix_acl to enable POSIX ACL, or an empty string. This option only takes effect when the fuse_default_permissions is set to false.
Type
String
Default
"" (no ACL enforcement)
client_cache_mid
Description
Set the client cache midpoint. The midpoint splits the least recently used lists into a hot and warm list.
Type
Float
Default
0.75
client_cache size
Description
Set the number of inodes that the client keeps in the metadata cache.
Type
Integer
Default
16384 (16 MB)
client_caps_release_delay
Description
Set the delay between capability releases in seconds. The delay sets how many seconds a client waits to release capabilities that it no longer needs in case the capabilities are needed for another user space operation.
Type
Integer
Default
5 (seconds)
client_debug_force_sync_read
Description
If set to true, clients read data directly from OSDs instead of using a local page cache.
Type
Boolean
Default
false
client_dirsize_rbytes
Description
If set to true, use the recursive size of a directory (that is, total of all descendants).
Type
Boolean
Default
true
client_max_inline_size
Description
Set the maximum size of inlined data stored in a file inode rather than in a separate data object in RADOS. This setting only applies if the inline_data flag is set on the MDS map.
Type
Integer
Default
4096
client_metadata
Description
Comma-delimited strings for client metadata sent to each MDS, in addition to the automatically generated version, host name, and other metadata.
Type
String
Default
"" (no additional metadata)
client_mount_gid
Description
Set the group ID of CephFS mount.
Type
Integer
Default
-1
client_mount_timeout
Description
Set the timeout for CephFS mount in seconds.
Type
Float
Default
300.0
client_mount_uid
Description
Set the user ID of CephFS mount.
Type
Integer
Default
-1
client_mountpoint
Description
An alternative to the -r option of the ceph-fuse command. See Path Restriction for details.
Type
String
Default
/
client_oc
Description
Enable object caching.
Type
Boolean
Default
true
client_oc_max_dirty
Description
Set the maximum number of dirty bytes in the object cache.
Type
Integer
Default
104857600 (100MB)
client_oc_max_dirty_age
Description
Set the maximum age in seconds of dirty data in the object cache before writeback.
Type
Float
Default
5.0 (seconds)
client_oc_max_objects
Description
Set the maximum number of objects in the object cache.
Type
Integer
Default
1000
client_oc_size
Description
Set how many bytes of data will the client cache.
Type
Integer
Default
209715200 (200 MB)
client_oc_target_dirty
Description
Set the target size of dirty data. Red Hat recommends to keep this number low.
Type
Integer
Default
8388608 (8MB)
client_permissions
Description
Check client permissions on all I/O operations.
Type
Boolean
Default
true
client_quota
Description
Enable client quotas if set to true.
Type
Boolean
Default
false
client_quota_df
Description
Report root directory quota for the statfs operation.
Type
Boolean
Default
true
client_readahead_max_bytes
Description
Set the maximum number of bytes that the kernel reads ahead for future read operations. Overridden by the client_readahead_max_periods setting.
Type
Integer
Default
0 (unlimited)
client_readahead_max_periods
Description
Set the number of file layout periods (object size * number of stripes) that the kernel reads ahead. Overrides the client_readahead_max_bytes setting.
Type
Integer
Default
4
client_readahead_min
Description
Set the minimum number bytes that the kernel reads ahead.
Type
Integer
Default
131072 (128KB)
client_snapdir
Description
Set the snapshot directory name.
Type
String
Default
".snap"
client_tick_interval
Description
Set the interval in seconds between capability renewal and other upkeep.
Type
Float
Default
1.0
client_use_random_mds
Description
Choose random MDS for each request.
Type
Boolean
Default
false
fuse_default_permissions
Description
When set to false, the ceph-fuse utility checks does its own permissions checking, instead of relying on the permissions enforcement in FUSE. Set to false together with the client acl type=posix_acl option to enable POSIX ACL.
Type
Boolean
Default
true

Developer Options

Important

These options are internal. They are listed here only to complete the list of options.

client_debug_getattr_caps
Description
Check if the reply from the MDS contains required capabilities.
Type
Boolean
Default
false
client_debug_inject_tick_delay
Description
Add artificial delay between client ticks.
Type
Integer
Default
0
client_inject_fixed_oldest_tid
Description, Type
Boolean
Default
false
client_inject_release_failure
Description, Type
Boolean
Default
false
client_trace
Description
The path to the trace file for all file operations. The output is designed to be used by the Ceph synthetic client. See the ceph-syn(8) manual page for details.
Type
String
Default
"" (disabled)