11.3. Supported Volume Options

The following table lists available volume options along with their description and default value.

Important

The default values are subject to change, and may not be the same for all versions of Red Hat Gluster Storage.

Table 11.1. Volume Options

OptionValue DescriptionAllowed ValuesDefault Value
auth.allowIP addresses or hostnames of the clients which are allowed to access the volume.Valid hostnames or IP addresses, which includes wild card patterns including *. For example, 192.168.1.*. A list of comma separated addresses is acceptable, but a single hostname must not exceed 256 characters.* (allow all)
auth.rejectIP addresses or hostnames of FUSE clients that are denied access to a volume. For NFS access control, use nfs.rpc-auth-* options instead.Valid hostnames or IP addresses, which includes wild card patterns including *. For example, 192.168.1.*. A list of comma separated addresses is acceptable, but a single hostname must not exceed 256 characters.none (reject none)
changelogEnables the changelog translator to record all the file operations.on | offoff
client.event-threadsSpecifies the number of network connections to be handled simultaneously by the client processes accessing a Red Hat Gluster Storage node.1 - 322
cluster.background-self-heal-countThe maximum number of heal operations that can occur simultaneously. Requests in excess of this number are stored in a queue whose length is defined by cluster.heal-wait-queue-leng.0–2568
cluster.brick-multiplex
Available as of Red Hat Gluster Storage 3.3 and later. Controls whether to use brick multiplexing on all volumes. Red Hat recommends restarting volumes after enabling or disabling brick multiplexing. When set to off (the default), each brick has its own process and uses its own port. When set to on, bricks that are compatible with each other use the same process and the same port. This reduces per-brick memory usage and port consumption.
Brick compatibility is determined at volume start, and depends on volume options shared between bricks. When multiplexing is enabled, restart volumes whenever volume configuration is changed in order to maintain the compatibility of the bricks grouped under a single process.
on | offoff
cluster.consistent-metadata
If set to on, the readdirp function in Automatic File Replication feature will always fetch metadata from their respective read children as long as it holds the good copy (the copy that does not need healing) of the file/directory. However, this could cause a reduction in performance where readdirps are involved.
This option requires that the volume is remounted on the client to take effect.
on | offoff
cluster.granular-entry-heal
If set to enable, stores more granular information about the entries which were created or deleted from a directory while a brick in a replica was down. This helps in faster self-heal of directories, especially in use cases where directories with large number of entries are modified by creating or deleting entries. If set to disable, it only stores that the directory needs heal without information about what entries within the directories need to be healed, and thereby requires entire directory crawl to identify the changes.
enable | disabledisable

Important

Execute the gluster volume set VOLNAME cluster.granular-entry-heal [enable | disable] command only if the volume is in Created state. If the volume is in any other state other than Created, for example, Started, Stopped, and so on, execute gluster volume heal VOLNAME granular-entry-heal [enable | disable] command to enable or disable granular-entry-heal option.
cluster.heal-wait-queue-lengThe maximum number of requests for heal operations that can be queued when heal operations equal to cluster.background-self-heal-count are already in progress. If more heal requests are made when this queue is full, those heal requests are ignored.0-10000128
cluster.lookup-optimize
If this option is set to on, when a hashed sub-volume does not return a lookup result, negative lookups are optimized by not continuing to look on non-hashed subvolumes.
For existing volumes, any directories created after the upgrade will have lookup-optimize behavior enabled. Rebalance operation has to be performed on all existing directories before they can use the lookup optimization.
For new volumes, the lookup-optimize behavior is enabled by default, except for the root of the volume. Run a rebalance operation in order to enable lookup-optimize for the root of the volume.
on|offon (in version 3.4 onward)
cluster.max-bricks-per-process
The maximum number of bricks that can run on a single instance of glusterfsd process.
As of Red Hat Gluster Storage 3.4 Batch 2 Update, the default value of this option is set to 250. This provides better control of resource usage for container-based workloads. In earlier versions, the default value was 0, which used a single process for all bricks on the node.
Updating the value of this option does not affect currently running bricks. Restart the volume to change this setting for existing bricks.
0 to system maximum (any positive integer greater than 1)250
cluster.min-free-diskSpecifies the percentage of disk space that must be kept free. This may be useful for non-uniform bricks.Percentage of required minimum free disk space.10%
cluster.op-versionAllows you to set the operating version of the cluster. The op-version number cannot be downgraded and is set for all volumes in the cluster. The op-version is not listed as part of gluster volume info command output.30708 | 30712 | 31001 | 31101 | 31302 | 31303 | 31304 | 31305 | 31306Default value depends on Red Hat Gluster Storage version first installed. For Red Hat Gluster Storage 3.4.4 async Update the value is set to 31306 for a new deployment.
cluster.read-freq-thresholdSpecifies the number of reads, in a promotion/demotion cycle, that would mark a file HOT for promotion. Any file that has read hits less than this value will be considered as COLD and will be demoted.0-200
cluster.self-heal-daemonSpecifies whether proactive self-healing on replicated volumes is activated.on | offon
cluster.data-self-healSpecifies whether proactive data self-healing on replicated volumes is activated.on | offon
cluster.entry-self-healSpecifies whether proactive self-healing of the contents of a directory on replicated volumes is activated.on | offon
cluster.metadata-self-healSpecifies whether proactive metadata self-healing on replicated volumes is activated.on | offon

Important

By default, cluster.data-self-heal, cluster.entry-self-heal, and cluster.metadata-self-heal options are turned on. These options are for client-side, that is, mounts, healing of files. These will not affect the self-heal daemon. As a file is accessed healing gets triggered on it without waiting for the Self-heal-daemon to select and heal the file. This maximizes the client-side I/O throughput. It is recommended to turn these options off, with the help of Red Hat support, in the case of severe CPU or memory consumption. When turned off, self-heal is not triggered when the client accesses the file and the entire network bandwidth is available for I/O from the clients (client-side healing and accessing files pending heal consumed some). The self-heal daemon will still heal the file during its crawl.
cluster.server-quorum-ratioSets the quorum percentage for the trusted storage pool.0 - 100>50%
cluster.server-quorum-typeIf set to server, this option enables the specified volume to participate in the server-side quorum. For more information on configuring the server-side quorum, see Section 11.15.1.1, “Configuring Server-Side Quorum”none | servernone
cluster.shd-max-threadsSpecifies the number of entries that can be self healed in parallel on each replica by self-heal daemon.1 - 641
cluster.shd-max-threadsSpecifies the number of entries that can be self healed in parallel on each replica by self-heal daemon.1 - 641
cluster.shd-wait-qlengthSpecifies the number of entries that must be kept in the queue for self-heal daemon threads to take up as soon as any of the threads are free to heal. This value should be changed based on how much memory self-heal daemon process can use for keeping the next set of entries that need to be healed.1 - 6555361024
cluster.shd-wait-qlengthSpecifies the number of entries that must be kept in the dispersed subvolume's queue for self-heal daemon threads to take up as soon as any of the threads are free to heal. This value should be changed based on how much memory self-heal daemon process can use for keeping the next set of entries that need to be healed.1 - 6555361024
cluster.tier-demote-frequencySpecifies how frequently the tier daemon must check for files to demote.1 - 172800 seconds3600 seconds
cluster.tier-max-filesSpecifies the maximum number of files that may be migrated in any direction from each node in a given cycle.1-100000 files10000
cluster.tier-max-mbSpecifies the maximum number of MB that may be migrated in any direction from each node in a given cycle.1 -100000 (100 GB)4000 MB
cluster.tier-modeIf set to cache mode, promotes or demotes files based on whether the cache is full or not, as specified with watermarks. If set to test mode, periodically demotes or promotes files automatically based on access.test | cachecache
cluster.tier-promote-frequencySpecifies how frequently the tier daemon must check for files to promote.1- 172800 seconds120 seconds
cluster.use-compound-fopsWhen enabled, write transactions that occur as part of Automatic File Replication are modified so that network round trips are reduced, improving performance.on | offoff
cluster.watermark-hiUpper percentage watermark for promotion. If hot tier fills above this percentage, no promotion will happen and demotion will happen with high probability.1- 99 %90%
cluster.watermark-lowLower percentage watermark. If hot tier is less full than this, promotion will happen and demotion will not happen. If greater than this, promotion/demotion will happen at a probability relative to how full the hot tier is.1- 99 %75%
cluster.write-freq-thresholdSpecifies the number of writes, in a promotion/demotion cycle, that would mark a file HOT for promotion. Any file that has write hits less than this value will be considered as COLD and will be demoted.0-200
config.transportSpecifies the type of transport(s) volume would support communicating over.tcp OR rdma OR tcp,rdmatcp
diagnostics.brick-log-buf-sizeThe maximum number of unique log messages that can be suppressed until the timeout or buffer overflow, whichever occurs first on the bricks.0 and 20 (0 and 20 included)5
diagnostics.brick-log-flush-timeoutThe length of time for which the log messages are buffered, before being flushed to the logging infrastructure (gluster or syslog files) on the bricks.30 - 300 seconds (30 and 300 included)120 seconds
diagnostics.brick-log-formatAllows you to configure the log format to log either with a message id or without one on the brick.no-msg-id | with-msg-idwith-msg-id
diagnostics.brick-log-levelChanges the log-level of the bricks.INFO | DEBUG | WARNING | ERROR | CRITICAL | NONE | TRACEinfo
diagnostics.brick-sys-log-levelDepending on the value defined for this option, log messages at and above the defined level are generated in the syslog and the brick log files.INFO | WARNING | ERROR | CRITICALCRITICAL
diagnostics.client-log-buf-sizeThe maximum number of unique log messages that can be suppressed until the timeout or buffer overflow, whichever occurs first on the clients.0 and 20 (0 and 20 included)5
diagnostics.client-log-flush-timeoutThe length of time for which the log messages are buffered, before being flushed to the logging infrastructure (gluster or syslog files) on the clients.30 - 300 seconds (30 and 300 included)120 seconds
diagnostics.client-log-formatAllows you to configure the log format to log either with a message ID or without one on the client.no-msg-id | with-msg-idwith-msg-id
diagnostics.client-log-levelChanges the log-level of the clients.INFO | DEBUG | WARNING | ERROR | CRITICAL | NONE | TRACEinfo
diagnostics.client-sys-log-levelDepending on the value defined for this option, log messages at and above the defined level are generated in the syslog and the client log files.INFO | WARNING | ERROR | CRITICALCRITICAL
disperse.eager-lockBefore a file operation starts, a lock is placed on the file. The lock remains in place until the file operation is complete. After the file operation completes, if eager-lock is on, the lock remains in place either until lock contention is detected, or for 1 second in order to check if there is another request for that file from the same client. If eager-lock is off, locks release immediately after file operations complete, improving performance for some operations, but reducing access efficiency.on | offon
disperse.other-eager-lockThis option is equivalent to the disperse.eager-lock option but applicable only for non regular files. When multiple clients access a particular directory, disabling disperse.other-eager-lockoption for the volume can improve performance for directory access without compromising performance of I/O's for regular files.on | offoff
disperse.shd-max-threadsSpecifies the number of entries that can be self healed in parallel on each disperse subvolume by self-heal daemon.1 - 641
disperse.shd-wait-qlengthSpecifies the number of entries that must be kept in the dispersed subvolume's queue for self-heal daemon threads to take up as soon as any of the threads are free to heal. This value should be changed based on how much memory self-heal daemon process can use for keeping the next set of entries that need to be healed.1 - 6555361024
features.ctr_link_consistencyEnables a crash consistent way of recording hardlink updates by Change Time Recorder translator. When recording in a crash consistent way the data operations will experience more latency.on | offoff
features.ctr-enabledEnables Change Time Recorder (CTR) translator for a tiered volume. This option is used in conjunction with features.record-counters option to enable recording write and read heat counters.on | offon
features.quota-deem-statfsWhen this option is set to on, it takes the quota limits into consideration while estimating the filesystem size. The limit will be treated as the total size instead of the actual size of filesystem.on | offon
features.read-onlySpecifies whether to mount the entire volume as read-only for all the clients accessing it.on | offoff
features.record-countersIf set to enabled, cluster.write-freq-thresholdand cluster.read-freq-thresholdoptions defines the number of writes and reads to a given file that are needed before triggering migration.on | offon
features.shardEnables or disables sharding on the volume. Affects files created after volume configuration.enable | disabledisable
features.shard-block-sizeSpecifies the maximum size of file pieces when sharding is enabled. Affects files created after volume configuration.512MB512MB
geo-replication.indexingEnables the marker translator to track the changes in the volume.on | offoff
network.ping-timeoutThe time the client waits for a response from the server. If a timeout occurs, all resources held by the server on behalf of the client are cleaned up. When the connection is reestablished, all resources need to be reacquired before the client can resume operations on the server. Additionally, locks are acquired and the lock tables are updated. A reconnect is a very expensive operation and must be avoided.42 seconds42 seconds
nfs.aclDisabling nfs.acl will remove support for the NFSACL sideband protocol. This is enabled by default.enable | disableenable
nfs.addr-namelookupSpecifies whether to lookup names for incoming client connections. In some configurations, the name server can take too long to reply to DNS queries, resulting in timeouts of mount requests. This option can be used to disable name lookups during address authentication. Note that disabling name lookups will prevent you from using hostnames in nfs.rpc-auth-*options.on | offoff
nfs.disableSpecifies whether to disable NFS exports of individual volumes.on | offoff
nfs.enable-ino32 For nfs clients or applciatons that do not support 64-bit inode numbers, use this option to make NFS return 32-bit inode numbers instead. Disabled by default, so NFS returns 64-bit inode numbers. This value is global and applies to all the volumes in the trusted storage pool. enable | disabledisable
nfs.export-volumesEnables or disables exporting entire volumes. If this option is disabled and the nfs.export-diroption is enabled, you can set subdirectories as the only exports.on | offon
nfs.mount-rmtabPath to the cache file that contains a list of NFS-clients and the volumes they have mounted. Change the location of this file to a mounted (with glusterfs-fuse, on all storage servers) volume to gain a trusted pool wide view of all NFS-clients that use the volumes. The contents of this file provide the information that can get obtained with the showmount command.Path to a directory/var/lib/glusterd/nfs/rmtab
nfs.mount-udpEnable UDP transport for the MOUNT sideband protocol. By default, UDP is not enabled, and MOUNT can only be used over TCP. Some NFS-clients (certain Solaris, HP-UX and others) do not support MOUNT over TCP and enabling nfs.mount-udpmakes it possible to use NFS exports provided by Red Hat Gluster Storage.disable | enabledisable
nfs.nlmBy default, the Network Lock Manager (NLMv4) is enabled. Use this option to disable NLM. Red Hat does not recommend disabling this option.on|offon
nfs.portAssociates glusterFS NFS with a non-default port.1025-6553538465- 38467
nfs.ports-insecureAllows client connections from unprivileged ports. By default only privileged ports are allowed. This is a global setting for allowing insecure ports for all exports using a single option.on | offoff
nfs.rdirplusThe default value is on. When this option is turned off, NFS falls back to standard readdir instead of readdirp. Turning this off would result in more lookup and stat requests being sent from the client which may impact performance.on|offon
nfs.rpc-auth-allow IP_ADRESSESA comma separated list of IP addresses allowed to connect to the server. By default, all clients are allowed.Comma separated list of IP addressesaccept all
nfs.rpc-auth-reject IP_ADRESSESA comma separated list of addresses not allowed to connect to the server. By default, all connections are allowed.Comma separated list of IP addressesreject none
nfs.server-aux-gidsWhen enabled, the NFS-server will resolve the groups of the user accessing the volume. NFSv3 is restricted by the RPC protocol (AUTH_UNIX/AUTH_SYS header) to 16 groups. By resolving the groups on the NFS-server, this limits can get by-passed.on|offoff
nfs.transport-typeSpecifies the transport used by GlusterFS NFS server to communicate with bricks.tcp OR rdmatcp
open-behindIt improves the application's ability to read data from a file by sending success notifications to the application whenever it receives an open call.on | offon
performance.cache-max-file-sizeSets the maximum file size cached by the io-cache translator. Can be specified using the normal size descriptors of KB, MB, GB, TB, or PB (for example, 6 GB).Size in bytes, or specified using size descriptors.2 ^ 64-1 bytes
performance.cache-min-file-sizeSets the minimum file size cached by the io-cache translator. Can be specified using the normal size descriptors of KB, MB, GB, TB, or PB (for example, 6 GB).Size in bytes, or specified using size descriptors.0
performance.cache-refresh-timeoutThe number of seconds cached data for a file will be retained. After this timeout, data re-validation will be performed.0 - 61 seconds1 second
performance.cache-sizeSize of the read cache.Size in bytes, or specified using size descriptors.32 MB
performance.client-io-threadsImproves performance for parallel I/O from a single mount point for dispersed (erasure-coded) volumes by allowing up to 16 threads to be used in parallel. When enabled, 1 thread is used by default, and further threads up to the maximum of 16 are created as required by client workload. This is useful for dispersed and distributed dispersed volumes. This feature is not recommended for distributed, replicated or distributed-replicated volumes. It is disabled by default on replicated and distributed-replicated volume types.on | offon, except for replicated and distributed-replicated volumes
performance.flush-behindSpecifies whether the write-behind translator performs flush operations in the background by returning (false) success to the application before flush file operations are sent to the backend file system.on | offon
performance.io-thread-countThe number of threads in the I/O threads translator.0 - 6516
performance.lazy-openThis option requires open-behind to be on. Perform an open in the backend only when a necessary FOP arrives (for example, write on the file descriptor, unlink of the file). When this option is disabled, perform backend open immediately after an unwinding open.Yes/NoYes
performance.md-cache-timeoutThe time period in seconds which controls when metadata cache has to be refreshed. If the age of cache is greater than this time-period, it is refreshed. Every time cache is refreshed, its age is reset to 0.0-600 seconds1 second
performance.nfs-strict-write-orderingSpecifies whether to prevent later writes from overtaking earlier writes for NFS, even if the writes do not relate to the same files or locations.on | offoff
performance.nfs.flush-behindSpecifies whether the write-behind translator performs flush operations in the background for NFS by returning (false) success to the application before flush file operations are sent to the backend file system.on | offon
performance.nfs.strict-o-directSpecifies whether to attempt to minimize the cache effects of I/O for a file on NFS. When this option is enabled and a file descriptor is opened using the O_DIRECT flag, write-back caching is disabled for writes that affect that file descriptor. When this option is disabled, O_DIRECT has no effect on caching. This option is ignored if performance.write-behind is disabled.on | offoff
performance.nfs.write-behind-trickling-writesEnables and disables trickling-write strategy for the write-behind translator for NFS clients.on | offon
performance.nfs.write-behind-window-sizeSpecifies the size of the write-behind buffer for a single file or inode for NFS.512 KB - 1 GB1 MB
performance.quick-readTo enable/disable quick-read translator in the volume.on | offon
performance.rda-cache-limitThe value specified for this option is the maximum size of cache consumed by the readdir-ahead translator. This value is global and the total memory consumption by readdir-ahead is capped by this value, irrespective of the number/size of directories cached.0-1GB10MB
performance.rda-request-sizeThe value specified for this option will be the size of buffer holding directory entries in readdirp response.4KB-128KB128KB
performance.resync-failed-syncs-after-fsyncIf syncing cached writes that were issued before an fsync operation fails, this option configures whether to reattempt the failed sync operations.on | offoff
performance.strict-o-directSpecifies whether to attempt to minimize the cache effects of I/O for a file. When this option is enabled and a file descriptor is opened using the O_DIRECT flag, write-back caching is disabled for writes that affect that file descriptor. When this option is disabled, O_DIRECT has no effect on caching. This option is ignored if performance.write-behind is disabled.on | offoff
performance.strict-write-orderingSpecifies whether to prevent later writes from overtaking earlier writes, even if the writes do not relate to the same files or locations.on | offoff
performance.use-anonymous-fdThis option requires open-behind to be on. For read operations, use anonymous file descriptor when the original file descriptor is open-behind and not yet opened in the backend.Yes | NoYes
performance.write-behindEnables and disables write-behind translator.on | offon
performance.write-behind-trickling-writesEnables and disables trickling-write strategy for the write-behind translator for FUSE clients.on | offon
performance.write-behind-window-sizeSpecifies the size of the write-behind buffer for a single file or inode.512 KB - 1 GB1 MB
rebal-throttleRebalance process is made multithreaded to handle multiple files migration for enhancing the performance. During multiple file migration, there can be a severe impact on storage system performance. The throttling mechanism is provided to manage it.lazy, normal, aggressivenormal
server.allow-insecureAllows FUSE-based client connections from unprivileged ports. By default, this is enabled, meaning that ports can accept and reject messages from insecure ports. When disabled, only privileged ports are allowed. This is a global setting for allowing insecure ports to be enabled for all FUSE-based exports using a single option. Use nfs.rpc-auth-* options for NFS access control.on | offon
server.anongidValue of the GID used for the anonymous user when root-squash is enabled. When root-squash is enabled, all the requests received from the root GID (that is 0) are changed to have the GID of the anonymous user.0 - 429496729565534 (this UID is also known as nfsnobody)
server.anonuidValue of the UID used for the anonymous user when root-squash is enabled. When root-squash is enabled, all the requests received from the root UID (that is 0) are changed to have the UID of the anonymous user.0 - 429496729565534 (this UID is also known as nfsnobody)
server.event-threadsSpecifies the number of network connections to be handled simultaneously by the server processes hosting a Red Hat Gluster Storage node.1 - 321
server.gid-timeoutThe time period in seconds which controls when cached groups has to expire. This is the cache that contains the groups (GIDs) where a specified user (UID) belongs to. This option is used only when server.manage-gids is enabled.0-4294967295 seconds2 seconds
server.manage-gidsResolve groups on the server-side. By enabling this option, the groups (GIDs) a user (UID) belongs to gets resolved on the server, instead of using the groups that were send in the RPC Call by the client. This option makes it possible to apply permission checks for users that belong to bigger group lists than the protocol supports (approximately 93).on|offoff
server.root-squashPrevents root users from having root privileges, and instead assigns them the privileges of nfsnobody. This squashes the power of the root users, preventing unauthorized modification of files on the Red Hat Gluster Storage servers. This option is used only for glusterFS NFS protocol.on | offoff
server.statedump-pathSpecifies the directory in which the statedumpfiles must be stored./var/run/gluster (for a default installation)Path to a directory
storage.fips-mode-rchecksumIf enabled, posix_rchecksum uses the FIPS compliant SHA256 checksum, else it uses MD5.on | offoff

Important

Do not enable fips-mode-rchecksum on volumes with clients that use Red Hat Gluster Storage 3.4 or earlier.
storage.create-maskMaximum set (upper limit) of permission for the files that will be created.0000 - 07770777
storage. create-directory-maskMaximum set (upper limit) of permission for the directories that will be created.0000 - 07770777
storage.force-create-modeMinimum set (lower limit) of permission for the files that will be created.0000 - 07770000
storage.force-create-directoryMinimum set (lower limit) of permission for the directories that will be created.0000 - 07770000

Important

Behavior is undefined in terms of calculated file access mode when both a mask and a matching forced mode are set simultaneously, create-directory-mask and force-directory-mode or create-mask and force-create-mode.
storage.health-check-intervalSets the time interval in seconds for a filesystem health check. You can set it to 0 to disable. The POSIX translator on the bricks performs a periodic health check. If this check fails, the file system exported by the brick is not usable anymore and the brick process (glusterfsd) logs a warning and exits.0-4294967295 seconds30 seconds
storage.owner-gidSets the GID for the bricks of the volume. This option may be required when some of the applications need the brick to have a specific GID to function correctly. Example: For QEMU integration the UID/GID must be qemu:qemu, that is, 107:107 (107 is the UID and GID of qemu).Any integer greater than or equal to -1.The GID of the bricks are not changed. This is denoted by -1.
storage.owner-uidSets the UID for the bricks of the volume. This option may be required when some of the applications need the brick to have a specific UID to function correctly. Example: For QEMU integration the UID/GID must be qemu:qemu, that is, 107:107 (107 is the UID and GID of qemu).Any integer greater than or equal to -1.The UID of the bricks are not changed. This is denoted by -1.
storage.reserveAs of Red Hat Gluster Storage 3.5, the POSIX translator includes an option storage.reserve to reserve storage space at the brick. This option accepts size in form of MB and also in form of percentage. If user has configured the storage.reserve option using size in MB earlier, and then wants to give the size in percentage, it can be done using the same option. Also, the newest set value is considered, if it was in MB before and then if it sent in percentage, the percentage value becomes new value and the older one is over-written0-1001 (1% of the brick size)
transport.listen-backlogThe maximum number of established TCP socket requests queued and waiting to be accepted at any one time.0 to system maximum10