Red Hat Training
A Red Hat training course is available for Red Hat Enterprise Linux
188.8.131.52.2. Optimizing for a large number of files in a single directory
Directory block size is fixed for the life of a file system, and cannot be changed except upon initial formatting with
mkfs. The minimum directory block is the file system block size, which defaults to
MAX(4 KB, file system block size). In general, there is no reason to reduce the directory block size.
Because the directory structure is b-tree based, changing the block size affects the amount of directory information that can be retrieved or modified per physical I/O. The larger the directory becomes, the more I/O each operation requires at a given block size.
However, when larger directory block sizes are in use, more CPU is consumed by each modification operation compared to the same operation on a file system with a smaller directory block size. This means that for small directory sizes, large directory block sizes will result in lower modification performance. When the directory reaches a size where I/O is the performance-limiting factor, large block size directories perform better.
If writing to the file system is more common or more important than reading, such as in a file system that stores backups (where writing backups to the file system happens more frequently than restoring the backups):
- file systems requiring no more than 1–2 million directory entries (with entry name lengths of 20–40 bytes) perform best with the default configuration of 4 KB file system block size and 4 KB directory block size.
- file systems requiring between 1–10 million directory entries perform better with a larger block size of 16 KB
- file systems requiring more than 10 million directory entries perform better with an even larger block size of 64 KB
If reading from the file system is more common or more important than writing, for example, in a mirrored download server, the contents of which are viewed more often than modified, the number of entries to which these block sizes apply should be reduced by a factor of ten. For example, the default system and directory block size is best for read-heavy file systems with no more than ten to twenty thousand directory entries.