Chapter 9. Getting started with FS-Cache

FS-Cache is a persistent local cache that file systems can use to take data retrieved from over the network and cache it on local disk. This helps minimize network traffic for users accessing data from a file system mounted over the network (for example, NFS).

9.1. Overview of the FS-Cache

The following diagram is a high-level illustration of how FS-Cache works:

Figure 9.1. FS-Cache Overview

FS-Cache Overview

FS-Cache is designed to be as transparent as possible to the users and administrators of a system. Unlike cachefs on Solaris, FS-Cache allows a file system on a server to interact directly with a client’s local cache without creating an overmounted file system. With NFS, a mount option instructs the client to mount the NFS share with FS-cache enabled. The mount point will cause automatic upload for two kernel modules: fscache and cachefiles. The cachefilesd daemon communicates with the kernel modules to implement the cache.

FS-Cache does not alter the basic operation of a file system that works over the network - it merely provides that file system with a persistent place in which it can cache data. For example, a client can still mount an NFS share whether or not FS-Cache is enabled. In addition, cached NFS can handle files that will not fit into the cache (whether individually or collectively) as files can be partially cached and do not have to be read completely up front. FS-Cache also hides all I/O errors that occur in the cache from the client file system driver.

To provide caching services, FS-Cache needs a cache back end. A cache back end is a storage driver configured to provide caching services, which is cachefiles. In this case, FS-Cache requires a mounted block-based file system, such as ext3, that supports bmap and extended attributes as its cache back end.

File systems that support functionalities required by FS-Cache cache back end include the Red Hat Enterprise Linux 9 implementations of the following file systems:

  • ext3 (with extended attributes enabled)
  • ext4
  • XFS

FS-Cache cannot arbitrarily cache any file system, whether through the network or otherwise: the shared file system’s driver must be altered to allow interaction with FS-Cache, data storage/retrieval, and metadata setup and validation. FS-Cache needs indexing keys and coherency data from the cached file system to support persistence: indexing keys to match file system objects to cache objects, and coherency data to determine whether the cache objects are still valid.

Note

In Red Hat Enterprise Linux 9, the cachefilesd package is not installed by default and needs to be installed manually.

9.2. Performance guarantee

FS-Cache does not guarantee increased performance. Using a cache incurs a performance penalty: for example, cached NFS shares add disk accesses to cross-network lookups. While FS-Cache tries to be as asynchronous as possible, there are synchronous paths, such as read operations, where this is not possible.

For example, using FS-Cache to cache an NFS share between two computers over an otherwise unladen GigE network likely will not demonstrate any performance improvements on file access. Rather, NFS requests would be satisfied faster from server memory rather than from local disk.

The use of FS-Cache, therefore, is a compromise between various factors. If FS-Cache is being used to cache NFS traffic, for example, it may slow the client down a little, but massively reduce the network and server loading by satisfying read requests locally without consuming network bandwidth.

9.3. Setting up a cache

Currently, Red Hat Enterprise Linux 9 only provides the cachefiles caching back end. The cachefilesd daemon initiates and manages cachefiles. The /etc/cachefilesd.conf file controls how cachefiles provides caching services.

The cache back end works by maintaining a certain amount of free space on the partition hosting the cache. It grows and shrinks the cache in response to other elements of the system using up free space, making it safe to use on the root file system (for example, on a laptop). FS-Cache sets defaults on this behavior, which can be configured via cache cull limits. For more information about configuring cache cull limits, see Cache cull limits configuration.

This procedure shows how to set up a cache.

Prerequisites

  • The cachefilesd package is installed and service has started successfully. To be sure the service is running, use the following command: 

    # systemctl start cachefilesd
# systemctl status cachefilesd

    The status must be active (running).

Procedure

  1. Configure in a cache back end which directory to use as a cache, use the following parameter: 

    $ dir /path/to/cache

  2. Typically, the cache back end directory is set in /etc/cachefilesd.conf as /var/cache/fscache, as in: 

    $ dir /var/cache/fscache

  3. If you want to change the cache back end directory, the selinux context must be same as /var/cache/fscache: 

    # semanage fcontext -a -e /var/cache/fscache /path/to/cache
# restorecon -Rv /path/to/cache
  4. Replace /path/to/cache with the directory name while setting up cache.

  5. If the given commands for setting selinux context did not work, use the following commands: 

    # semanage permissive -a cachefilesd_t
# semanage permissive -a cachefiles_kernel_t

    FS-Cache will store the cache in the file system that hosts /path/to/cache. On a laptop, it is advisable to use the root file system (/) as the host file system, but for a desktop machine it would be more prudent to mount a disk partition specifically for the cache.

  6. The host file system must support user-defined extended attributes. FS-Cache uses these attributes to store coherency maintenance information. To enable user-defined extended attributes on a device with ext3 file systems, enter: 

    # tune2fs -o user_xattr /dev/device

  7. To enable extended attributes for a file system at the mount time, as an alternative, use the following command: 

    # mount /dev/device /path/to/cache -o user_xattr

  8. Once the configuration file is in place, start up the cachefilesd service: 

    # systemctl start cachefilesd

  9. To configure cachefilesd to start at boot time, execute the following command as root: 

    # systemctl enable cachefilesd

9.4. Cache cull limits configuration

The cachefilesd daemon works by caching remote data from shared file systems to free space on the disk. This could potentially consume all available free space, which could be bad if the disk also contains the root partition. To control this, cachefilesd tries to maintain a certain amount of free space by discarding old objects, such as less-recently accessed objects, from the cache. This behavior is known as cache culling.

Cache culling is done on the basis of the percentage of blocks and the percentage of files available in the underlying file system. There are settings in /etc/cachefilesd.conf which control six limits:

brun N% (percentage of blocks), frun N% (percentage of files)
If the amount of free space and the number of available files in the cache rises above both these limits, then culling is turned off.
bcull N% (percentage of blocks), fcull N% (percentage of files)
If the amount of available space or the number of files in the cache falls below either of these limits, then culling is started.
bstop N% (percentage of blocks), fstop N% (percentage of files)
If the amount of available space or the number of available files in the cache falls below either of these limits, then no further allocation of disk space or files is permitted until culling has raised things above these limits again.

The default value of N for each setting is as follows:

  • brun/frun - 10%
  • bcull/fcull - 7%
  • bstop/fstop - 3%

When configuring these settings, the following must hold true:

  • 0 ≤ bstop < bcull < brun < 100
  • 0 ≤ fstop < fcull < frun < 100

These are the percentages of available space and available files and do not appear as 100 minus the percentage displayed by the df program.

Important

Culling depends on both bxxx and fxxx pairs simultaneously; the user can not treat them separately.

9.5. Retrieving statistical information from the fscache kernel module

FS-Cache also keeps track of general statistical information. This procedure shows how to get this information.

Procedure

  1. To view the statistical information about FS-Cache, use the following command: 

    # cat /proc/fs/fscache/stats

FS-Cache statistics includes information about decision points and object counters. For more information, see the following kernel document:

/usr/share/doc/kernel-doc-4.18.0/Documentation/filesystems/caching/fscache.txt

9.6. FS-Cache references

This section provides reference information for FS-Cache.

  1. For more information about cachefilesd and how to configure it, see man cachefilesd and man cachefilesd.conf. The following kernel documents also provide additional information:

    • /usr/share/doc/cachefilesd/README
    • /usr/share/man/man5/cachefilesd.conf.5.gz
    • /usr/share/man/man8/cachefilesd.8.gz

  2. For general information about FS-Cache, including details on its design constraints, available statistics, and capabilities, see the following kernel document:

    /usr/share/doc/kernel-doc-4.18.0/Documentation/filesystems/caching/fscache.txt