Chapter 7. Ceph File System quotas

As a storage administrator, you can view, set, and remove quotas on any directory in the file system. You can place quota restrictions on the number of bytes or the number of files within the directory.

7.1. Prerequisites

  • A running, and healthy Red Hat Ceph Storage cluster.
  • Deployment of a Ceph File System.
  • Make sure that the attr package is installed.

7.2. Ceph File System quotas

The Ceph File System (CephFS) quotas allow you to restrict the number of bytes or the number of files stored in the directory structure. Ceph File System quotas are fully supported using a FUSE client or using Kernel clients, version 4.17 or newer.

Limitations

  • CephFS quotas rely on the cooperation of the client mounting the file system to stop writing data when it reaches the configured limit. However, quotas alone cannot prevent an adversarial, untrusted client from filling the file system.
  • Once processes that write data to the file system reach the configured limit, a short period of time elapses between when the amount of data reaches the quota limit, and when the processes stop writing data. The time period generally measures in the tenths of seconds. However, processes continue to write data during that time. The amount of additional data that the processes write depends on the amount of time elapsed before they stop.
  • When using path-based access restrictions, be sure to configure the quota on the directory to which the client is restricted, or to a directory nested beneath it. If the client has restricted access to a specific path based on the MDS capability, and the quota is configured on an ancestor directory that the client cannot access, the client will not enforce the quota. For example, if the client cannot access the /home/ directory and the quota is configured on /home/, the client cannot enforce that quota on the directory /home/user/.
  • Snapshot file data that has been deleted or changed does not count towards the quota.
  • No support for quotas with NFS clients when using setxattr, and no support for file-level quotas on NFS. To use quotas on NFS shares, you can export them using subvolumes and setting the --size option.

7.3. Viewing quotas

Use the getfattr command and the ceph.quota extended attributes to view the quota settings for a directory.

Note

If the attributes appear on a directory inode, then that directory has a configured quota. If the attributes do not appear on the inode, then the directory does not have a quota set, although its parent directory might have a quota configured. If the value of the extended attribute is 0, the quota is not set.

Prerequisites

  • Root-level access to the Ceph client node.
  • The attr package is installed.

Procedure

  1. To view CephFS quotas.

    1. Using a byte-limit quota:

      Syntax

      getfattr -n ceph.quota.max_bytes DIRECTORY

      Example

      [root@client ~]# getfattr -n ceph.quota.max_bytes /mnt/cephfs/
      getfattr: Removing leading '/' from absolute path names
      # file: mnt/cephfs/
      ceph.quota.max_bytes="100000000"

      In this example, 100000000 equals 100 MB.

    2. Using a file-limit quota:

      Syntax

      getfattr -n ceph.quota.max_files DIRECTORY

      Example

      [root@client ~]# getfattr -n ceph.quota.max_files /mnt/cephfs/
      getfattr: Removing leading '/' from absolute path names
      # file: mnt/cephfs/
      ceph.quota.max_files="10000"

      In this example, 10000 equals 10,000 files.

Additional Resources

  • See the getfattr(1) manual page for more information.

7.4. Setting quotas

This section describes how to use the setfattr command and the ceph.quota extended attributes to set the quota for a directory.

Prerequisites

  • Root-level access to the Ceph client node.
  • The attr package is installed.

Procedure

  1. To set CephFS quotas.

    1. Using a byte-limit quota:

      Syntax

      setfattr -n ceph.quota.max_bytes -v 100000000 DIRECTORY

      Example

      [root@client ~]# setfattr -n ceph.quota.max_bytes -v 100000000 /cephfs/

      In this example, 100000000 bytes equals 100 MB.

    2. Using a file-limit quota:

      Syntax

      setfattr -n ceph.quota.max_files -v 10000 DIRECTORY

      Example

      [root@client ~]# setfattr -n ceph.quota.max_files -v 10000 /cephfs/

      In this example, 10000 equals 10,000 files.

Additional Resources

  • See the setfattr(1) manual page for more information.

7.5. Removing quotas

This section describes how to use the setfattr command and the ceph.quota extended attributes to remove a quota from a directory.

Prerequisites

  • Root-level access to the Ceph client node.
  • Make sure that the attr package is installed.

Procedure

  1. To remove CephFS quotas.

    1. Using a byte-limit quota:

      Syntax

      setfattr -n ceph.quota.max_bytes -v 0 DIRECTORY

      Example

      [root@client ~]# setfattr -n ceph.quota.max_bytes -v 0 /mnt/cephfs/

    2. Using a file-limit quota:

      Syntax

      setfattr -n ceph.quota.max_files -v 0 DIRECTORY

      Example

      [root@client ~]# setfattr -n ceph.quota.max_files -v 0 /mnt/cephfs/

Additional Resources

  • See the setfattr(1) manual page for more information.

7.6. Additional Resources

  • See the Deployment of the Ceph File System section in the Red Hat Ceph Storage File System Guide.
  • See the getfattr(1) manual page for more information.
  • See the setfattr(1) manual page for more information.