What is the maximum value and default value for fs.nr_open in Red Hat Enterprise Linux?

Solution Verified - Updated -

Environment

  • Red Hat Enterprise Linux 7
  • Red Hat Enterprise Linux 8
  • Red Hat Enterprise Linux 9

Issue

  • What is the default value of fs.nr_open?
  • What is the maximum value of fs.nr_open?

Resolution

  • The default value fs.nr_open is 1024*1024 = 1048576 defined in kernel code. Below is the snippet from source code.

    fs/file.c

  27 unsigned int sysctl_nr_open __read_mostly = 1024*1024;

  • The maximum value of fs.nr_open is limited to sysctl_nr_open_max in kernel, which is 2147483584 on x86_64.

Note: The value of "Max open files"(ulimit -n) is limited to fs.nr_open value.

  • In RHEL9, the default value of fs.nr_open is way larger than what is defined by kernel.

    # cat /proc/sys/fs/nr_open 
1073741816

Root Cause

  • The max value gets calculated from following piece of code and assigned to sysctl_nr_open_max variable.

    fs/file.c

  27 unsigned int sysctl_nr_open __read_mostly = 1024*1024;
  28 unsigned int sysctl_nr_open_min = BITS_PER_LONG;
  29 /* our min() is unusable in constant expressions ;-/ */
  30 #define __const_min(x, y) ((x) < (y) ? (x) : (y))
  31 unsigned int sysctl_nr_open_max =
  32         __const_min(INT_MAX, ~(size_t)0/sizeof(void *)) & -BITS_PER_LONG;

  • Lets evaluate this,

    include/vdso/limits.h
  8 #define INT_MAX         ((int)(~0U >> 1))

crash> p ((int)(~0U >> 1))
$39 = 2147483647

crash> p ~(size_t)0/sizeof(void *)
$40 = 2305843009213693951

include/asm-generic/bitsperlong.h

  8 #ifdef CONFIG_64BIT
  9 #define BITS_PER_LONG 64   <=----
 10 #else
 11 #define BITS_PER_LONG 32
 12 #endif /* CONFIG_64BIT */

__const_min(2147483647,2305843009213693951) & -64

crash> p 2147483647&-64
$41 = 2147483584    [max value]

  • The reason behind large value of nr_open in RHEL 9 is due to following commit from systemd.

Diagnostic Steps

  • Below is demonstration on how ulimit depends on nr_open value and testing max value of nr_open.

    # ulimit -n 2147483584
-bash: ulimit: open files: cannot modify limit: Operation not permitted

# sysctl fs.nr_open
fs.nr_open = 1048576

# sysctl -w fs.nr_open=2147483584
fs.nr_open = 2147483584

# ulimit -n 2147483584

# ulimit -n
2147483584

# sysctl -w fs.nr_open=2147483585
sysctl: setting key "fs.nr_open": Invalid argument
fs.nr_open = 2147483585

  • On RHEL 9 if system is booted with systemd.log_level=debug we can see that systemd modifies fs.nr_open value.

    # journalctl -b|grep nr_open
Mar 10 01:40:34 localhost systemd[1]: Setting 'fs/nr_open' to '2147483640'.
Mar 10 01:40:34 localhost systemd[1]: Couldn't write fs.nr_open as 2147483640, halving it.
Mar 10 01:40:34 localhost systemd[1]: Setting 'fs/nr_open' to '1073741816'.
Mar 10 01:40:34 localhost systemd[1]: Successfully bumped fs.nr_open to 1073741816
Mar 10 01:40:36 localhost systemd[1]: Setting 'fs/nr_open' to '2147483640'.
Mar 10 01:40:36 localhost systemd[1]: Couldn't write fs.nr_open as 2147483640, halving it.

This solution is part of Red Hat’s fast-track publication program, providing a huge library of solutions that Red Hat engineers have created while supporting our customers. To give you the knowledge you need the instant it becomes available, these articles may be presented in a raw and unedited form.

Comments