Chapter 5. Configuring kernel parameters at runtime
As a system administrator, you can modify many facets of the Red Hat Enterprise Linux kernel’s behavior at runtime. This section describes how to configure kernel parameters at runtime by using the sysctl command and by modifying the configuration files in the /etc/sysctl.d/ and /proc/sys/ directories.
5.1. What are kernel parameters
Kernel parameters are tunable values which you can adjust while the system is running. There is no requirement to reboot or recompile the kernel for changes to take effect.
It is possible to address the kernel parameters through:
+ * The sysctl command
+ * The virtual file system mounted at the /proc/sys/ directory
+ * The configuration files in the /etc/sysctl.d/ directory
Tunables are divided into classes by the kernel subsystem. Red Hat Enterprise Linux has the following tunable classes:
Table 5.1. Table of sysctl classes
| Tunable class | Subsystem |
|---|---|
| abi | Execution domains and personalities |
| crypto | Cryptographic interfaces |
| debug | Kernel debugging interfaces |
| dev | Device-specific information |
| fs | Global and specific file system tunables |
| kernel | Global kernel tunables |
| net | Network tunables |
| sunrpc | Sun Remote Procedure Call (NFS) |
| user | User Namespace limits |
| vm | Tuning and management of memory, buffers, and cache |
Additional resources
-
For more information about
sysctl, seesysctl(8)manual pages. -
For more information about
/etc/sysctl.d/see,sysctl.d(5)manual pages.
5.2. Setting kernel parameters at runtime
Configuring kernel parameters on a production system requires careful planning. Unplanned changes may render the kernel unstable, requiring a system reboot. Verify that you are using valid options before changing any kernel values.
5.2.1. Configuring kernel parameters temporarily with sysctl
The following procedure describes how to use the sysctl command to temporarily set kernel parameters at runtime. The command is also useful for listing and filtering tunables.
Prerequisites
- Kernel parameters introduction
- Root permissions
Procedure
To list all parameters and their values, use the following:
# sysctl -aNoteThe
# sysctl -acommand displays kernel parameters, which can be adjusted at runtime and at boot time.To configure a parameter temporarily, use the command as in the following example:
# sysctl <TUNABLE_CLASS>.<PARAMETER>=<TARGET_VALUE>The sample command above changes the parameter value while the system is running. The changes take effect immediately, without a need for restart.
NoteThe changes return back to default after your system reboots.
Additional resources
-
For more information about
sysctl, see thesysctl(8)manual page. -
To permanently modify kernel parameters, either use the
sysctlcommand to write the values to the/etc/sysctl.conffile or make manual changes to the configuration files in the/etc/sysctl.d/directory.
5.2.2. Configuring kernel parameters permanently with sysctl
The following procedure describes how to use the sysctl command to permanently set kernel parameters.
Prerequisites
- Kernel parameters introduction
- Root permissions
Procedure
To list all parameters, use the following:
# sysctl -aThe command displays all kernel parameters that can be configured at runtime.
To configure a parameter permanently:
# sysctl -w <TUNABLE_CLASS>.<PARAMETER>=<TARGET_VALUE> >> /etc/sysctl.confThe sample command changes the tunable value and writes it to the
/etc/sysctl.conffile, which overrides the default values of kernel parameters. The changes take effect immediately and persistently, without a need for restart.
To permanently modify kernel parameters you can also make manual changes to the configuration files in the /etc/sysctl.d/ directory.
Additional resources
-
For more information about
sysctl, see thesysctl(8)andsysctl.conf(5)manual pages. -
For more information about using the configuration files in the
/etc/sysctl.d/directory to make permanent changes to kernel parameters, see Using configuration files in /etc/sysctl.d/ to adjust kernel parameters section.
5.2.3. Using configuration files in /etc/sysctl.d/ to adjust kernel parameters
The following procedure describes how to manually modify configuration files in the /etc/sysctl.d/ directory to permanently set kernel parameters.
Prerequisites
- Kernel parameters introduction
- Root permissions
Procedure
Create a new configuration file in
/etc/sysctl.d/:# vim /etc/sysctl.d/<some_file.conf>Include kernel parameters, one per line, as follows:
<TUNABLE_CLASS>.<PARAMETER>=<TARGET_VALUE><TUNABLE_CLASS>.<PARAMETER>=<TARGET_VALUE>- Save the configuration file.
Reboot the machine for the changes to take effect.
Alternatively, to apply changes without rebooting, execute:
# sysctl -p /etc/sysctl.d/<some_file.conf>The command enables you to read values from the configuration file, which you created earlier.
Additional resources
-
For more information about
sysctl, see thesysctl(8)manual page. -
For more information about
/etc/sysctl.d/, see thesysctl.d(5)manual page.
5.2.4. Configuring kernel parameters temporarily through /proc/sys/
The following procedure describes how to set kernel parameters temporarily through the files in the virtual file system /proc/sys/ directory.
Prerequisites
- Kernel parameters introduction
- Root permissions
Procedure
Identify a kernel parameter you want to configure:
# ls -l /proc/sys/<TUNABLE_CLASS>/The writable files returned by the command can be used to configure the kernel. The files with read-only permissions provide feedback on the current settings.
Assign a target value to the kernel parameter:
# echo <TARGET_VALUE> > /proc/sys/<TUNABLE_CLASS>/<PARAMETER>The command makes configuration changes that will disappear once the system is restarted.
Optionally, verify the value of the newly set kernel parameter:
# cat /proc/sys/<TUNABLE_CLASS>/<PARAMETER>
Additional resources
-
To permanently modify kernel parameters, either use the
sysctlcommand or make manual changes to the configuration files in the/etc/sysctl.d/directory.
5.3. Keeping kernel panic parameters disabled in virtualized environments
When configuring a virtualized environment in Red Hat Enterprise Linux 8 (RHEL 8), you should not enable the softlockup_panic and nmi_watchdog kernel parameters, as the virtualized environment may trigger a spurious soft lockup that should not require a system panic.
The following sections explain the reasons behind this advice by summarizing:
- What causes a soft lockup.
- Describing the kernel parameters that control a system’s behavior on a soft lockup.
- Explaining how soft lockups may be triggered in a virtualized environment.
5.3.1. What is a soft lockup
A soft lockup is a situation usually caused by a bug, when a task is executing in kernel space on a CPU without rescheduling. The task also does not allow any other task to execute on that particular CPU. As a result, a warning is displayed to a user through the system console. This problem is also referred to as the soft lockup firing.
Additional resources
- For a technical reason behind a soft lockup, example log messages, and other details, see the following Knowledge Article.
5.3.2. Parameters controlling kernel panic
The following kernel parameters can be set to control a system’s behavior when a soft lockup is detected.
- softlockup_panic
Controls whether or not the kernel will panic when a soft lockup is detected.
Type Value Effect Integer
0
kernel does not panic on soft lockup
Integer
1
kernel panics on soft lockup
By default, on RHEL8 this value is 0.
In order to panic, the system needs to detect a hard lockup first. The detection is controlled by the
nmi_watchdogparameter.- nmi_watchdog
Controls whether lockup detection mechanisms (
watchdogs) are active or not. This parameter is of integer type.Value Effect 0
disables lockup detector
1
enables lockup detector
The hard lockup detector monitors each CPU for its ability to respond to interrupts.
- watchdog_thresh
Controls frequency of watchdog
hrtimer, NMI events, and soft/hard lockup thresholds.Default threshold Soft lockup threshold 10 seconds
2 *
watchdog_threshSetting this parameter to zero disables lockup detection altogether.
Additional resources
-
For further information about
nmi_watchdogandsoftlockup_panic, see the Softlockup detector and hardlockup detector document. -
For more details about
watchdog_thresh, see the Kernel sysctl document.
5.3.3. Spurious soft lockups in virtualized environments
The soft lockup firing on physical hosts, as described in Section 5.3.1, “What is a soft lockup”, usually represents a kernel or hardware bug. The same phenomenon happening on guest operating systems in virtualized environments may represent a false warning.
Heavy work-load on a host or high contention over some specific resource such as memory, usually causes a spurious soft lockup firing. This is because the host may schedule out the guest CPU for a period longer than 20 seconds. Then when the guest CPU is again scheduled to run on the host, it experiences a time jump which triggers due timers. The timers include also watchdog hrtimer, which can consequently report a soft lockup on the guest CPU.
Because a soft lockup in a virtualized environment may be spurious, you should not enable the kernel parameters that would cause a system panic when a soft lockup is reported on a guest CPU.
To understand soft lockups in guests, it is essential to know that the host schedules the guest as a task, and the guest then schedules its own tasks.
Additional resources
- For soft lockup definition and technicalities behind its functioning, see Section 5.3.1, “What is a soft lockup”.
- To learn about components of RHEL 8 virtualized environments and their interaction, see RHEL 8 virtual machine components and their interaction.
5.4. Adjusting kernel parameters for database servers
There are different sets of kernel parameters which can affect performance of specific database applications. The following sections explain what kernel parameters to configure to secure efficient operation of database servers and databases.
5.4.1. Introduction to database servers
A database server is a hardware device which has a certain amount of main memory, and a database (DB) application installed. This DB application provides services as a means of writing the cached data from the main memory, which is usually small and expensive, to DB files (database). These services are provided to multiple clients on a network. There can be as many DB servers as a machine’s main memory and storage allows.
Red Hat Enterprise Linux 8 provides the following database applications:
- MariaDB 10.3
- MySQL 8.0
- PostgreSQL 10
- PostgreSQL 9.6
- PostgreSQL 12 - available since RHEL 8.1.1
5.4.2. Parameters affecting performance of database applications
The following kernel parameters affect performance of database applications.
- fs.aio-max-nr
Defines the maximum number of asynchronous I/O operations the system can handle on the server.
NoteRaising the
fs.aio-max-nrparameter produces no additional changes beyond increasing the aio limit.- fs.file-max
Defines the maximum number of file handles (temporary file names or IDs assigned to open files) the system supports at any instance.
The kernel dynamically allocates file handles whenever a file handle is requested by an application. The kernel however does not free these file handles when they are released by the application. The kernel recycles these file handles instead. This means that over time the total number of allocated file handles will increase even though the number of currently used file handles may be low.
- kernel.shmall
-
Defines the total number of shared memory pages that can be used system-wide. To use the entire main memory, the value of the
kernel.shmallparameter should be ≤ total main memory size. - kernel.shmmax
- Defines the maximum size in bytes of a single shared memory segment that a Linux process can allocate in its virtual address space.
- kernel.shmmni
- Defines the maximum number of shared memory segments the database server is able to handle.
- net.ipv4.ip_local_port_range
- Defines the port range the system can use for programs which want to connect to a database server without a specific port number.
- net.core.rmem_default
- Defines the default receive socket memory through Transmission Control Protocol (TCP).
- net.core.rmem_max
- Defines the maximum receive socket memory through Transmission Control Protocol (TCP).
- net.core.wmem_default
- Defines the default send socket memory through Transmission Control Protocol (TCP).
- net.core.wmem_max
- Defines the maximum send socket memory through Transmission Control Protocol (TCP).
- vm.dirty_bytes / vm.dirty_ratio
-
Defines a threshold in bytes / in percentage of dirty-able memory at which a process generating dirty data is started in the
write()function.
Either vm.dirty_bytes or vm.dirty_ratio can be specified at a time.
- vm.dirty_background_bytes / vm.dirty_background_ratio
- Defines a threshold in bytes / in percentage of dirty-able memory at which the kernel tries to actively write dirty data to hard-disk.
Either vm.dirty_background_bytes or vm.dirty_background_ratio can be specified at a time.
- vm.dirty_writeback_centisecs
Defines a time interval between periodic wake-ups of the kernel threads responsible for writing dirty data to hard-disk.
This kernel parameters measures in 100th’s of a second.
- vm.dirty_expire_centisecs
Defines the time after which dirty data is old enough to be written to hard-disk.
This kernel parameters measures in 100th’s of a second.
Additional resources
- For explanation of dirty data writebacks, how they work, and what kernel parameters relate to them, see the Dirty pagecache writeback and vm.dirty parameters document.