Is there any system performance penalty to enable auditing ?
Environment
- Red Hat Enterprise Linux 8
- Red Hat Enterprise Linux 7
- Red Hat Enterprise Linux 6
- Red Hat Enterprise Linux 5
Issue
- Is there any system
performance
penalty to enableauditing
- Is there any alternatives to
audit
to trace akiller
, that have less impact on systemperformance
to tracekiller
?
Resolution
- In order to
trace
the event before happening, the only way is setting theaudit
rule. - If using
64 bit
softwares in the system, andkill
is the onlysyscall
that iskilling
yourprocess
then the correct rule should be:
# /sbin/auditctl -a exit,always -F arch=b64 -S kill -S tkill -S tgkill -F a1=10 -k signal10
- Adding a rule for both
32 bit
and64 bit
is likely to add overhead without any benefit consideringperformance
of the system. - If system is running both
32 bit
and64 bit
compiledprocesses
, addaudit
rule for both. - There is also performance impact to capture more rules for syscalls at once as below.
-a exit,always -F arch=b64 -S execve -S exit -S exit_group -S fork -S clone -F auid!=4294967295 -k MSexecve64
-a exit,always -F arch=b32 -S execve -S exit -S exit_group -S fork -S clone -F auid!=4294967295 -k MSexecve32
- Please check if stopping auditd helps for the performance by reading How to stop and disable auditd on RHEL 7? .
Root Cause
- The
auditd
is the userspace component to the Linux Auditing System. It's responsible for writing audit records to the disk. while a number of audit rules are configured, high %sys of CPUs are utilised accordingly. As long as lots of syscalls are captured, lots of logs such as flooding are also written to the disk. If the disk is type of spinning HDD it takes longer to write it down, SSD is much faster.
Diagnostic Steps
- audit performance impact test while running test script.
- P:rhel 7.7 / Intel(R) Core(TM) i7-4790 CPU @ 3.60GHz - SSD
elapsed 0.6 user 0.4 sys 0.2 CPU 104.33 - no audit rule
elapsed 4.4 user 0.5 sys 0.4 CPU 20.90 - audit rule
- P:rhel 7.7 / Intel(R) Core(TM) i7-3740QM CPU @ 2.70GHz - Spinning disk, HDD
elapsed 0.7 user 0.4 sys 0.2 CPU 100.65
elapsed 39.5 user 0.5 sys 0.4 CPU 2.66 - audit rule
This solution is part of Red Hat’s fast-track publication program, providing a huge library of solutions that Red Hat engineers have created while supporting our customers. To give you the knowledge you need the instant it becomes available, these articles may be presented in a raw and unedited form.
Comments