How to implement audit log rotation with compression based on time instead of size

Solution Verified - Updated -

Environment

  • Red Hat Enterprise Linux (All versions)
  • SELinux not in Enforcing mode

Issue

  • How to rotate audit logs daily?
  • Why audit logs are rotated after 6 MB of size? We want them to rotate based on a cron job like /var/log/messages.
  • How can we configure audit log compression?
  • What is the supported method for audit log rotation and compression?

Resolution

Disclaimer: The following information has been provided by Red Hat, but is outside the scope of the posted Service Level Agreements and support procedures. The information is provided as-is and any configuration settings or installed applications made from the information in this article could make the Operating System unsupported by Red Hat Global Support Services. The intent of this article is to provide information to accomplish the system's needs. Use of the information in this article at the user's own risk.

By default, auditd in all versions of Red Hat Enterprise Linux rotates its own log files automatically when they reach a certain size, as determined by the max_log_file setting in auditd.conf (which defaults to 6 megabytes)

Replacing auto-rotation based on size with auto-rotation based on time

  1. Disable rotation in /etc/audit/auditd.conf so that:

    max_log_file_action = ignore
    
  2. Tell auditd to reconfigure itself (applying your changes) by doing one of the following:
    kill -HUP $(pidof auditd)   (Any version)
    systemctl reload auditd   (RHEL7)
    service auditd reload   (RHEL6 and earlier)

  3. To manually trigger auditd to rotate, it needs to receive a USR1 signal
    Simple solution for daily rotation: copy auditd.cron to cron.daily

    ~]# cp /usr/share/doc/audit-*/auditd.cron /etc/cron.daily
    ~]# chmod +x /etc/cron.daily/auditd.cron
    ~]# cat /etc/cron.daily/auditd.cron
    #!/bin/sh
    
    ##########
    # This script can be installed to get a daily log rotation
    # based on a cron job.
    ##########
    
    /sbin/service auditd rotate
    EXITVALUE=$?
    if [ $EXITVALUE != 0 ]; then
        /usr/bin/logger -t auditd "ALERT exited abnormally with [$EXITVALUE]"
    fi
    exit 0
    

Implementing log compression

auditd does not support log compression; however, it's trivial to update the above script to rename old audit.log.n files and compress them. A working example is provided for demonstration purposes.

  1. Follow the steps above to disable auto-rotation based on size

  2. Replace the previously-created script with the following code:

    #!/bin/bash
    export PATH=/sbin:/bin:/usr/sbin:/usr/bin
    
    FORMAT="%F_%T"  # Customize timestamp format as desired, per `man date`
                    # %F_%T will lead to files like: audit.log.2015-02-26_15:43:46
    COMPRESS=gzip   # Change to bzip2 or xz as desired
    KEEP=5          # Number of compressed log files to keep
    ROTATE_TIME=5   # Amount of time in seconds to wait for auditd to rotate its logs. Adjust this as necessary
    
    rename_and_compress_old_logs() {
        for file in $(find /var/log/audit/ -name 'audit.log.[0-9]'); do
            timestamp=$(ls -l --time-style="+${FORMAT}" ${file} | awk '{print $6}')
            newfile=${file%.[0-9]}.${timestamp}
            # Optional: remove "-v" verbose flag from next 2 lines to hide output
            mv -v ${file} ${newfile}
            ${COMPRESS} -v ${newfile}
        done
    }
    
    delete_old_compressed_logs() {
        # Optional: remove "-v" verbose flag to hide output
        rm -v $(find /var/log/audit/ -regextype posix-extended -regex '.*audit\.log\..*(xz|gz|bz2)$' | sort -n | head -n -${KEEP})
    }
    
    rename_and_compress_old_logs
    service auditd rotate
    sleep $ROTATE_TIME
    rename_and_compress_old_logs
    delete_old_compressed_logs
    
  3. Modify the declarations of FORMAT, COMPRESS, and KEEP as desired

  4. Ensure the script is marked executable and set it to be called by cron at desired times (either via a normal cron job or by putting it in cron.daily as demonstrated above)

This solution is part of Red Hat’s fast-track publication program, providing a huge library of solutions that Red Hat engineers have created while supporting our customers. To give you the knowledge you need the instant it becomes available, these articles may be presented in a raw and unedited form.

17 Comments

This article needs to be updated (or there is a bug directly affecting this article) for recent versions of RHEL/auditd. Setting num_logs to 0 and then calling service auditd rotate does not rotate the logs. Num_logs must be 2 or greater.

Strangely, setting num_logs to 0 and max_log_file_action to ignore (which is not precisely instructed by this article) and then calling service auditd rotate results in auditd going bonkers - the logs aren't rotated, logging stops, 100% of one CPU is taken up by auditd, and you can no longer stop the process gracefully (kill -9 is required).

TL; DR - set num_logs to 2 and max_log_file_action to IGNORE (at least on EL 7). Everything else is good.

Jason,

Thanks for the post. I have been looking at this off and on for a week. I am sure you saved me a bunch of time.

IMO, auditd maxing out a CPU because of a setting combination is a bug, but someone else makes that call. This article really does need updated or the bug needs fixed.

Seth.

Hi Jason,

You are absolutely correct on this one. One more thing worth mentioning is that it does not work with SELinux in Enforcing mode.

Stepan

I implemented the log compression fix and script then I call it via cron.daily and it is working.
However, one annoyance is that every day anacron sends root and email saying: Rotating logs: [ OK ] rm: missing operand Try 'rm --help' for more information.

In the rotation & compression script, I am setting "KEEP=180" and I do not yet have 180 logs, so it is trying to remove nothing. Removing the "-v" flag does not hide this particular output.

Perhaps this "trivial" script should have a logical test "if [ something ]" before it attempts to delete, but I have not figured out what I can test. Any help is much appreciated.

Hi Clark,

the fault is in the removal

... | sort -n | head -n -${KEEP})

the script gives 2 parameters to head in place of 1. Correct line is:

rm -v $(find /var/log/audit/ -regextype posix-extended -regex '.*audit\.log\..*(xz|gz|bz2)$' | sort -n | head -n ${KEEP})

or

rm -v $(find /var/log/audit/ -regextype posix-extended -regex '.*audit\.log\..*(xz|gz|bz2)$' | sort -n | !!!head -${KEEP})

Well spotted! Working on a fix. EDIT: Fixed

Thank you Stepan Broz, however, are you sure that it is fixed? Please correct me if I am wrong, but I think that the only thing you changed was removing the - that proceeded ${KEEP}. In other words, you changed: head -n -${KEEP} to instead be head -n ${KEEP} That change does not result in deleting all but the most recent ${KEEP} files.

You are right, Clark. I have rolled back the latest change. Please understand that the script above is meant to be an example only, it does not work for everyone.

How to configure audit log rotation & compression with SELinux in Enforcing mode.

For SELinux, you will need to compile and install a module. This definition should help:

module logrotate-auditd 1.0;

require {
        type auditd_etc_t;
        type logrotate_t;
        type auditd_log_t;
   type auditd_unit_file_t;
        class file { create getattr ioctl open read rename setattr unlink write };
        class dir { add_name read remove_name write };
   class service { start stop status reload kill};
}

#============= logrotate_t ==============
allow logrotate_t auditd_etc_t:file getattr;
allow logrotate_t auditd_log_t:dir { read write add_name remove_name };
allow logrotate_t auditd_log_t:file { create ioctl open read rename getattr setattr unlink write };
allow logrotate_t auditd_unit_file_t:service { start stop status reload kill };

Hey,

In the article :

kill -HUP $(pidof auditd) (Any version) systemctl reload auditd (RHEL7)

service auditd reload (RHEL6 and earlier)

On RHEL 7.6, no one works ... But found in comment : need to call "service auditd rotate". You should update this part.

EDIT: my bad, was indeed to reload config, not rotate ..

Regards, Laurent

A little modification to consider more audit logs:

#!/bin/sh
##########
# This script can be installed to get a daily log rotation
# based on a cron job.
##########
export PATH=/sbin:/bin:/usr/sbin:/usr/bin

FORMAT="%F_%T"  # Customize timestamp format as desired, per `man date`
                # %F_%T will lead to files like: audit.log.2015-02-26_15:43:46
COMPRESS=gzip   # Change to bzip2 or xz as desired
KEEP=10          # Number of compressed log files to keep
ROTATE_TIME=5   # Amount of time in seconds to wait for auditd to rotate its logs. Adjust this as necessary

rename_and_compress_old_logs() {
    for file in $(find /var/log/audit/ -regextype posix-extended -regex '.*audit.log.[0-9]{1,}$'); do
        timestamp=$(ls -l --time-style="+${FORMAT}" ${file} | awk '{print $6}')
        newfile=${file%.[0-9]}.${timestamp}
        # Optional: remove "-v" verbose flag from next 2 lines to hide output
        mv -v ${file} ${newfile}
        ${COMPRESS} -v ${newfile}
    done
}

delete_old_compressed_logs() {
    # Optional: remove "-v" verbose flag to hide output
    rm -v $(find /var/log/audit/ -regextype posix-extended -regex '.*audit\.log\..*(xz|gz|bz2)$' | sort -n | head -n -${KEEP})
}

rename_and_compress_old_logs
service auditd rotate
sleep $ROTATE_TIME
rename_and_compress_old_logs
delete_old_compressed_logs

Changing into the find command the argument with regular expression: -- This line :

for file in $(find /var/log/audit/ -name 'audit.log.[0-9]'); do

++ For this line:

for file in $(find /var/log/audit/ -regextype posix-extended -regex '.*audit.log.[0-9]{1,}$'); do

At first I ran the command as kill -HUP $(pidof auditd) but it certainly didn't work.

So, I ran ps -ef | grep auditd root 118 2 0 Dec24 ? 00:00:00 [kauditd] root 11738 1 0 16:40 ? 00:00:00 /sbin/auditd root 11802 10551 0 16:50 pts/0 00:00:00 grep --color=auto auditd

then took the process id and ran Kill -9 11738 I have Linux 3.10.0-957.10.1.el7.x86_64, I ran systemctl reload auditd, it failed so I ran systemctl reload auditd.service then ran systemctl status auditd.service and it is working.

For what I am doing, I don't need the files compress yet. So I didn't do the compression option. Will test the rotation first, once it works then I will implement the compression later.

Thanks for the instruction

I removed %.[0-9] from newfile variable because of a small bug for one digit numbers:

'/tmp/tmplogs/audit.log.6' -> '/tmp/tmplogs/audit.log.2021-01-07_11:52:16'
gzip: /tmp/tmplogs/audit.log.2021-01-07_11:52:16.gz already exists; do you wish to overwrite (y or n)?

Fix:

newfile=${file}.${timestamp}

I see small issues in the script, my proposals:

1) to support >2 digits numbers

newfile=${file%.[0-9]}.${timestamp}

could be replaced with

newfile=${file%.*}.${timestamp}

2) implement Ricardo's fix:

[. . . . .]
for file in $(find /var/log/audit/ -type f -regextype posix-extended -regex '.*audit.log.[0-9]{1,}$'); do
[. . . . .]

3) implement solution for rm issue reported by Gary, maybe such replacement for existing lines in the script:

toclean="$(find /var/log/audit/ -type f -regextype posix-extended -regex '.*audit\.log\..*(xz|gz|bz2)$' | sort -n | head -n -${KEEP})"
[ -s "${toclean}" ] && rm -v ${toclean}

or

toclean="$(find /var/log/audit/ -type f -regextype posix-extended -regex '.*audit\.log\..*(xz|gz|bz2)$' | sort -n | head -n -${KEEP})"
[ -s "${toclean}" ] && rm -v $(find /var/log/audit/ -type f -regextype posix-extended -regex '.*audit\.log\..*(xz|gz|bz2)$' | sort -n | head -n -${KEEP})

Hi,

is there any reason to not use just logrotate and insert an own script?

Thomas

Am I missing an option to configure auditd delete old audit log files? I only see rotation. If not, that's just poor programming.
Certainly in some situations, you would rather shutdown the server than lose an audit log. But those cases are the minority. Most places would like to keep a reasonable amount of logs (one or two months) online and delete the old ones rather than filling up the filesystem. You can pull the really old ones from tape if you have to. A well-behaved program will not force sysadmins to hack together their own scripts to compress and delete logs that your program created.