I am getting 'logrotate: ALERT exited abnormally with [1]' messages in logs when SELinux is in the Enforcing mode

Solution Verified - Updated -

Environment

  • Red Hat Enterprise Linux
  • SELinux in Enforcing mode

Issue

  • /var/log/messages show that logrotate fails with error ALERT exited abnormally with [1]
  • logrotate fails to rotate application logs

Resolution

  • To get rid of the message ALERT exited abnormally with [1], check if logrotate is being used to rotate logs other than those in /var/log. If so, then SELinux can be the cause of this issue. The directories outside of /var/log should have the same context as /var/log has. Set the following SELinux context on the directories where logrotate should rotate the logs.

Example: logrotate has been configured to rotate files in /backup/mysql

  • Run the following command:
# semanage fcontext -a -t var_log_t '/backup/mysql(/.*)?'
#

The above command will define the context that would be automatically set on new files under /backup/mysql. This definition will be stored in /etc/selinux/targeted/contexts/files/file_contexts.local, so that the changes will be persistent. This can be verified by looking into the file:

# cat /etc/selinux/targeted/contexts/files/file_contexts.local
# This file is auto-generated by libsemanage
# Do not edit directly.

/backup/mysql(/.*)?    system_u:object_r:var_log_t:s0
#
  • Run the following command to recursively set the context for files under /backup/mysql according to the newly defined definition:
# restorecon -Frvv /backup/mysql
restorecon reset /backup/mysql context system_u:object_r:default_t:s0->system_u:object_r:var_log_t:s0
restorecon reset /backup/mysql/backup.tar context system_u:object_r:default_t:s0->system_u:object_r:var_log_t:s0
#
  • If SELinux is not in the Enforcing mode, this solution does not apply. Please, refer to this article in order to get more information on logrotate troubleshooting.

Root Cause

  • SELinux denies logrotate to check the attributes of the rotated log file, this happens when logrotate has to rotate files outside of /var/log (defined in logrotate.conf).

  • logrotate indeed sends a logger message logrotate: ALERT exited abnormally with [1] because of SELinux, but it doesn't stop logrotate from completing successfully. The reason is that the /etc/cron.daily/logrotate script checks if the logrotate command ran succesfully by checking the exit status of the logrotate command. This is non-zero not because logrotate was not able to run, but because logrotate was trying to get the attributes (permissions) on the rotated file created in a custom directory, which SELinux denied by the following message:

    CALL msg=audit(1335727501.512:1389214): arch=c000003e syscall=4 success=no exit=-13 a0=12ec760 a1=7fffdba0de60 a2=7fffdba0de60 a3=b 
items=0 ppid=14260 pid=14262 auid=0 uid=0 gid=0 euid=0 suid=0 fsuid=0 egid=0 sgid=0 fsgid=0 tty=(none) ses=44399 comm="logrotate" 
exe="/usr/sbin/logrotate" subj=system_u:system_r:logrotate_t:s0-s0:c0.c1023 key=(null)

type=AVC msg=audit(1335727501.512:1389214): avc:  denied  { getattr } for  pid=14262 comm="logrotate" path="/backup/mysql/backup.gz" 
dev=dm-0 ino=131206 scontext=system_u:system_r:logrotate_t:s0-s0:c0.c1023 tcontext=unconfined_u:object_r:default_t:s0 tclass=file

  • In the environments where logrotate.conf or the /etc/logrotate.d directory has custom logrotate scripts where logrotate has to rotate files out of the /var/log directory.

Example:

    /backup/mysql/backup.tar {
            missingok
            daily
            nocompress
            rotate 5
    }
  • In the above example logrotate rotates /backup/mysql/backup.tar daily. The above file by default would have the SELinux context unconfined_u:object_r:default_t:s0. This would cause SELinux to deny logrotate to get an attribute list (getattr). For logrotate to get getattr on the files created in the /backup/mysql directory. The files should have the var_log_t context.

Diagnostic Steps

  • Check audit log denials for logrotate process in /var/log/audit/audit.log
   # ausearch -c logrotate -m AVC

  • Check if /etc/logrotate.conf or the /etc/logrotate.d directory has custom scripts that require logrotate to rotate files from directories outside of /var/log.

  • Check the SELinux context on those custom directories. They should have the "var_log_t" type on those files.

  • The message logrotate: ALERT exited abnormally with [1] comes from the /etc/cron.daily/logrotate script:

#!/bin/sh

/usr/sbin/logrotate /etc/logrotate.conf >/dev/null 2>&1
EXITVALUE=$?
if [ $EXITVALUE != 0 ]; then
    /usr/bin/logger -t logrotate "ALERT exited abnormally with [$EXITVALUE]"
fi
exit 0

This solution is part of Red Hat’s fast-track publication program, providing a huge library of solutions that Red Hat engineers have created while supporting our customers. To give you the knowledge you need the instant it becomes available, these articles may be presented in a raw and unedited form.

Comments