Rsyslog stops revciving logs

Latest response

Hi all,

I have a problem regarding a rsyslog. We have a Rhel7 that is used for receiving all the log of network devices. Firts we had a lot of problems rotating the logs, but finally we solve it with your great help!

Now, the problem it's different. I don't know why but suddently at 3:15 AM rsyslog stops storaging logs.

When we arrive at the morning the procedure is very simply:

systemctl restart rsyslog.service

And it works perfectly.

We have verified the cron

[root@syslogcom1 morabanc]# crontab -l
0 5 * * * /usr/sbin/aide --check

And no procceses are scheduled at that hour.

Do you know what I can verify? Could you be so kind to give me some help?

Many thanks in advance,
Regards,
Roger

Responses

Hello Roger,

As per your details, I could understand that the rsyslog service stops generating logs @3:15am and hence, there are no logs captured in messages file after this time stamp? It would start to work after the restart of the service. Is this correct?

  • Check if you could notice any errors or activities being noticed before rsyslog daemon stops generating logs.
  • Check the respective /etc/rsyslog.conf file and corresponding include files /etc/rsyslog.d/.. (you may post the contents here as well).
  • Also, tell us about the redhat release, kernel version and rsyslog package version as well.

Hello Roger,

Configuration files to add to Sadashiva's great list:

  • /etc/logrotate.conf

  • /etc/logrotate.d/*

If logrotate fails to complete due to a configuration error rsyslog does not get restarted.

Regards,

Jan Gerrit

Hi Sadashiva,

Many thanks for your answer. Yes, you understood right. At 3:15AM everyday, all the logs are not being received. Then log files are empty since we restart the rsyslog.service. When we type the command, automatically starts working fine.

  • Check if you could notice any errors or activities being noticed before rsyslog daemon stops generating logs. It seems all right at this time.

  • Check the respective /etc/rsyslog.conf file and corresponding include files /etc/rsyslog.d/.. (you may post the contents here as well).

[root@syslogcom1 logrotate.d]# cd /etc/rsyslog.d/
[root@syslogcom1 rsyslog.d]# ls
listen.conf
[root@syslogcom1 rsyslog.d]# cat listen.conf
$SystemLogSocketName /run/systemd/journal/syslog
  • Also, tell us about the redhat release, kernel version and rsyslog package version as well.
[root@syslogcom1 rsyslog.d]# cat /etc/redhat-release
Red Hat Enterprise Linux Server release 7.7 (Maipo)

[root@syslogcom1 rsyslog.d]# uname -r
3.10.0-1062.el7.x86_64

/etc/logrotate.conf

# see "man logrotate" for details
# rotate log files weekly
weekly

# keep 4 weeks worth of backlogs
rotate 4

# create new (empty) log files after rotating old ones
create

# use date as a suffix of the rotated file
#dateext

# uncomment this if you want your log files compressed
#compress

# RPM packages drop log rotation information into this directory
include /etc/logrotate.d

# no packages own wtmp and btmp -- we'll rotate them here
/var/log/wtmp {
    missingok
    monthly
    create 0664 root utmp
    rotate 1
}

/var/log/btmp {
    missingok
    monthly
    create 0660 root utmp
    rotate 1
}

# system-specific logs may be configured here

/etc/logrotate.d/

/etc/logrotate.d/rsyslog

[root@syslogcom1 logrotate.d]# cat rsyslog /var/log/mail.info /var/log/mail.warn /var/log/mail.err /var/log/mail.log /var/log/daemon.log /var/log/kern.log /var/log/user.log /var/log/lpr.log /var/log/cron.log /var/log/debug /var/log/messages { rotate 7 daily missingok notifempty delaycompress compress create postrotate /usr/bin/systemctl reload-or-try-restart rsyslog.service

invoke-rc.d rsyslog reload > /dev/null
    endscript

}

/var/log/auth.log /var/log/syslog { rotate 7 daily missingok notifempty delaycompress compress create postrotate /usr/bin/systemctl reload-or-try-restart rsyslog.service

invoke-rc.d rsyslog reload > /dev/null
    endscript

} /var/log/morabanc/auth.log /var/log/morabanc/LTM-LC/F5.log /var/log/morabanc/intrushield.log /var/log/morabanc/HSM/hsm.log /var/log/morabanc/juniper-junos.log /var/log/morabanc/cisco-firewall.log /var/log/morabanc/juniper-firewall.log /var/log/morabanc/nessus.log /var/log/morabanc/DOMINO_NOTES/notes.log /var/log/morabanc/SNARE/snare.log /var/log/morabanc/nagios.log /var/log/morabanc/apache.log /var/log/morabanc/cisco.log /var/log/morabanc/COM/sw_distributed_virtual.log /var/log/morabanc/paloalto_IDS.log /var/log/morabanc/WIFI/cisco-wlc.log /var/log/morabanc/SISTEMES/aix.log /var/log/morabanc/COM/3com.log /var/log/morabanc/COM/sw_blade_fujitsu.log /var/log/morabanc/COM/WIFI_M96/cisco-wlc.log /var/log/morabanc/MICRO/controladora_vnx.log /var/log/morabanc/MICRO/oracle.log /var/log/morabanc/MUREX/solaris.log /var/log/morabanc/SISTEMES/pureflex.log /var/log/morabanc/SISTEMES/nonstop.log /var/log/morabanc/SISTEMES/caixers.log /var/log/morabanc/fortinet.log /var/log/morabanc/UNIX/unix.log /var/log/morabanc/BLUECOAT/bluecoat.log /var/log/morabanc/hp.log /var/log/morabanc/paloalto_traps.log { rotate 30 daily missingok notifempty delaycompress compress postrotate invoke-rc.d rsyslog reload > /dev/null endscript }

/var/log/morabanc/paloalto.log { rotate 30 daily missingok notifempty delaycompress compress postrotate invoke-rc.d rsyslog reload > /dev/null endscript }

/var/log/morabanc/paloalto_swift.log { rotate 120 daily missingok notifempty delaycompress compress postrotate invoke-rc.d rsyslog reload > /dev/null endscript }

/var/log/morabanc/PROVES.log { rotate 30 size 6M #daily missingok notifempty delaycompress compress postrotate invoke-rc.d rsyslog reload > /dev/null endscript }

~~~

As Jan said if there any configuration errors either in the main logroate.conf or in any include files then rsyslog service may stop working. Could you please send the output of this command and format the output so that it is reader friendly:

journalctl -u rsyslog