in.tftpd stops logging after rsyslogd is HUP'd or restarted

Solution Verified - Updated -

Environment

  • Red Hat Enterprise Linux 5
  • Red Hat Enterprise Linux 6
  • xinetd + in.tftpd + rsyslog

Issue

  • When rsyslog3 (RHEL5) or rsyslog4 (early RHEL6) is HUP'd (that is, when you do a service rsyslog reload or killall -HUP rsyslogd) or restarted (service rsyslog restart), it stops receiving log entries from any of xinetd's already-running in.tftpd instances.

  • Note that this can be reproduced with rsyslog on either RHEL 5 or 6, but not on RHEL5 with old vanilla syslog.

Resolution

  • On HUP
    To avoid this issue when rsyslogd receives a HUP signal, do one of the following:

    • Update to rsyslog v5 (rsyslog5 rpm in RHEL5; in RHEL6 the rsyslog package was rebased) which has a much leaner reload behavior -- the /dev/log socket is no longer recreated on receipt of HUP

    • Stop using HUP
      In all seriousness, the most common cause of rsyslogd getting a HUP is logrotate
      To avoid this, one could add the copytruncate directive to all logrotate config stanzas which apply to files that rsyslog touches ... and then of course comment out or disable the HUP logic as well
      This approach is not recommended as logs can be lost -- i.e., any logs which rsyslog writes to a file while logrotate is doing the copy (see man logrotate description for copytruncate)

  • On full restart
    To avoid this issue when rsyslogd is restarted, there are a couple options:

    • Update to RHEL 7
      This issue will not be seen in RHEL7 since systemd (specifically, systemd-journald.socket is responsible for the creation of /dev/log

    • Update to rsyslog v7 (not available in RHEL5; rsyslog7 rpm in RHEL6) and change the $ModLoad imuxsock line in rsyslog.conf to module(load="imuxsock" SysSock.Unlink="off")1

  1. See upstream documentation ↩

Diagnostic Steps

To reproduce:

  1. Add the following (set instance timeout to 200s and increase verbosity) to the server_args line in /etc/xinetd.d/tftp and start/reload xinetd

    -t 200 -v

  2. Ensure you have a couple files in your tftpboot dir and then do a tail -f /var/log/messages | grep tftp

  3. on the client: Use tftp SERVER -c get file to download a file or two, ensuring you see something like the following in the server's log

    Jun 10 18:24:21 58-up in.tftpd[22611]: RRQ from 192.168.122.22 filename file1
Jun 10 18:24:30 58-up in.tftpd[22611]: RRQ from 192.168.122.22 filename file2

  4. on the server: Do a killall -HUP rsyslogd or a service rsyslog reload

  5. on the client: Use tftp on the client again to get the files and you should notice that the log events do not appear on the server like before

The same buggy behavior cannot be reproduced with classic syslog in RHEL5.

This solution is part of Red Hat’s fast-track publication program, providing a huge library of solutions that Red Hat engineers have created while supporting our customers. To give you the knowledge you need the instant it becomes available, these articles may be presented in a raw and unedited form.

Close

Welcome! Check out the Getting Started with Red Hat page for quick tours and guides for common tasks.