rsyslogd crashing while service restart

Solution Verified - Updated -

Environment

  • Red Hat Enterprise Linux (RHEL) 6
  • rsyslog-5.8.10-8.el6

Issue

  • rsyslogd crashing while service restart in case imtcp and regex modules are set to be loaded in /etc/rsyslog.conf.
  • corresponding snippets from the /etc/rsyslog.conf file:
# Provides TCP syslog reception
$ModLoad imtcp
$InputTCPServerRun 514

...

#### RULES ####

# Log InfiniBand Switch
:fromhost-ip, regex, "10.0.0.[0-9]" /tmp/my_syslog.log
& ~

Resolution

Update rsyslogd to version rsyslog-5.8.10-10.el6_6 released with Advisory RHBA-2014:2004-1 or newer.

Root Cause

A missing cleanup procedure in the regex module could cause the rsyslogd daemon to terminate unexpectedly if it was configured to use the regex module in combination with some other external modules. As a consequence, rsyslogd could abort during the shutdown phase. The packages have been updated to perform a proper cleanup during unloading of the regex module, thus fixing this bug.

Diagnostic Steps

  • From the coredump file:
Core was generated by `/sbin/rsyslogd -i /var/run/syslogd.pid -c 5'.
Program terminated with signal 6, Aborted.
#0  0x00007faec4cd4925 in raise (sig=6) at ../nptl/sysdeps/unix/sysv/linux/raise.c:64
64    return INLINE_SYSCALL (tgkill, 3, pid, selftid, sig);
+++++++++++


(gdb) bt
#0  0x00007faec4cd4925 in raise (sig=6) at ../nptl/sysdeps/unix/sysv/linux/raise.c:64
#1  0x00007faec4cd6105 in abort () at abort.c:92
#2  0x00007faec5cd0e17 in sigsegvHdlr (signum=11) at debug.c:830
#3  <signal handler called>
#4  __strcmp_sse42 () at ../sysdeps/x86_64/multiarch/strcmp.S:259
#5  0x00007faec5cd2262 in ustrcmp (pszObjName=0x7faec101ecd3 "nsdpoll_ptcp") at unicode-helper.h:53
#6  UnregisterObj (pszObjName=0x7faec101ecd3 "nsdpoll_ptcp") at obj.c:1095
#7  0x00007faec101cc19 in modExit () at nsd_ptcp.c:836
#8  0x00007faec5cd3d63 in modPrepareUnload (ppThis=0x7fff62c74ee8) at modules.c:400
#9  modUnlinkAndDestroy (ppThis=0x7fff62c74ee8) at modules.c:700
#10 0x00007faec5cd3de0 in Release (srcFile=<value optimized out>, ppThis=<value optimized out>) at modules.c:984
#11 0x00007faec5cd27f7 in ReleaseObj (srcFile=0x7faec423830d "netstrms.c", pObjName=<value optimized out>, pObjFile=0x7faeb00009a0 "lmnsd_ptcp", pIf=0x7faeb00008f8)
    at obj.c:1211
#12 0x00007faec4237679 in netstrmsDestruct (ppThis=0x7faec7c7c410) at netstrms.c:106
#13 0x00007faec4032dec in tcpsrvDestruct (ppThis=0x7faec463d3e8) at tcpsrv.c:774
#14 0x00007faec443ba1a in modExit () at imtcp.c:270
#15 0x00007faec5cd3d63 in modPrepareUnload (ppThis=0x7fff62c74ff8) at modules.c:400
#16 modUnlinkAndDestroy (ppThis=0x7fff62c74ff8) at modules.c:700
#17 0x00007faec5cd3e55 in modUnloadAndDestructAll (modLinkTypesToUnload=eMOD_LINK_ALL) at modules.c:725
#18 0x00007faec5cbb43c in die (sig=15) at syslogd.c:1117
#19 0x00007faec5cbda10 in realMain (argc=<value optimized out>, argv=<value optimized out>) at syslogd.c:2937
#20 0x00007faec4cc0d1d in __libc_start_main (main=0x7faec5cbdf90 <main>, argc=5, ubp_av=0x7fff62c75388, init=<value optimized out>, fini=<value optimized out>, 
    rtld_fini=<value optimized out>, stack_end=0x7fff62c75378) at libc-start.c:226
#21 0x00007faec5cb9629 in _start ()

This solution is part of Red Hat’s fast-track publication program, providing a huge library of solutions that Red Hat engineers have created while supporting our customers. To give you the knowledge you need the instant it becomes available, these articles may be presented in a raw and unedited form.

Comments