Why rsyslog imptcp module generates hung TCP connections ?

Solution In Progress - Updated -

Environment

Red Hat Enterprise Linux 6.5

Issue

  • Rsyslog is configured to use TCP (imptcp) module
  • Hung rsyslog on established TCP connections between production servers and the central rsyslog server.

Resolution

  • rsyslog7 package in RHEL6 contains the fix for this issue.

Root Cause

  • rsysolog imptcp module not detecting dead tcp peer

Diagnostic Steps

/etc/rsyslog.conf

# Provides TCP syslog reception                                                                                                                                                                                
$ModLoad imtcp
$InputTCPServerRun 514
# Provides TCP syslog reception                                                                                                                                                                                
$ModLoad imtcp
$InputTCPServerRun 514
$inputtcpserverkeepalive on <====================== Keep alive not supported

After patch
rsyslog debug log:

1309.441560958:7fb009c3d700: --------imuxsock calling select, active file descriptors (max 3): 3 
1312.047322939:7fb003fff700: imptcp: new connection on listen socket 6
1312.047370787:7fb003fff700: KEEPALIVE enabled for socket 9
1312.123092204:7fb003fff700: imptcp: added socket 9 to epoll[5] set
1312.123140824:7fb003fff700: imptcp going on epoll_wait

netstat:

tcp        0      0 a.b.c.f:514           r.e.w.d:55695         ESTABLISHED -                   keepalive (6781.40/0/0)

conf:

$ModLoad imptcp
$InputPTCPServerRun 514
$InputPTCPServerKeepalive on

This solution is part of Red Hat’s fast-track publication program, providing a huge library of solutions that Red Hat engineers have created while supporting our customers. To give you the knowledge you need the instant it becomes available, these articles may be presented in a raw and unedited form.

Close

Welcome! Check out the Getting Started with Red Hat page for quick tours and guides for common tasks.