Why rsyslog imptcp module generates hung TCP connections ?
Environment
Red Hat Enterprise Linux 6.5
Issue
- Rsyslog is configured to use TCP (imptcp) module
- Hung rsyslog on established TCP connections between production servers and the central rsyslog server.
Resolution
- rsyslog7 package in RHEL6 contains the fix for this issue.
Root Cause
- rsysolog imptcp module not detecting dead tcp peer
Diagnostic Steps
/etc/rsyslog.conf
# Provides TCP syslog reception
$ModLoad imtcp
$InputTCPServerRun 514
# Provides TCP syslog reception
$ModLoad imtcp
$InputTCPServerRun 514
$inputtcpserverkeepalive on <====================== Keep alive not supported
After patch
rsyslog debug log:
1309.441560958:7fb009c3d700: --------imuxsock calling select, active file descriptors (max 3): 3
1312.047322939:7fb003fff700: imptcp: new connection on listen socket 6
1312.047370787:7fb003fff700: KEEPALIVE enabled for socket 9
1312.123092204:7fb003fff700: imptcp: added socket 9 to epoll[5] set
1312.123140824:7fb003fff700: imptcp going on epoll_wait
netstat:
tcp 0 0 a.b.c.f:514 r.e.w.d:55695 ESTABLISHED - keepalive (6781.40/0/0)
conf:
$ModLoad imptcp
$InputPTCPServerRun 514
$InputPTCPServerKeepalive on
This solution is part of Red Hat’s fast-track publication program, providing a huge library of solutions that Red Hat engineers have created while supporting our customers. To give you the knowledge you need the instant it becomes available, these articles may be presented in a raw and unedited form.
Welcome! Check out the Getting Started with Red Hat page for quick tours and guides for common tasks.
