Why a connection is only present in one side of the servers (zombie connections)in Red Hat Enterprise Linux?

Solution Verified - Updated -

Environment

  • Red Hat Enterprise Linux 5 all versions

  • Third party firewall / appliance

Issue

  • Connections between a web server and a database server is active only in one side of the servers when you have a firewall in the middle. You can check this behavior using netstat command:

    Web Server:
    [root@WEB ~]# netstat -apn | grep 200.192.192.50:47303
    
    tcp 0 0 IP_WEB:47303 IP_DB:3306 ESTABLISHED 19001/java 
    
    Database Server:
    [root@DB ~]# netstat -apn | grep 47303
    (no established connections related to port 47303)
    

The connection appears to be alive only in the web server side. What can cause it?

Resolution

  • Check firewall configuration to see if the connection timeout value is less than the timeout value defined in operation system.

  • Check connection timeout values defined in operation system. The most important parameters are:

     net.ipv4.netfilter.ip_conntrack_tcp_timeout_close_wait
     net.ipv4.netfilter.ip_conntrack_tcp_timeout_fin_wait
     net.ipv4.netfilter.ip_conntrack_tcp_timeout_time_wait
     net.ipv4.tcp_fin_timeout
     net.ipv4.tcp_keepalive_time
    

Root Cause

  • If firewall has a timeout value less than the timeout value defined in operation system, firewall will terminate the connection without send the FYN signal to finish the connection for both sides/servers correctly.
  • The better way to avoid this behavior is maintain the timeout values aligned between firewall and operation system.

This solution is part of Red Hat’s fast-track publication program, providing a huge library of solutions that Red Hat engineers have created while supporting our customers. To give you the knowledge you need the instant it becomes available, these articles may be presented in a raw and unedited form.

Comments