Why container application is not able to connect to the remote server within 60 seconds after upgrading podman to podman-5.2.X in Red Hat Enterprise Linux 9 ?

Solution Verified - Updated -

Environment

  • Red Hat Enterprise Linux 9
  • Rootless containers with podman-5.2.X
  • Pasta user mode networking

Issue

  • Why container application is not able to reconnect to the remote server within 60 seconds after upgrading podman to podman-5.2.X in Red Hat Enterprise Linux 9 ?. With podman 4.9/slirp4netns, application constantly retries to connect to the remote server and a successful connection is made under 40 seconds

Resolution

To resolve the issue, upgrade the passt package to passt-0^20240806.gee36266-6.el9_5 or higher.

Root Cause

From the tcpdump, client is seen trying to establish a connect with the remote server multiple times. Client socket at this time is in SYN-SENT state. When pasta realizes it is not able to make a outbound connection during a network outage or if the server closes connection, instead of RST-ACK, RST packet is send by pasta. RST packet without ACK is dropped and the client continues to remain in SYN-SENT state until connection timeout of 60 seconds is reached(application specific)

  tshark -r pasta_retries.pcap -Y 'frame.number in { 50 .. 51 }'
   50  21.331190   192.168.0.16 → 192.168.30.4  74 TCP 2048 → 8080 [SYN] Seq=0 Win=32120 Len=0 MSS=1460 SACK_PERM TSval=689444093 TSecr=0 WS=1 
   51  21.332283  192.168.30.4 → 192.168.0.16   54 TCP 8080 → 2048 [RST] Seq=1 Win=14600 Len=0

Following patch has been proposed upstream to fix the issue
tcp: Set ACK flag on all RST segments, even for client in SYN-SENT state

This solution is part of Red Hat’s fast-track publication program, providing a huge library of solutions that Red Hat engineers have created while supporting our customers. To give you the knowledge you need the instant it becomes available, these articles may be presented in a raw and unedited form.

Comments