Kubelet is not deleting pods after they are deleted from Etcd in RHOCP 4

Issue

• The connection sometimes not being available or intermittent.

• A zombie pod in the cluster is keeping active its network configuration.

• Connection problem between Pod and Virtual Servers, resets are seen in a network capture:

  $ tshark -r netcapture.pcap
  ...
     64  36.034217 10.0.0.13 → 10.0.1.2 TCP 74 41439 → 9092 [SYN] Seq=0 Win=26880 Len=0 MSS=8960 SACK_PERM=1 TSval=2192963733 TSecr=0 WS=512
     66  36.034378 10.0.0.13 → 10.0.1.2 TCP 56 41439 → 9092 [RST] Seq=1 Win=0 Len=0
     67  37.052023 10.0.0.13 → 10.0.1.2 TCP 74 [TCP Retransmission] [TCP Port numbers reused] 41439 → 9092 [SYN] Seq=0 Win=26880 Len=0 MSS=8960 SACK_PERM=1 TSval=2192964751 TSecr=0 WS=512
     69  37.052161 10.0.0.13 → 10.0.1.2 TCP 56 41439 → 9092 [RST] Seq=1 Win=0 Len=0
  

• Pods are not removed properly and Kubelet continuously reports the following error:

  Sep 21 16:11:10 node0.example.com hyperkube[6015]: E0921 16:11:10.053424    6015 nestedpendingoperations.go:335] Operation for "{volumeName:kubernetes.io/configmap/426b9acd-afd1-481f-a507-ae35785fd81c-config-volume podName:426b9acd-afd1-481f-a507-ae35785fd81c nodeName:}" failed. No retries permitted until 2022-09-21 16:13:12.05339816 +0000 UTC m=+6828882.543348406 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "config-volume" (UniqueName: "kubernetes.io/configmap/426b9acd-afd1-481f-a507-ae35785fd81c-config-volume") pod "mypod-a0a0a0a0a0-a0a0a0" (UID: "426b9acd-afd1-481f-a507-ae35785fd81c") : configmap "mypod-a0a0a0a0a0" not found