Failed to stop sandbox -- CNI failed to retrieve network namespace path

Issue

• We are seeing the node logs being rather chatty with CNI issues as well as docker image and docker container not found errors
• Errors are mentioning pods / containers that are long gone and cleaned up
• What would be querying for these containers after a reinstall?
• Is there a cache or process that we need to restart to get in sync with the currently available pods?

Nov 08 12:04:41 datanode.example.com atomic-openshift-node[31162]: E1108 12:04:41.581729   31162 remote_runtime.go:109] StopPodSandbox "9b21a1f5bbcf6955e3577b5b7b2b92052bac7bfac736166d8ed07e7f11e25959" from runtime service failed: rpc error: code = 2 desc = NetworkPlugin cni failed to teardown pod "logging-fluentd-zbt4x_logging" network: CNI failed to retrieve network namespace path: Error: No such container: 9b21a1f5bbcf6955e3577b5b7b2b92052bac7bfac736166d8ed07e7f11e25959
Nov 08 12:04:41 datanode.example.com atomic-openshift-node[31162]: E1108 12:04:41.581746   31162 kuberuntime_gc.go:138] Failed to stop sandbox "9b21a1f5bbcf6955e3577b5b7b2b92052bac7bfac736166d8ed07e7f11e25959" before removing: rpc error: code = 2 desc = NetworkPlugin cni failed to teardown pod "logging-fluentd-zbt4x_logging" network: CNI failed to retrieve network namespace path: Error: No such container: 9b21a1f5bbcf6955e3577b5b7b2b92052bac7bfac736166d8ed07e7f11e25959
Nov 08 12:04:41 datanode.example.com atomic-openshift-node[31162]: E1108 12:04:41.583561   31162 docker_sandbox.go:205] Failed to stop sandbox "b0de3356d604bf4a53aecf24b360111036b8b32ab836e50d2e3e5cb7bf0c2820": Error response from daemon: {"message":"No such container: b0de3356d604bf4a53aecf24b360111036b8b32ab836e50d2e3e5cb7bf0c2820"}
Nov 08 12:04:41 datanode.example.com atomic-openshift-node[31162]: E1108 12:04:41.583637   31162 remote_runtime.go:109] StopPodSandbox "b0de3356d604bf4a53aecf24b360111036b8b32ab836e50d2e3e5cb7bf0c2820" from runtime service failed: rpc error: code = 2 desc = NetworkPlugin cni failed to teardown pod "robot-1-build_sandbox" network: CNI failed to retrieve network namespace path: Error: No such container: b0de3356d604bf4a53aecf24b360111036b8b32ab836e50d2e3e5cb7bf0c2820
Nov 08 12:04:41 datanode.example.com atomic-openshift-node[31162]: E1108 12:04:41.583652   31162 kuberuntime_gc.go:138] Failed to stop sandbox "b0de3356d604bf4a53aecf24b360111036b8b32ab836e50d2e3e5cb7bf0c2820" before removing: rpc error: code = 2 desc = NetworkPlugin cni failed to teardown pod "robot-1-build_sandbox" network: CNI failed to retrieve network namespace path: Error: No such container: b0de3356d604bf4a53aecf24b360111036b8b32ab836e50d2e3e5cb7bf0c2820
Nov 08 12:04:41 datanode.example.com atomic-openshift-node[31162]: E1108 12:04:41.585456   31162 docker_sandbox.go:205] Failed to stop sandbox "dc0d9dcc6212bd46b7facc72c2c078287dad493b90184a78de1a3a92f1e2cc79": Error response from daemon: {"message":"No such container: dc0d9dcc6212bd46b7facc72c2c078287dad493b90184a78de1a3a92f1e2cc79"}
Nov 08 12:04:41 datanode.example.com atomic-openshift-node[31162]: E1108 12:04:41.585533   31162 remote_runtime.go:109] StopPodSandbox "dc0d9dcc6212bd46b7facc72c2c078287dad493b90184a78de1a3a92f1e2cc79" from runtime service failed: rpc error: code = 2 desc = NetworkPlugin cni failed to teardown pod "cyborg-1-build_testing" network: CNI failed to retrieve network namespace path: Error: No such container: dc0d9dcc6212bd46b7facc72c2c078287dad493b90184a78de1a3a92f1e2cc79
Nov 08 12:04:41 datanode.example.com atomic-openshift-node[31162]: E1108 12:04:41.585546   31162 kuberuntime_gc.go:138] Failed to stop sandbox "dc0d9dcc6212bd46b7facc72c2c078287dad493b90184a78de1a3a92f1e2cc79" before removing: rpc error: code = 2 desc = NetworkPlugin cni failed to teardown pod "cyborg-1-build_testing" network: CNI failed to retrieve network namespace path: Error: No such container: dc0d9dcc6212bd46b7facc72c2c078287dad493b90184a78de1a3a92f1e2cc79