"ERROR AbstractOperator:xxx - Watcher closed with exception in namespace example-project io.fabric8.kubernetes.client.WatcherException: too old resource version: xxx (xxx)" log occur

Issue

• "ERROR AbstractOperator:xxx - Watcher closed with exception in namespace example-project io.fabric8.kubernetes.client.WatcherException: too old resource version: xxx (xxx)" log occur in the Strimzi operators such as the Strimzi cluster operator and the Strimzi entity operator
  • This error message indicates that the watcher in the operator was closed due to trying to reconnect using too old resource version.
  • The resource version, stored in the resource metadata, is updated each time the resource is changed. The watcher opens a connection to the API server based on a particular resource version from which it wants to start watching.
  • If the watcher disconnects and then tries to reconnect, it will typically try to start the watch from the last resource version. This is done to ensure that the client does not miss any changes that happened while it was disconnected. However, Kubernetes/OpenShift only retains a limited history of updates; when the resource version gets too old, it is no longer able to provide changes, leading to the "too old resource version" error. This could happen due to, for example:
    • The API server, i.e., the OpenShift master server, is restarted or down once.
    • The watcher or network delay or issues that caused to disconnect to the API server and remain disconnected for a while.
    • High rate of updates on the Kubernetes resource that causes the watcher's last known resource version to become too old before it can reconnect.

2023-04-28 02:04:14 ERROR AbstractOperator:483 - Watcher closed with exception in namespace example-project
io.fabric8.kubernetes.client.WatcherException: too old resource version: 25747580 (25747657)
    at io.fabric8.kubernetes.client.dsl.internal.AbstractWatchManager.onStatus(AbstractWatchManager.java:288) ~[io.fabric8.kubernetes-client-6.2.0.redhat-00003.jar:?]
    at io.fabric8.kubernetes.client.dsl.internal.AbstractWatchManager.onMessage(AbstractWatchManager.java:260) ~[io.fabric8.kubernetes-client-6.2.0.redhat-00003.jar:?]
    at io.fabric8.kubernetes.client.dsl.internal.WatchConnectionManager.onMessage(WatchConnectionManager.java:113) ~[io.fabric8.kubernetes-client-6.2.0.redhat-00003.jar:?]
    at io.fabric8.kubernetes.client.dsl.internal.WatcherWebSocketListener.onMessage(WatcherWebSocketListener.java:55) ~[io.fabric8.kubernetes-client-6.2.0.redhat-00003.jar:?]
    at io.fabric8.kubernetes.client.okhttp.OkHttpWebSocketImpl$BuilderImpl$1.onMessage(OkHttpWebSocketImpl.java:105) ~[io.fabric8.kubernetes-httpclient-okhttp-6.2.0.redhat-00003.jar:?]
    at okhttp3.internal.ws.RealWebSocket.onReadMessage(RealWebSocket.java:323) ~[com.squareup.okhttp3.okhttp-3.12.12.redhat-00001.jar:3.12.12.redhat-00001]
    at okhttp3.internal.ws.WebSocketReader.readMessageFrame(WebSocketReader.java:219) ~[com.squareup.okhttp3.okhttp-3.12.12.redhat-00001.jar:3.12.12.redhat-00001]
    at okhttp3.internal.ws.WebSocketReader.processNextFrame(WebSocketReader.java:105) ~[com.squareup.okhttp3.okhttp-3.12.12.redhat-00001.jar:3.12.12.redhat-00001]
    at okhttp3.internal.ws.RealWebSocket.loopReader(RealWebSocket.java:274) ~[com.squareup.okhttp3.okhttp-3.12.12.redhat-00001.jar:3.12.12.redhat-00001]
    at okhttp3.internal.ws.RealWebSocket$2.onResponse(RealWebSocket.java:214) ~[com.squareup.okhttp3.okhttp-3.12.12.redhat-00001.jar:3.12.12.redhat-00001]
    at okhttp3.RealCall$AsyncCall.execute(RealCall.java:203) ~[com.squareup.okhttp3.okhttp-3.12.12.redhat-00001.jar:3.12.12.redhat-00001]
    at okhttp3.internal.NamedRunnable.run(NamedRunnable.java:32) ~[com.squareup.okhttp3.okhttp-3.12.12.redhat-00001.jar:3.12.12.redhat-00001]
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) ~[?:?]
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) ~[?:?]
    at java.lang.Thread.run(Thread.java:829) ~[?:?]
Caused by: io.fabric8.kubernetes.client.KubernetesClientException: too old resource version: 25747580 (25747657)