kube-apiserver pods are restarting and consuming a lot of cpu/memory in OpenShift 4.6.16 or lower

Issue

• The kube-apiserver pods are constantly restarting and having problems with cpu/memory, making the cluster unstable. Continuous errors come from the OLM operator pod log stating reconcile errors:

  $ oc logs -n openshift-operator-lifecycle-manager olm-operator-[id]
  [...]
  DEBUG controller-runtime.controller   Successfully Reconciled {"controller": "operator", "request": "/namespace-configuration-operator.openshift-stakater-multi-tenan"}
  DEBUG controllers.operator    reconciling operator    {"request": "/namespace-configuration-operator.openshift-stakater-multi-tenan"}
  DEBUG controller-runtime.controller   Successfully Reconciled {"controller": "clusterserviceversion", "request": "mod2-dev/namespace-configuration-operator.v1.0.0"}
  DEBUG controller-runtime.controller   Successfully Reconciled {"controller": "clusterserviceversion", "request": "openshift-kube-controller-manager/redhat-openshift-pipelines-operator.v1.2.3"}
  DEBUG controller-runtime.controller   Successfully Reconciled {"controller": "operator", "request": "/namespace-configuration-operator.openshift-stakater-multi-tenan"}
  DEBUG controllers.operator    reconciling operator    {"request": "/namespace-configuration-operator.openshift-stakater-multi-tenan"}
  DEBUG controllers.operator    Operator is already up-to-date  {"request": "/namespace-configuration-operator.openshift-stakater-multi-tenan"}
  DEBUG controller-runtime.controller   Successfully Reconciled {"controller": "operator", "request": "/namespace-configuration-operator.openshift-stakater-multi-tenan"}
  DEBUG controllers.operator    reconciling operator    {"request": "/namespace-configuration-operator.openshift-stakater-multi-tenan"}
  DEBUG controller-runtime.controller   Successfully Reconciled {"controller": "clusterserviceversion", "request": "gabbar-pr/namespace-configuration-operator.v1.0.0"}
  DEBUG controller-runtime.controller   Successfully Reconciled {"controller": "clusterserviceversion", "request": "enentoapi-pr/redhat-openshift-pipelines-operator.v1.2.3"}
  DEBUG controller-runtime.controller   Successfully Reconciled {"controller": "clusterserviceversion", "request": "mod2-pr/namespace-configuration-operator.v1.0.0"}
  DEBUG controller-runtime.controller   Successfully Reconciled {"controller": "operator", "request": "/namespace-configuration-operator.openshift-stakater-multi-tenan"}
  DEBUG controllers.operator    reconciling operator    {"request": "/namespace-configuration-operator.openshift-stakater-multi-tenan"}
  DEBUG controller-runtime.controller   Successfully Reconciled {"controller": "clusterserviceversion", "request": "openshift-marketplace/redhat-openshift-pipelines-operator.v1.2.3"}
  DEBUG controller-runtime.controller   Successfully Reconciled {"controller": "clusterserviceversion", "request": "openshift-stakater-istio-system/namespace-configuration-operator.v1.0.0"}
  ERROR controllers.operator    Could not update Operator status    {"request": "/namespace-configuration-operator.openshift-stakater-multi-tenan", "error": "Operation cannot be fulfilled on operators.operators.coreos.com \"namespace-configuration-operator.openshift-stakater-multi-tenan\": the object has been modified; please apply your changes to the latest version and try again"}