Red Hat OpenShift Container Platform web console pods constantly being redeployed
Environment
- Red Hat OpenShift Container Platform (RHOCP)
- 4.20
- 4.21
- 4.22
Issue
-
Consolepods are redeployed frequently.$ oc get pod -n openshift-console NAME READY STATUS RESTARTS AGE console-xxx-xx 1/1 Running 0 2m console-xxx-xx 1/1 Running 0 2m downloads-xxx-xx 1/1 Running 0 23h downloads-xxx-xx 1/1 Running 0 23h -
There are
SyncLoopRefreshProgressingandDeploymentSyncDegradedlogs in theconsole operatorpod.I0130 15:32:06.413640 1 event.go:377] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"openshift-console-operator", Name:"console-operator", UID:"exxx0-8xxf-4xx-xx1-12xxxae", APIVersion:"apps/v1", ResourceVersion: "", FieldPath:""}): type: 'Normal' reason: 'ConfigMapUpdated' Updated ConfigMap/console-config -n openshift-console: cause by changes in data.console-config.yaml I0130 15:32:06.420483 1 apps.go:155] Deployment "openshift-console/console" changes: {"metadata":{"annotations":{"console.openshift.io/console-config-version":"5317571","operator.openshift.io/spec-hash":"ab4xxxxx94 acxxxx4745"}},"spec":{"progressDeadlineSeconds":null,"revisionHistoryLimit":null,"template":{"metadata":{"annotations":{"console.openshift.io/console-config-version":"5317571"}},"spec":{"containers":[{"comm and":["/opt/bridge/bin/bridge","--public-dir=/opt/bridge/static","--config=/var/console-config/console-config.yaml","--service-ca-file=/var/service-ca/service-ca.crt","--v=2"],"env":[{"name":"POD_NAME","valueFrom":{"fieldRef":{"apiVers ion":"v1","fieldPath":"metadata.name"}}}],"image":"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:422e91c6e6xxxxf70dfe514","imagePullPolicy":"IfNotPresent","lifecycle":{"preStop":{"exec": {"command":["sleep","25"]}}},"livenessProbe":{"failureThreshold":1,"httpGet":{"path":"/health","port":8443,"scheme":"HTTPS"},"periodSeconds":10,"successThreshold":1,"timeoutSeconds":10},"name":"console","ports":[{"containerPort":8443," name":"https","protocol":"TCP"}],"readinessProbe":{"failureThreshold":3,"httpGet":{"path":"/health","port":8443,"scheme":"HTTPS"},"periodSeconds":10,"successThreshold":1,"timeoutSeconds":1},"resources":{"requests":{"cpu":"10m","memory" :"100Mi"}},"securityContext":{"allowPrivilegeEscalation":false,"capabilities":{"drop":["ALL"]},"readOnlyRootFilesystem":false},"startupProbe":{"failureThreshold":30,"httpGet":{"path":"/health","port":8443,"scheme":"HTTPS"},"periodSecon ds":10,"successThreshold":1,"timeoutSeconds":10},"terminationMessagePolicy":"FallbackToLogsOnError","volumeMounts":[{"mountPath":"/var/serving-cert","name":"console-serving-cert","readOnly":true},{"mountPath":"/var/oauth-config","name" :"console-oauth-config","readOnly":true},{"mountPath":"/var/console-config","name":"console-config","readOnly":true},{"mountPath":"/var/service-ca","name":"service-ca","readOnly":true},{"mountPath":"/etc/pki/ca-trust/extracted/pem","na me":"trusted-ca-bundle","readOnly":true},{"mountPath":"/var/oauth-serving-cert","name":"oauth-serving-cert","readOnly":true}]}],"dnsPolicy":null,"serviceAccount":null,"volumes":[{"name":"console-serving-cert","secret":{"secretName":"co nsole-serving-cert"}},{"name":"console-oauth-config","secret":{"secretName":"console-oauth-config"}},{"configMap":{"name":"console-config"},"name":"console-config"},{"configMap":{"name":"service-ca"},"name":"service-ca"},{"configMap":{ "items":[{"key":"ca-bundle.crt","path":"tls-ca-bundle.pem"}],"name":"trusted-ca-bundle"},"name":"trusted-ca-bundle"},{"configMap":{"name":"oauth-serving-cert"},"name":"oauth-serving-cert"}]}}}} I0130 15:32:06.427216 1 warnings.go:110] "Warning: spec.template.spec.nodeSelector[node-role.kubernetes.io/master]: use \"node-role.kubernetes.io/control-plane\" instead" E0130 15:32:06.427365 1 status.go:130] SyncLoopRefreshProgressing InProgress working toward version 4.22.0-ec.1, 2 replicas available I0130 15:32:06.427635 1 event.go:377] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"openshift-console-operator", Name:"console-operator", UID:"exxxxxf-4xx-9xx-1xxxae", APIVersion:"apps/v1", ResourceVersion: "", FieldPath:""}): type: 'Normal' reason: 'DeploymentUpdated' Updated Deployment.apps/console -n openshift-console because it changed I0130 15:32:06.450639 1 status_controller.go:230] clusteroperator/console diff {"status":{"conditions":[{"lastTransitionTime":"2026-01-27T17:07:32Z","message":"All is well","reason":"AsExpected","status":"False","type":"Degraded" },{"lastTransitionTime":"2026-01-30T15:32:06Z","message":"SyncLoopRefreshProgressing: working toward version 4.22.0-ec.1, 2 replicas available","reason":"SyncLoopRefresh_InProgress","status":"True","type":"Progressing"},{"lastTransitio nTime":"2026-01-27T17:07:32Z","message":"All is well","reason":"AsExpected","status":"True","type":"Available"},{"lastTransitionTime":"2026-01-30T08:14:45Z","message":"All is well","reason":"AsExpected","status":"True","type":"Upgradea ble"},{"lastTransitionTime":"2026-01-27T16:53:37Z","reason":"NoData","status":"Unknown","type":"EvaluationConditionsDetected"}]}} I0130 15:32:06.454436 1 apps.go:155] Deployment "openshift-console/console" changes: {"spec":{"progressDeadlineSeconds":null,"revisionHistoryLimit":null,"template":{"spec":{"containers":[{"command":["/opt/bridge/bin/bridge","--pu blic-dir=/opt/bridge/static","--config=/var/console-config/console-config.yaml","--service-ca-file=/var/service-ca/service-ca.crt","--v=2"],"env":[{"name":"POD_NAME","valueFrom":{"fieldRef":{"apiVersion":"v1","fieldPath":"metadata.name "}}}],"image":"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:422xxxxxxe0xxxdfe514","imagePullPolicy":"IfNotPresent","lifecycle":{"preStop":{"exec":{"command":["sleep","25"]}}},"livene ssProbe":{"failureThreshold":1,"httpGet":{"path":"/health","port":8443,"scheme":"HTTPS"},"periodSeconds":10,"successThreshold":1,"timeoutSeconds":10},"name":"console","ports":[{"containerPort":8443,"name":"https","protocol":"TCP"}],"re adinessProbe":{"failureThreshold":3,"httpGet":{"path":"/health","port":8443,"scheme":"HTTPS"},"periodSeconds":10,"successThreshold":1,"timeoutSeconds":1},"resources":{"requests":{"cpu":"10m","memory":"100Mi"}},"securityContext":{"allow PrivilegeEscalation":false,"capabilities":{"drop":["ALL"]},"readOnlyRootFilesystem":false},"startupProbe":{"failureThreshold":30,"httpGet":{"path":"/health","port":8443,"scheme":"HTTPS"},"periodSeconds":10,"successThreshold":1,"timeout Seconds":10},"terminationMessagePolicy":"FallbackToLogsOnError","volumeMounts":[{"mountPath":"/var/serving-cert","name":"console-serving-cert","readOnly":true},{"mountPath":"/var/oauth-config","name":"console-oauth-config","readOnly":t rue},{"mountPath":"/var/console-config","name":"console-config","readOnly":true},{"mountPath":"/var/service-ca","name":"service-ca","readOnly":true},{"mountPath":"/etc/pki/ca-trust/extracted/pem","name":"trusted-ca-bundle","readOnly":t rue},{"mountPath":"/var/oauth-serving-cert","name":"oauth-serving-cert","readOnly":true}]}],"dnsPolicy":null,"serviceAccount":null,"volumes":[{"name":"console-serving-cert","secret":{"secretName":"console-serving-cert"}},{"name":"conso le-oauth-config","secret":{"secretName":"console-oauth-config"}},{"configMap":{"name":"console-config"},"name":"console-config"},{"configMap":{"name":"service-ca"},"name":"service-ca"},{"configMap":{"items":[{"key":"ca-bundle.crt","pat h":"tls-ca-bundle.pem"}],"name":"trusted-ca-bundle"},"name":"trusted-ca-bundle"},{"configMap":{"name":"oauth-serving-cert"},"name":"oauth-serving-cert"}]}}}} I0130 15:32:06.459548 1 event.go:377] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"openshift-console-operator", Name:"console-operator", UID:"exxxx-4bxx9-9xx-12xxxae", APIVersion:"apps/v1", ResourceVersion: "", FieldPath:""}): type: 'Normal' reason: 'OperatorStatusChanged' Status for clusteroperator/console changed: Progressing changed from False to True ("SyncLoopRefreshProgressing: working toward version 4.22.0-ec.1, 2 replicas availabl e") E0130 15:32:06.461719 1 status.go:130] DeploymentSyncDegraded FailedApply Operation cannot be fulfilled on deployments.apps "console": the object has been modified; please apply your changes to the latest version and try again I0130 15:32:06.461807 1 event.go:377] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"openshift-console-operator", Name:"console-operator", UID:"exxxf-4bxx9-xx-1xxx", APIVersion:"apps/v1", ResourceVersion: "", FieldPath:""}): type: 'Warning' reason: 'DeploymentUpdateFailed' Failed to update Deployment.apps/console -n openshift-console: Operation cannot be fulfilled on deployments.apps "console": the object has been modified; please apply your changes to the latest version and try again E0130 15:32:06.477349 1 base_controller.go:279] "Unhandled Error" err="ConsoleOperator reconciliation failed: Operation cannot be fulfilled on deployments.apps \"console\": the object has been modified; please apply your changes to the latest version and try again" E0130 15:32:06.488808 1 status.go:130] SyncLoopRefreshProgressing InProgress working toward version 4.22.0-ec.1, 1 replicas available I0130 15:32:06.553503 1 status_controller.go:230] clusteroperator/console diff {"status":{"conditions":[{"lastTransitionTime":"2026-01-27T17:07:32Z","message":"DeploymentSyncDegraded: Operation cannot be fulfilled on deployments. apps \"console\": the object has been modified; please apply your changes to the latest version and try again","reason":"AsExpected","status":"False","type":"Degraded"},{"lastTransitionTime":"2026-01-30T15:32:06Z","message":"All is wel l","reason":"AsExpected","status":"False","type":"Progressing"},{"lastTransitionTime":"2026-01-27T17:07:32Z","message":"All is well","reason":"AsExpected","status":"True","type":"Available"},{"lastTransitionTime":"2026-01-30T08:14:45Z" ,"message":"All is well","reason":"AsExpected","status":"True","type":"Upgradeable"},{"lastTransitionTime":"2026-01-27T16:53:37Z","reason":"NoData","status":"Unknown","type":"EvaluationConditionsDetected"}]}} I0130 15:32:06.822109 1 event.go:377] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"openshift-console-operator", Name:"console-operator", UID:"exxxf-4bxx-9xxx-1xxxae", APIVersion:"apps/v1", ResourceVersion: "", FieldPath:""}): type: 'Normal' reason: 'OperatorStatusChanged' Status for clusteroperator/console changed: Degraded message changed from "All is well" to "DeploymentSyncDegraded: Operation cannot be fulfilled on deployments.apps \" console\": the object has been modified; please apply your changes to the latest version and try again",Progressing changed from True to False ("All is well")
Resolution
This issue has been reported to Red Hat engineering. It is being tracked in OCPBUGS-74872. For more information, please open a new support case on the Red Hat Customer Portal referring to this solution.
Workaround
Disable the OpenShift Data Foundation (ODF) Console plugin.
Diagnostic Steps
-
Check and verify if the
ODFconsole plugin is installed :$ oc get consoleplugins -
Check the age of the
consolepods:$ oc get pod -n openshift-console -
Check the logs of the
consoleoperator pods ifSyncLoopRefreshProgressingandDeploymentSyncDegradedlogs as described in issue section:$ oc logs console-operator-9xxx-9xxx -n openshift-console-operator .. E0130 15:32:06.461719 1 status.go:130] DeploymentSyncDegraded FailedApply Operation cannot be fulfilled on deployments.apps "console": the object has been modified; please apply your changes to the latest version and try again}, {"lastTransitionTime":"2026-01-30T15:32:06Z","message":"SyncLoopRefreshProgressing: working toward version 4.22.0-ec.1, 2 replicas available","reason":"SyncLoopRefresh_InProgress","status":"True","type":"Progressing"},{"lastTransition
This solution is part of Red Hat’s fast-track publication program, providing a huge library of solutions that Red Hat engineers have created while supporting our customers. To give you the knowledge you need the instant it becomes available, these articles may be presented in a raw and unedited form.
Comments