Red Hat OpenShift Container Platform web console pods constantly being redeployed

Solution Verified - Updated -

Environment

  • Red Hat OpenShift Container Platform (RHOCP)
    • 4.20
    • 4.21
    • 4.22

Issue

  • Console pods are redeployed frequently.

    $ oc get pod -n openshift-console
    NAME                         READY   STATUS    RESTARTS   AGE
    console-xxx-xx     1/1     Running   0          2m
    console-xxx-xx     1/1     Running   0          2m
    downloads-xxx-xx   1/1     Running   0          23h
    downloads-xxx-xx   1/1     Running   0          23h
    
  • There are SyncLoopRefreshProgressing and DeploymentSyncDegraded logs in the console operator pod.

    I0130 15:32:06.413640       1 event.go:377] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"openshift-console-operator", Name:"console-operator", UID:"exxx0-8xxf-4xx-xx1-12xxxae", APIVersion:"apps/v1", ResourceVersion:
    "", FieldPath:""}): type: 'Normal' reason: 'ConfigMapUpdated' Updated ConfigMap/console-config -n openshift-console:
    cause by changes in data.console-config.yaml
    I0130 15:32:06.420483       1 apps.go:155] Deployment "openshift-console/console" changes: {"metadata":{"annotations":{"console.openshift.io/console-config-version":"5317571","operator.openshift.io/spec-hash":"ab4xxxxx94
    acxxxx4745"}},"spec":{"progressDeadlineSeconds":null,"revisionHistoryLimit":null,"template":{"metadata":{"annotations":{"console.openshift.io/console-config-version":"5317571"}},"spec":{"containers":[{"comm
    and":["/opt/bridge/bin/bridge","--public-dir=/opt/bridge/static","--config=/var/console-config/console-config.yaml","--service-ca-file=/var/service-ca/service-ca.crt","--v=2"],"env":[{"name":"POD_NAME","valueFrom":{"fieldRef":{"apiVers
    ion":"v1","fieldPath":"metadata.name"}}}],"image":"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:422e91c6e6xxxxf70dfe514","imagePullPolicy":"IfNotPresent","lifecycle":{"preStop":{"exec":
    {"command":["sleep","25"]}}},"livenessProbe":{"failureThreshold":1,"httpGet":{"path":"/health","port":8443,"scheme":"HTTPS"},"periodSeconds":10,"successThreshold":1,"timeoutSeconds":10},"name":"console","ports":[{"containerPort":8443,"
    name":"https","protocol":"TCP"}],"readinessProbe":{"failureThreshold":3,"httpGet":{"path":"/health","port":8443,"scheme":"HTTPS"},"periodSeconds":10,"successThreshold":1,"timeoutSeconds":1},"resources":{"requests":{"cpu":"10m","memory"
    :"100Mi"}},"securityContext":{"allowPrivilegeEscalation":false,"capabilities":{"drop":["ALL"]},"readOnlyRootFilesystem":false},"startupProbe":{"failureThreshold":30,"httpGet":{"path":"/health","port":8443,"scheme":"HTTPS"},"periodSecon
    ds":10,"successThreshold":1,"timeoutSeconds":10},"terminationMessagePolicy":"FallbackToLogsOnError","volumeMounts":[{"mountPath":"/var/serving-cert","name":"console-serving-cert","readOnly":true},{"mountPath":"/var/oauth-config","name"
    :"console-oauth-config","readOnly":true},{"mountPath":"/var/console-config","name":"console-config","readOnly":true},{"mountPath":"/var/service-ca","name":"service-ca","readOnly":true},{"mountPath":"/etc/pki/ca-trust/extracted/pem","na
    me":"trusted-ca-bundle","readOnly":true},{"mountPath":"/var/oauth-serving-cert","name":"oauth-serving-cert","readOnly":true}]}],"dnsPolicy":null,"serviceAccount":null,"volumes":[{"name":"console-serving-cert","secret":{"secretName":"co
    nsole-serving-cert"}},{"name":"console-oauth-config","secret":{"secretName":"console-oauth-config"}},{"configMap":{"name":"console-config"},"name":"console-config"},{"configMap":{"name":"service-ca"},"name":"service-ca"},{"configMap":{
    "items":[{"key":"ca-bundle.crt","path":"tls-ca-bundle.pem"}],"name":"trusted-ca-bundle"},"name":"trusted-ca-bundle"},{"configMap":{"name":"oauth-serving-cert"},"name":"oauth-serving-cert"}]}}}}
    I0130 15:32:06.427216       1 warnings.go:110] "Warning: spec.template.spec.nodeSelector[node-role.kubernetes.io/master]: use \"node-role.kubernetes.io/control-plane\" instead"
    E0130 15:32:06.427365       1 status.go:130] SyncLoopRefreshProgressing InProgress working toward version 4.22.0-ec.1, 2 replicas available
    I0130 15:32:06.427635       1 event.go:377] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"openshift-console-operator", Name:"console-operator", UID:"exxxxxf-4xx-9xx-1xxxae", APIVersion:"apps/v1", ResourceVersion:
    "", FieldPath:""}): type: 'Normal' reason: 'DeploymentUpdated' Updated Deployment.apps/console -n openshift-console because it changed
    I0130 15:32:06.450639       1 status_controller.go:230] clusteroperator/console diff {"status":{"conditions":[{"lastTransitionTime":"2026-01-27T17:07:32Z","message":"All is well","reason":"AsExpected","status":"False","type":"Degraded"
    },{"lastTransitionTime":"2026-01-30T15:32:06Z","message":"SyncLoopRefreshProgressing: working toward version 4.22.0-ec.1, 2 replicas available","reason":"SyncLoopRefresh_InProgress","status":"True","type":"Progressing"},{"lastTransitio
    nTime":"2026-01-27T17:07:32Z","message":"All is well","reason":"AsExpected","status":"True","type":"Available"},{"lastTransitionTime":"2026-01-30T08:14:45Z","message":"All is well","reason":"AsExpected","status":"True","type":"Upgradea
    ble"},{"lastTransitionTime":"2026-01-27T16:53:37Z","reason":"NoData","status":"Unknown","type":"EvaluationConditionsDetected"}]}}
    I0130 15:32:06.454436       1 apps.go:155] Deployment "openshift-console/console" changes: {"spec":{"progressDeadlineSeconds":null,"revisionHistoryLimit":null,"template":{"spec":{"containers":[{"command":["/opt/bridge/bin/bridge","--pu
    blic-dir=/opt/bridge/static","--config=/var/console-config/console-config.yaml","--service-ca-file=/var/service-ca/service-ca.crt","--v=2"],"env":[{"name":"POD_NAME","valueFrom":{"fieldRef":{"apiVersion":"v1","fieldPath":"metadata.name
    "}}}],"image":"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:422xxxxxxe0xxxdfe514","imagePullPolicy":"IfNotPresent","lifecycle":{"preStop":{"exec":{"command":["sleep","25"]}}},"livene
    ssProbe":{"failureThreshold":1,"httpGet":{"path":"/health","port":8443,"scheme":"HTTPS"},"periodSeconds":10,"successThreshold":1,"timeoutSeconds":10},"name":"console","ports":[{"containerPort":8443,"name":"https","protocol":"TCP"}],"re
    adinessProbe":{"failureThreshold":3,"httpGet":{"path":"/health","port":8443,"scheme":"HTTPS"},"periodSeconds":10,"successThreshold":1,"timeoutSeconds":1},"resources":{"requests":{"cpu":"10m","memory":"100Mi"}},"securityContext":{"allow
    PrivilegeEscalation":false,"capabilities":{"drop":["ALL"]},"readOnlyRootFilesystem":false},"startupProbe":{"failureThreshold":30,"httpGet":{"path":"/health","port":8443,"scheme":"HTTPS"},"periodSeconds":10,"successThreshold":1,"timeout
    Seconds":10},"terminationMessagePolicy":"FallbackToLogsOnError","volumeMounts":[{"mountPath":"/var/serving-cert","name":"console-serving-cert","readOnly":true},{"mountPath":"/var/oauth-config","name":"console-oauth-config","readOnly":t
    rue},{"mountPath":"/var/console-config","name":"console-config","readOnly":true},{"mountPath":"/var/service-ca","name":"service-ca","readOnly":true},{"mountPath":"/etc/pki/ca-trust/extracted/pem","name":"trusted-ca-bundle","readOnly":t
    rue},{"mountPath":"/var/oauth-serving-cert","name":"oauth-serving-cert","readOnly":true}]}],"dnsPolicy":null,"serviceAccount":null,"volumes":[{"name":"console-serving-cert","secret":{"secretName":"console-serving-cert"}},{"name":"conso
    le-oauth-config","secret":{"secretName":"console-oauth-config"}},{"configMap":{"name":"console-config"},"name":"console-config"},{"configMap":{"name":"service-ca"},"name":"service-ca"},{"configMap":{"items":[{"key":"ca-bundle.crt","pat
    h":"tls-ca-bundle.pem"}],"name":"trusted-ca-bundle"},"name":"trusted-ca-bundle"},{"configMap":{"name":"oauth-serving-cert"},"name":"oauth-serving-cert"}]}}}}
    I0130 15:32:06.459548       1 event.go:377] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"openshift-console-operator", Name:"console-operator", UID:"exxxx-4bxx9-9xx-12xxxae", APIVersion:"apps/v1", ResourceVersion:
    "", FieldPath:""}): type: 'Normal' reason: 'OperatorStatusChanged' Status for clusteroperator/console changed: Progressing changed from False to True ("SyncLoopRefreshProgressing: working toward version 4.22.0-ec.1, 2 replicas availabl
    e")
    E0130 15:32:06.461719       1 status.go:130] DeploymentSyncDegraded FailedApply Operation cannot be fulfilled on deployments.apps "console": the object has been modified; please apply your changes to the latest version and try again
    I0130 15:32:06.461807       1 event.go:377] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"openshift-console-operator", Name:"console-operator", UID:"exxxf-4bxx9-xx-1xxx", APIVersion:"apps/v1", ResourceVersion:
    "", FieldPath:""}): type: 'Warning' reason: 'DeploymentUpdateFailed' Failed to update Deployment.apps/console -n openshift-console: Operation cannot be fulfilled on deployments.apps "console": the object has been modified; please apply
     your changes to the latest version and try again
    E0130 15:32:06.477349       1 base_controller.go:279] "Unhandled Error" err="ConsoleOperator reconciliation failed: Operation cannot be fulfilled on deployments.apps \"console\": the object has been modified; please apply your changes 
    to the latest version and try again"
    E0130 15:32:06.488808       1 status.go:130] SyncLoopRefreshProgressing InProgress working toward version 4.22.0-ec.1, 1 replicas available
    I0130 15:32:06.553503       1 status_controller.go:230] clusteroperator/console diff {"status":{"conditions":[{"lastTransitionTime":"2026-01-27T17:07:32Z","message":"DeploymentSyncDegraded: Operation cannot be fulfilled on deployments.
    apps \"console\": the object has been modified; please apply your changes to the latest version and try again","reason":"AsExpected","status":"False","type":"Degraded"},{"lastTransitionTime":"2026-01-30T15:32:06Z","message":"All is wel
    l","reason":"AsExpected","status":"False","type":"Progressing"},{"lastTransitionTime":"2026-01-27T17:07:32Z","message":"All is well","reason":"AsExpected","status":"True","type":"Available"},{"lastTransitionTime":"2026-01-30T08:14:45Z"
    ,"message":"All is well","reason":"AsExpected","status":"True","type":"Upgradeable"},{"lastTransitionTime":"2026-01-27T16:53:37Z","reason":"NoData","status":"Unknown","type":"EvaluationConditionsDetected"}]}}
    I0130 15:32:06.822109       1 event.go:377] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"openshift-console-operator", Name:"console-operator", UID:"exxxf-4bxx-9xxx-1xxxae", APIVersion:"apps/v1", ResourceVersion:
    "", FieldPath:""}): type: 'Normal' reason: 'OperatorStatusChanged' Status for clusteroperator/console changed: Degraded message changed from "All is well" to "DeploymentSyncDegraded: Operation cannot be fulfilled on deployments.apps \"
    console\": the object has been modified; please apply your changes to the latest version and try again",Progressing changed from True to False ("All is well")
    

Resolution

This issue has been reported to Red Hat engineering. It is being tracked in OCPBUGS-74872. For more information, please open a new support case on the Red Hat Customer Portal referring to this solution.

Workaround

Disable the OpenShift Data Foundation (ODF) Console plugin.

Diagnostic Steps

  • Check and verify if the ODF console plugin is installed :

    $ oc get consoleplugins
    
  • Check the age of the console pods:

    $ oc get pod -n openshift-console
    
  • Check the logs of the console operator pods if SyncLoopRefreshProgressing and DeploymentSyncDegraded logs as described in issue section:

    $ oc logs console-operator-9xxx-9xxx -n openshift-console-operator
    ..
    E0130 15:32:06.461719      1 status.go:130] DeploymentSyncDegraded FailedApply Operation cannot be fulfilled on deployments.apps "console": the object has been modified; please apply your changes to the latest version and try again}, {"lastTransitionTime":"2026-01-30T15:32:06Z","message":"SyncLoopRefreshProgressing: working toward version 4.22.0-ec.1, 2 replicas available","reason":"SyncLoopRefresh_InProgress","status":"True","type":"Progressing"},{"lastTransition
    

This solution is part of Red Hat’s fast-track publication program, providing a huge library of solutions that Red Hat engineers have created while supporting our customers. To give you the knowledge you need the instant it becomes available, these articles may be presented in a raw and unedited form.

Comments