Cluster upgrade stuck as Monitoring operator is in degraded state with alertmanager-main-x pod in CreateContainerError state.

Solution Verified - Updated -

Issue

  • Upgrade stuck from RHOCP 4.7.x to 4.7.y.

  • alertmanager-main-x pod is not starting with the below error:

  - image: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:c31a553a10235882871ceb358b12e82d0a7d28996873d2a0b70bdd878a3d5146
    imageID: ""
    lastState: {}
    name: alertmanager
    ready: false
    restartCount: 0
    started: false
    state:
      waiting:
        message: |
          container create failed: time="2021-05-25T09:11:24Z" level=error msg="container_linux.go:366: starting container process caused: exec: \"/bin/alertmanager\": stat /bin/alertmanager: no such file or directory"
        reason: CreateContainerError
  • Monitoring operator is in a degraded state with the below errors:
 lastTransitionTime: '2021-05-18T02:17:04Z'
    message: 'Failed to rollout the stack. Error: running task Updating Alertmanager
      failed: waiting for Alertmanager object changes failed: waiting for Alertmanager
      openshift-monitoring/main: expected 3 replicas, got 0 updated replicas'
    reason: UpdatingAlertmanagerFailed

Environment

  • Red Hat OpenShift Container Platform (RHOCP)
    • 4.7

Subscriber exclusive content

A Red Hat subscription provides unlimited access to our knowledgebase, tools, and much more.

Current Customers and Partners

Log in for full access

Log In