UserWorkloadMonitoring prometheus pods crashing due to invalid variable in RHOCP4

Solution Verified - Updated -

Issue

Prometheus pod under openshift-user-workload-montoring namespace crashing.

$ oc get pods | grep -i prometheus-user-workload

prometheus-user-workload-0            0/6     Init:CrashLoopBackOff   6          17m
prometheus-user-workload-1            0/6     Init:CrashLoopBackOff   5          12m

$ oc get pods prometheus-user-workload-0  -o yaml

 ...
 ...
          terminated:
        containerID: cri-o://xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
        exitCode: 1
        finishedAt: "2023-11-13T12:54:50Z"
        message: |
          level=info ts=2023-11-13T12:54:50.903025602Z caller=main.go:111 msg="Starting prometheus-config-reloader" version="(version=0.60.1, branch=rhaos-4.12-rhel-8, revision=6e20871)"
          level=info ts=2023-11-13T12:54:50.903108902Z caller=main.go:112 build_context="(go=go1.19.13 X:strictfipsruntime, user=root, date=20231020-18:56:32)"
          expand environment variables: found reference to unset environment variable "application"
        reason: Error
        startedAt: "2023-11-13T12:54:50Z"
...
...

Environment

  • Red Hat OpenShift Container Platform (RHOCP)
    • 4

Subscriber exclusive content

A Red Hat subscription provides unlimited access to our knowledgebase, tools, and much more.

Current Customers and Partners

Log in for full access

Log In

New to Red Hat?

Learn more about Red Hat subscriptions

Using a Red Hat product through a public cloud?

How to access this content