Proxy URL NOT injected in UWM Prometheus after configuring remote-write

Solution Verified - Updated -

Issue

  • Proxy URL is not injected automatically in prometheus user-workload custom resource.
  • Promtheus-user-workload pods stream below logs indefinitely:

    $ oc logs -c prometheus prometheus-k8s-1 -n openshift-monitoring 
    ...
    2024-08-06T14:56:36.607159251Z ts=2024-08-06T14:56:36.607Z caller=dedupe.go:112 component=remote level=error remote_name=8dee62 url=http://example-prometheus.test.svc.cluster.local:9090/api/v1/receive msg="non-recoverable error" count=9 exemplarCount=0 err="server returned HTTP status 403 Forbidden: <!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.01//EN" "http://www.w3.org/TR/html4/strict.dtd\">"
    2024-08-06T14:56:36.607239708Z ts=2024-08-06T14:56:36.607Z caller=dedupe.go:112 component=remote level=error remote_name=8dee62 url=http://example-prometheus.test.svc.cluster.local:9090/api/v1/receive msg="non-recoverab
    

Environment

  • Red Hat OpenShift Container Platform (RHOCP)
    • 4.16+

Subscriber exclusive content

A Red Hat subscription provides unlimited access to our knowledgebase, tools, and much more.

Current Customers and Partners

Log in for full access

Log In

New to Red Hat?

Learn more about Red Hat subscriptions

Using a Red Hat product through a public cloud?

How to access this content