After updating OpenShift GitOps to 1.10, PrometheusOperatorRejectedResources alerts are firing

Solution Verified - Updated -

Issue

  • After updating the OpenShift GitOps Operator from version 1.9 to version 1.10, a PrometheusOperatorRejectedResources alert is firing
  • The Prometheus Operator Pod shows the following error:

    level=warn ts=2023-10-03T08:48:13.911617623Z caller=operator.go:2255 component=prometheusoperator msg="skipping servicemonitor" error="it accesses file system via bearer token file which Prometheus specification prohibits" servicemonitor=openshift-operators/openshift-gitops-operator-metrics-monitor namespace=openshift-user-workload-monitoring prometheus=user-workload
    
  • There is also a TargetDown alert firing in the openshift-operators namespace:

    100% of the openshift-gitops-operator-metrics-service/openshift-gitops-operator-metrics-service targets in Namespace openshift-operators have been unreachable for more than 15 minutes. This may be a symptom of network connectivity issues, down nodes, or failures within these components. Assess the health of the infrastructure and nodes running these targets and then contact support.
    
  • The target shows the following error:

    Scrape failed
    Get "https://10.129.20.239:8443/metrics": x509: certificate is valid for openshift-gitops-operator-metrics-service.openshift-operators.svc, openshift-gitops-operator-metrics-service.openshift-operators.svc.cluster.local, not openshift-gitops-operator-metrics-service.openshift-gitops-operator.svc
    

Environment

  • Red Hat OpenShift Container Platform (OCP) 4.12
  • OpenShift GitOps 1.10

Subscriber exclusive content

A Red Hat subscription provides unlimited access to our knowledgebase, tools, and much more.

Current Customers and Partners

Log in for full access

Log In

New to Red Hat?

Learn more about Red Hat subscriptions

Using a Red Hat product through a public cloud?

How to access this content