After updating OpenShift GitOps to 1.10, PrometheusOperatorRejectedResources alerts are firing
Issue
- After updating the OpenShift GitOps Operator from version 1.9 to version 1.10, a
PrometheusOperatorRejectedResources
alert is firing -
The Prometheus Operator Pod shows the following error:
level=warn ts=2023-10-03T08:48:13.911617623Z caller=operator.go:2255 component=prometheusoperator msg="skipping servicemonitor" error="it accesses file system via bearer token file which Prometheus specification prohibits" servicemonitor=openshift-operators/openshift-gitops-operator-metrics-monitor namespace=openshift-user-workload-monitoring prometheus=user-workload
-
There is also a
TargetDown
alert firing in theopenshift-operators
namespace:100% of the openshift-gitops-operator-metrics-service/openshift-gitops-operator-metrics-service targets in Namespace openshift-operators have been unreachable for more than 15 minutes. This may be a symptom of network connectivity issues, down nodes, or failures within these components. Assess the health of the infrastructure and nodes running these targets and then contact support.
-
The target shows the following error:
Scrape failed Get "https://10.129.20.239:8443/metrics": x509: certificate is valid for openshift-gitops-operator-metrics-service.openshift-operators.svc, openshift-gitops-operator-metrics-service.openshift-operators.svc.cluster.local, not openshift-gitops-operator-metrics-service.openshift-gitops-operator.svc
Environment
- Red Hat OpenShift Container Platform (OCP) 4.12
- OpenShift GitOps 1.10
Subscriber exclusive content
A Red Hat subscription provides unlimited access to our knowledgebase, tools, and much more.