Prometheus warns "multiple matches for labels grouping labels must ensure unique matches" in OpenShift 4
Issue
- Prometheus pods are constantly showing below errors in the logs causing
rule evaluation failure
alerts to fire:
caller=manager.go:525 component="rule manager" group=node.rules msg="Evaluating rule failed" rule="record: node:node_num_cpu:sum\nexpr: count by(cluster, node) (sum by(node, cpu) (node_cpu_seconds_total{job=\"node-exporter\"}\n * on(namespace, pod) group_left(node) node_namespace_pod:kube_pod_info:))\n" err="multiple matches for labels: grouping labels must ensure unique matches
- How to troubleshoot this?
Environment
- Red Hat OpenShift Container Platform
- 4.4
Subscriber exclusive content
A Red Hat subscription provides unlimited access to our knowledgebase, tools, and much more.