Prometheus shows: Evaluating rule failed: many-to-many matching not allowed: matching labels must be unique on one side

Issue

• After upgrading OpenShift cluster to v4.5.37, getting following alert on OpenShift web console.

PrometheusRulefailures
Prometheus openshift-monitoring/prometheus-k8s-0 has failed to evaluate 50 rules in the last 5m.

PrometheusRulefailures
Prometheus openshift-monitoring/prometheus-k8s-0 has failed to evaluate 50 rules in the last 5m.

• Prometheus shows following warnings in the logs.

level=warn caller=manager.go:525 component="rule manager" group=kubernetes-system-kubelet msg="Evaluating rule failed" rule="alert: KubeletPodStartUpLatencyHigh\nexpr: max by(node) (max by(instance) (kubelet_running_pod_count{job=\"kubelet\",metrics_path=\"/metrics\"})\n  * on(instance) group_left(node) kubelet_node_name{job=\"kubelet\",metrics_path=\"/metrics\"})\n  / max by(node) (kube_node_status_capacity_pods{job=\"kube-state-metrics\"} != 1) >\n  0.95\nfor: 15m\nlabels:\n  severity: warning\nannotations:\n  message: Kubelet '{{ $labels.node }}' is running at {{ $value | humanizePercentage\n    }} of its Pod capacity.\n" err="found duplicate series for the match group {instance=\"x.x.x.x:10250\"} on the right hand-side of the operation: [{__name__=\"kubelet_node_name\", endpoint=\"https-metrics\", instance=\"x.x.x.x:10250\", job=\"kubelet\", metrics_path=\"/metrics\", namespace=\"kube-system\", node=\"node1.example.com\", service=\"prometheusoperator-kube-pr-kubelet\"}, {__name__=\"kubelet_node_name\", endpoint=\"https-metrics\", instance=\"x.x.x.x:10250\", job=\"kubelet\", metrics_path=\"/metrics\", namespace=\"kube-system\", node=\"node1.example.com\", service=\"kubelet\"}];many-to-many matching not allowed: matching labels must be unique on one side"