warning alert in rule "cluster:kubelet_volume_stats_used_bytes:provisioner:sum" showing "found duplicate series for the match group on the left hand-side of the operation"
Issue
- alert is active while evaluating rule for kubelet_volume_stats_used_bytes. We can see this sort of warning:
level=warn ts=2020-11-16T15:46:06.776Z caller=manager.go:586 component="rule manager" group=kubernetes.rules msg="Evaluating rule failed" rule="record: cluster:kubelet_volume_stats_used_bytes:provisioner:sum\nexpr: sum by(provisioner) (kubelet_volume_stats_used_bytes * on(namespace, persistentvolumeclaim) group_right() (kube_persistentvolumeclaim_info * on(storageclass) group_left(provisioner) kube_storageclass_info))\n" err="found duplicate series for the match group {namespace=\"openshift-namespace\", persistentvolumeclaim=\"some-persistent-volume-claim\"} on the left hand-side of the operation: [{__name__=\"kubelet_volume_stats_used_bytes\", endpoint=\"https-metrics\", instance=\"<ip address1>:10250\", job=\"kubelet\", metrics_path=\"/metrics\", namespace=\"openshift-namespace\", node=\"worker-3\", persistentvolumeclaim=\"some-persistent-volume-claim\", service=\"kubelet\"}, {__name__=\"kubelet_volume_stats_used_bytes\", endpoint=\"https-metrics\", instance=\"<ip address2>:10250\", job=\"kubelet\", metrics_path=\"/metrics\", namespace=\"openshift-namespace\", node=\"worker-1\", persistentvolumeclaim=\"some-persistent-volume-claim\", service=\"kubelet\"}];many-to-many matching not allowed: matching labels must be unique on one side"
Environment
- Red Hat Openshift Container Platform 4.6.X
Subscriber exclusive content
A Red Hat subscription provides unlimited access to our knowledgebase, tools, and much more.