warning alert in rule "cluster:kubelet_volume_stats_used_bytes:provisioner:sum" showing "found duplicate series for the match group on the left hand-side of the operation"

Solution Verified - Updated -

Issue

  • alert is active while evaluating rule for kubelet_volume_stats_used_bytes. We can see this sort of warning:
level=warn ts=2020-11-16T15:46:06.776Z caller=manager.go:586 component="rule manager" group=kubernetes.rules msg="Evaluating rule failed" rule="record: cluster:kubelet_volume_stats_used_bytes:provisioner:sum\nexpr: sum by(provisioner) (kubelet_volume_stats_used_bytes * on(namespace, persistentvolumeclaim) group_right() (kube_persistentvolumeclaim_info * on(storageclass) group_left(provisioner) kube_storageclass_info))\n" err="found duplicate series for the match group {namespace=\"openshift-namespace\", persistentvolumeclaim=\"some-persistent-volume-claim\"} on the left hand-side of the operation: [{__name__=\"kubelet_volume_stats_used_bytes\", endpoint=\"https-metrics\", instance=\"<ip address1>:10250\", job=\"kubelet\", metrics_path=\"/metrics\", namespace=\"openshift-namespace\", node=\"worker-3\", persistentvolumeclaim=\"some-persistent-volume-claim\", service=\"kubelet\"}, {__name__=\"kubelet_volume_stats_used_bytes\", endpoint=\"https-metrics\", instance=\"<ip address2>:10250\", job=\"kubelet\", metrics_path=\"/metrics\", namespace=\"openshift-namespace\", node=\"worker-1\", persistentvolumeclaim=\"some-persistent-volume-claim\", service=\"kubelet\"}];many-to-many matching not allowed: matching labels must be unique on one side"

Environment

  • Red Hat Openshift Container Platform 4.6.X

Subscriber exclusive content

A Red Hat subscription provides unlimited access to our knowledgebase, tools, and much more.

Current Customers and Partners

Log in for full access

Log In

New to Red Hat?

Learn more about Red Hat subscriptions

Using a Red Hat product through a public cloud?

How to access this content