ODF: Rook Ceph OSD Key Rotation Pods create "PrometheusDuplicateTimestamps alerts".
Issue
- Alert PrometheusDuplicateTimestamps keeps on firing indefinitely.
- Prometheus pods running in openshift-monitoring project streams below logs:
- Follow KCS How to determine the cause for PrometheusDuplicateTimestamps alerts. to ensure the source of the
Duplicates
.
$ oc logs -c prometheus prometheus-k8s-0 -n openshift-monitoring | grep "Error on ingesting"
...
2024-12-27T05:36:01.630586521Z ts=2024-12-27T05:36:01.630Z caller=scrape.go:1738 level=warn component="scrape manager" scrape_pool=serviceMonitor/openshift-monitoring/kube-state-metrics/0 target=https://<pod-ip>:8443/metrics msg="Error on ingesting samples with different value but same timestamp" num_dropped=69
2024-12-27T05:37:01.619302172Z ts=2024-12-27T05:37:01.619Z caller=scrape.go:1738 level=warn component="scrape manager" scrape_pool=serviceMonitor/openshift-monitoring/kube-state-metrics/0 target=https://<pod-ip>:8443/metrics msg="Error on ingesting samples with different value but same timestamp" num_dropped=69
Environment
Red Hat OpenShift Container Platform (OCP) 4.x
Red Hat OpenShift Data Foundation (ODF) 4.x
Subscriber exclusive content
A Red Hat subscription provides unlimited access to our knowledgebase, tools, and much more.