Create new alerting rules based on platform-defined metrics in OpenShift

Issue

• How to create an alerting rule for platform-level metrics?
• How to create PrometheusRules for cluster-related metrics (openshift-, kube- and default namespaces)?

• We have one custom metrics whose main purpose is to define an alert using the kube_node_labels metric. However, inside of Thanos ruler the expression is modified to have namespace filter:

  kube_node_labels{namespace="platform"}
  

• In the transformed expression, kube_node_labels{namespace="platform"} will not work since the kube_node_labels metrics do not have a namespace tag.

• How to scrape the metrics for kube-state-metrics which are by default only scrapped by Prometheus in openshift-monitoring?