5.2. Configuring monitoring to use OpenShift Container Storage

OpenShift Container Storage provides a monitoring stack that comprises of Prometheus and Alert Manager.

Follow the instructions in this section to configure OpenShift Container Storage as storage for the monitoring stack.

重要

Monitoring will not function if it runs out of storage space. Always ensure that you have plenty of storage capacity for monitoring.

Red Hat recommends configuring a short retention interval for this service. See the Modifying retention time for Prometheus metrics data of Monitoring guide in the OpenShift Container Platform documentation for details.

Prerequisites

  • You have administrative access to OpenShift Web Console.
  • OpenShift Container Storage Operator is installed and running in the openshift-storage namespace. In the OpenShift Web Console, click OperatorsInstalled Operators to view installed operators.
  • Monitoring Operator is installed and running in the openshift-monitoring namespace. In the OpenShift Web Console, click AdministrationCluster SettingsCluster Operators to view cluster operators.
  • A storage class with provisioner openshift-storage.rbd.csi.ceph.com is available. In the OpenShift Web Console, click Storage → Storage Classes to view available storage classes.

Procedure

  1. In the OpenShift Web Console, go to WorkloadsConfig Maps.
  2. Set the Project dropdown to openshift-monitoring.
  3. Click Create Config Map.
  4. Define a new cluster-monitoring-config Config Map using the following example.

    Replace the content in angle brackets (<, >) with your own values, for example, retention: 24h or storage: 40Gi.

    Replace the storageClassName with the storageclass that uses the provisioner openshift-storage.rbd.csi.ceph.com. In the example given below the name of the storageclass is ocs-storagecluster-ceph-rbd.

    Example cluster-monitoring-config Config Map

    apiVersion: v1
    kind: ConfigMap
    metadata:
      name: cluster-monitoring-config
      namespace: openshift-monitoring
    data:
      config.yaml: |
          prometheusK8s:
            retention: <time to retain monitoring files, e.g. 24h>
            volumeClaimTemplate:
              metadata:
                name: ocs-prometheus-claim
              spec:
                storageClassName: ocs-storagecluster-ceph-rbd
                resources:
                  requests:
                    storage: <size of claim, e.g. 40Gi>
          alertmanagerMain:
            volumeClaimTemplate:
              metadata:
                name: ocs-alertmanager-claim
              spec:
                storageClassName: ocs-storagecluster-ceph-rbd
                resources:
                  requests:
                    storage: <size of claim, e.g. 40Gi>

  5. Click Create to save and create the Config Map.

Verification steps

  1. Verify that the Persistent Volume Claims are bound to the pods.

    1. Go to StoragePersistent Volume Claims.
    2. Set the Project dropdown to openshift-monitoring.
    3. Verify that 5 Persistent Volume Claims are visible with a state of Bound, attached to three alertmanager-main-* pods, and two prometheus-k8s-* pods.

      Monitoring storage created and bound

      Screenshot of OpenShift Web Console showing five pods with persistent volume claims bound in the openshift-monitoring project

  2. Verify that the new alertmanager-main-* pods appear with a state of Running.

    1. Go to WorkloadsPods.
    2. Click the new alertmanager-main-* pods to view the pod details.
    3. Scroll down to Volumes and verify that the volume has a Type, ocs-alertmanager-claim that matches one of your new Persistent Volume Claims, for example, ocs-alertmanager-claim-alertmanager-main-0.

      Persistent Volume Claims attached to alertmanager-main-* pod

      Screenshot of OpenShift Web Console showing persistent volume claim attached to the altermanager pod

  3. Verify that the new prometheus-k8s-* pods appear with a state of Running.

    1. Click the new prometheus-k8s-* pods to view the pod details.
    2. Scroll down to Volumes and verify that the volume has a Type, ocs-prometheus-claim that matches one of your new Persistent Volume Claims, for example, ocs-prometheus-claim-prometheus-k8s-0.

      Persistent Volume Claims attached to prometheus-k8s-* pod

      Screenshot of OpenShift Web Console showing persistent volume claim attached to the prometheus pod