Chapter 2. Fuse Online

Procedure

1. If there is an existing openshift-monitoring configuration, skip to Step 2.

   Otherwise, create an openshift-monitoring configuration, that sets the user workload monitoring option to true and then skip to Step 3:

   oc apply -f - <<EOF
   apiVersion: v1
   kind: ConfigMap
   metadata:
     name: cluster-monitoring-config
     namespace: openshift-monitoring
   data:
     config.yaml:
      enableUserWorkload: true
   EOF

2. If there is an existing openshift-monitoring configuration:

   1. Check the existing openshift-monitoring configuration to determine whether the user workload monitoring option is set to true:

      oc get -n openshift-monitoring cm/cluster-monitoring-config -ojsonpath='{.data.config\.yaml}'

      If the result is enableUserWorkload: true, the user workload monitoring option is set to true. Skip to Step 3.

      If the result shows any other configurations, continue to the next step to enable the monitoring of user workloads by editing the ConfigMap.

   2. Open the ConfigMap file in an editor, for example:

      oc -n openshift-monitoring edit cm/cluster-monitoring-config

   3. Set enableUserWorkload to true. For example:

      apiVersion: v1
      kind: ConfigMap
      metadata:
        name: cluster-monitoring-config
        namespace: openshift-monitoring
      data:
        config.yaml:
         enableUserWorkload: true

   4. Save the ConfigMap file.

3. Use the following command to watch the status of the pods in the openshift-user-workload-monitoring namespace:

   oc -n openshift-user-workload-monitoring get pods -w

   Wait until the status of the pods is Running, for example:

   prometheus-operator-5d989f48fd-2qbzd   2/2     Running
   prometheus-user-workload-0             5/5     Running   prometheus-user-workload-1             5/5     Running
   thanos-ruler-user-workload-0           3/3     Running
   thanos-ruler-user-workload-1           3/3     Running

4. Verify that the Fuse Online alert rules are enabled in Prometheus:

   1. Access the internal prometheus instance

      oc port-forward -n openshift-user-workload-monitoring pod/prometheus-user-workload-0 9090

   2. Open your browser to localhost:9090
   3. Select Status> Targets. You should see three syndesis endpoints.
   4. Press CTRL-C to terminate the port-forward process.
5. From the OperatorHub, install the Grafana Operator 4.1.0 to a namespace of your choosing, for example, to the grafana-middleware namespace.

6. Add a cluster role and a cluster role binding to allow the grafana-operator to list nodes and namespaces:

   1. Download the cluster role YAML file from the grafana-operator website:

      curl https://raw.githubusercontent.com/grafana-operator/grafana-operator/master/deploy/cluster_roles/cluster_role_grafana_operator.yaml > tmp_role.yaml

   2. Add cluster permission for the grafana-operator to read other namespaces and nodes:

      cat <<EOF >> tmp_role.yaml
        - apiGroups:
            - ""
          resources:
            - namespaces
            - nodes
          verbs:
            - get
            - list
            - watch
      EOF

      oc apply -f tmp_role.yaml

      oc apply -f - <<EOF
      apiVersion: rbac.authorization.k8s.io/v1
      kind: ClusterRoleBinding
      metadata:
        name: grafana-operator
      roleRef:
        name: grafana-operator
        kind: ClusterRole
        apiGroup: ""
      subjects:
        - kind: ServiceAccount
          name: grafana-operator-controller-manager
          namespace: grafana-middleware
      EOF

7. Enable the grafana-operator to read Grafana dashboards from other namespaces by using the DASHBOARD_NAMESPACES_ALL environment variable to limit the namespaces:

   oc -n grafana-middleware patch subs/grafana-operator --type=merge -p '{"spec":{"config":{"env":[{"name":"DASHBOARD_NAMESPACES_ALL","value":"true"}]}}}'

8. Check that the grafana pods are recreated:

   oc -n grafana-middleware get pods -w

9. Optionally, view the grafana-operator logs:

   oc -n grafana-middleware logs -f `oc -n grafana-middleware get pods -oname|grep grafana-operator-controller-manager` -c manager

10. Add a Grafana custom resource to start a Grafana server pod, for example:

    oc apply -f - <<EOF
    apiVersion: integreatly.org/v1alpha1
    kind: Grafana
    metadata:
      name: grafana-middleware
      namespace: grafana-middleware
    spec:
      config:
        auth:
          disable_signout_menu: true
        auth.anonymous:
          enabled: true
        log:
          level: warn
          mode: console
        security:
          admin_password: secret
          admin_user: root
      dashboardLabelSelector:
      - matchExpressions:
        - key: app
          operator: In
          values:
          - grafana
          - syndesis
      ingress:
        enabled: true
    EOF

11. Allow the grafana-operator to read monitoring information:

    oc -n grafana-middleware adm policy add-cluster-role-to-user cluster-monitoring-view -z grafana-serviceaccount

12. Add a GrafanaDatasource to query thanos-querier:

    oc apply -f - <<EOF
    apiVersion: integreatly.org/v1alpha1
    kind: GrafanaDataSource
    metadata:
      name: prometheus-grafanadatasource
      namespace: grafana-middleware
    spec:
      datasources:
        - access: proxy
          editable: true
          isDefault: true
          jsonData:
            httpHeaderName1: 'Authorization'
            timeInterval: 5s
            tlsSkipVerify: true
          name: Prometheus
          secureJsonData:
          httpHeaderValue1: "Bearer $(oc -n grafana-middleware serviceaccounts get-token grafana-serviceaccount)"
          type: prometheus
          url: "https://$(oc get route thanos-querier -n openshift-monitoring -ojsonpath='{.spec.host}')"
      name: prometheus-grafanadatasource.yaml
    EOF

13. View the grafana server log:

    oc logs -f `oc get pods -l app=grafana -oname`

14. Access the grafana URL and view the Fuse Online dashboards:

    echo "https://"$(oc -n grafana-middleware get route/grafana-route -ojsonpath='{.spec.host}')