Chapter 7. Upgrading Service Telemetry Framework to version 1.5

To upgrade Service Telemetry Framework (STF) 1.4 to STF 1.5, you must complete the following steps:

  • Replace AMQ Certificate Manager with Certificate Manager.
  • Remove the ClusterServiceVersion and Subscription objects for Smart Gateway Operator and Service Telemetry Operator in the service-telemetry namespace on your Red Hat OpenShift Container Platform environment.
  • Upgrade Red Hat OpenShift Container Platform from 4.8 to 4.10.
  • Re-enable the operators that you removed.

Prerequisites

  • You have backed up your data. There is an outage during the Red Hat OpenShift Container Platform upgrade. You cannot reconfigure the ServiceTelemetry and SmartGateway objects during the Operators replacement.
  • You have prepared your environment for upgrade from Red Hat OpenShift Container Platform 4.8 to the supported version, 4.10.
  • The Red Hat OpenShift Container Platform cluster is fully-connected. STF does not support disconnected or restricted-network clusters.

7.1. Removing the Service Telemetry Framework 1.4 Operators

Remove the Service Telemetry Framework (STF) 1.4 Operators and the AMQ Certificate Manager Operator from the Red Hat OpenShift Container Platform 4.8.

Procedure

  1. Remove the Service Telemetry Operator.
  2. Remove the Smart Gateway Operator.
  3. Remove the AMQ Certificate Manager Operator.
  4. Remove the Grafana Operator.

Additional resources

7.1.1. Removing the Service Telemetry Operator

As part of upgrading your Service Telemetry Framework (STF) installation, you must remove the Service Telemetry Operator in the service-telemetry namespace on your Red Hat OpenShift Container Platform environment.

Procedure

  1. Change to the service-telemetry project:

    $ oc project service-telemetry
  2. Remove the Service Telemetry Operator Subscription:

    $ oc delete sub --selector=operators.coreos.com/service-telemetry-operator.service-telemetry
    
    subscription.operators.coreos.com "service-telemetry-operator" deleted
  3. Remove the Service Telemetry Operator ClusterServiceVersion:

    $ oc delete csv --selector=operators.coreos.com/service-telemetry-operator.service-telemetry
    
    clusterserviceversion.operators.coreos.com "service-telemetry-operator.v1.4.1669718959" deleted

Verification

  1. Verify that the Service Telemetry Operator deployment is not running:

    $ oc get deploy --selector=operators.coreos.com/service-telemetry-operator.service-telemetry
    
    No resources found in service-telemetry namespace.
  2. Verify the Service Telemetry Operator subscription is absent:

    $ oc get sub --selector=operators.coreos.com/service-telemetry-operator.service-telemetry
    
    No resources found in service-telemetry namespace.
  3. Verify the Service Telemetry Operator ClusterServiceVersion is absent:

    $ oc get csv --selector=operators.coreos.com/service-telemetry-operator.service-telemetry
    
    No resources found in service-telemetry namespace.

7.1.2. Removing the Smart Gateway Operator

As part of upgrading your Service Telemetry Framework (STF) installation, you must remove the Smart Gateway Operator in the service-telemetry namespace on your Red Hat OpenShift Container Platform environment.

Procedure

  1. Change to the service-telemetry project:

    $ oc project service-telemetry
  2. Remove the Smart Gateway Operator Subscription:

    $ oc delete sub --selector=operators.coreos.com/smart-gateway-operator.service-telemetry
    
    subscription.operators.coreos.com "smart-gateway-operator-stable-1.4-redhat-operators-openshift-marketplace" deleted
  3. Remove the Smart Gateway Operator ClusterServiceVersion:

    $ oc delete csv --selector=operators.coreos.com/smart-gateway-operator.service-telemetry
    
    clusterserviceversion.operators.coreos.com "smart-gateway-operator.v4.0.1669718962" deleted

Verification

  1. Verify that the Smart Gateway Operator deployment is not running:

    $ oc get deploy --selector=operators.coreos.com/smart-gateway-operator.service-telemetry
    
    No resources found in service-telemetry namespace.
  2. Verify the Smart Gateway Operator subscription is absent:

    $ oc get sub --selector=operators.coreos.com/smart-gateway-operator.service-telemetry
    
    No resources found in service-telemetry namespace.
  3. Verify the Smart Gateway Operator ClusterServiceVersion is absent:

    $ oc get csv --selector=operators.coreos.com/smart-gateway-operator.service-telemetry
    
    No resources found in service-telemetry namespace.

7.1.3. Removing the AMQ Certificate Manager Operator

Procedure

  1. Remove the AMQ Certificate Manager Operator Subscription:

    $ oc delete sub --namespace openshift-operators --selector=operators.coreos.com/amq7-cert-manager-operator.openshift-operators
    
    subscription.operators.coreos.com "amq7-cert-manager-operator" deleted
  2. Remove the AMQ Certificate Manager Operator ClusterServiceVersion:

    $ oc delete csv --namespace openshift-operators --selector=operators.coreos.com/amq7-cert-manager-operator.openshift-operators
    
    clusterserviceversion.operators.coreos.com "amq7-cert-manager.v1.0.11" deleted

Verification

  1. Verify that the AMQ Certificate Manager Operator deployment is not running:

    $ oc get deploy --namespace openshift-operators --selector=operators.coreos.com/amq7-cert-manager-operator.openshift-operators
    
    No resources found in openshift-operators namespace.
  2. Verify that the AMQ Certificate Manager Operator subscription is absent:

    $ oc get sub --namespace openshift-operators --selector=operators.coreos.com/amq7-cert-manager-operator.service-telemetry
    
    No resources found in openshift-operators namespace.
  3. Verify that the AMQ Certificate Manager Operator Cluster Service Version is absent:

    $ oc get csv --namespace openshift-operators --selector=operators.coreos.com/amq7-cert-manager-operator.openshift-operators
    
    No resources found in openshift-operators namespace.

7.1.4. Removing the Grafana Operator

Procedure

  1. Remove the Grafana Operator Subscription:

    $ oc delete sub --selector=operators.coreos.com/grafana-operator.service-telemetry
    
    subscription.operators.coreos.com "grafana-operator" deleted
  2. Remove the Grafana Operator ClusterServiceVersion:

    $ oc delete csv --selector=operators.coreos.com/grafana-operator.service-telemetry
    
    clusterserviceversion.operators.coreos.com "grafana-operator.v3.10.3" deleted

Verification

  1. Verify the Grafana Operator deployment is not running:

    $ oc get deploy --selector=operators.coreos.com/grafana-operator.service-telemetry
    
    No resources found in service-telemetry namespace.
  2. Verify the Grafana Operator subscription is absent:

    $ oc get sub --selector=operators.coreos.com/grafana-operator.service-telemetry
    
    No resources found in service-telemetry namespace.
  3. Verify the Grafana Operator Cluster Service Version is absent:

    $ oc get csv --selector=operators.coreos.com/grafana-operator.service-telemetry
    
    No resources found in service-telemetry namespace.

7.2. Upgrading Red Hat OpenShift Container Platform to 4.10

Service Telemetry Framework (STF) 1.5 is only compatible with Red Hat OpenShift Container Platform 4.10. For more information about upgrading your Red Hat OpenShift Container Platform from 4.8 to 4.10, see Updating clusters overview.

Prerequisites

  • You removed the STF 1.4 Operators.
  • You removed the AMQ Certificate Manager Operator and Grafana Operator. You must remove the Operators before you upgrade Red Hat OpenShift Container Platform because the Operator APIs are incompatible with 4.10. For more information about preparing your Red Hat OpenShift Container Platform for upgrade from 4.8 to 4.10, see Understanding OpenShift Container Platform updates.
  • Verify the suitability of your Red Hat OpenShift Container Platform upgrade:

    $ oc adm upgrade

    You cannot upgrade the cluster if you encounter the following error message:

    Cluster operator operator-lifecycle-manager should not be upgraded between minor versions: ClusterServiceVersions blocking cluster upgrade: service-telemetry/grafana-operator.v3.10.3 is incompatible with OpenShift minor versions greater than 4.8,openshift-operators/amq7-cert-manager.v1.0.11 is incompatible with OpenShift minor versions greater than 4.8

7.3. Installing the Service Telemetry Framework 1.5 Operators

Install the Service Telemetry Framework (STF) 1.5 Operators and the Certificate Manager for OpenShift Operator on your Red Hat OpenShift Container Platform environment. See Section 1.1, “Support for Service Telemetry Framework” for more information about STF support status and life cycle.

Prerequisites

Procedure

  1. Change to the service-telemetry project:

    $ oc project service-telemetry
  2. Create a namespace for the cert-manager Operator:

    $ oc create -f - <<EOF
    apiVersion: project.openshift.io/v1
    kind: Project
    metadata:
      name: openshift-cert-manager-operator
    spec:
      finalizers:
      - kubernetes
    EOF
  3. Create an OperatorGroup for the cert-manager Operator:

    $ oc create -f - <<EOF
    apiVersion: operators.coreos.com/v1
    kind: OperatorGroup
    metadata:
      name: openshift-cert-manager-operator
      namespace: openshift-cert-manager-operator
    spec: {}
    EOF
  4. Subscribe to the cert-manager Operator with the redhat-operators CatalogSource:

    $ oc create -f - <<EOF
    apiVersion: operators.coreos.com/v1alpha1
    kind: Subscription
    metadata:
      name: openshift-cert-manager-operator
      namespace: openshift-cert-manager-operator
    spec:
      channel: tech-preview
      installPlanApproval: Automatic
      name: openshift-cert-manager-operator
      source: redhat-operators
      sourceNamespace: openshift-marketplace
    EOF
  5. Validate your ClusterServiceVersion. Ensure that the phase of cert-manager Operator is Succeeded:

    $ oc get csv --namespace openshift-cert-manager-operator --selector=operators.coreos.com/openshift-cert-manager-operator.openshift-cert-manager-operator
    
    NAME                            DISPLAY                                       VERSION   REPLACES   PHASE
    openshift-cert-manager.v1.7.1   cert-manager Operator for Red Hat OpenShift   1.7.1-1              Succeeded
  6. Optional: Resubscribe to the Grafana Operator. For more information, see: test Section 5.1.1, “Configuring Grafana to host the dashboard”.
  7. Create the Service Telemetry Operator subscription to manage the STF instances:

    $ oc create -f - <<EOF
    apiVersion: operators.coreos.com/v1alpha1
    kind: Subscription
    metadata:
      name: service-telemetry-operator
      namespace: service-telemetry
    spec:
      channel: stable-1.5
      installPlanApproval: Automatic
      name: service-telemetry-operator
      source: redhat-operators
      sourceNamespace: openshift-marketplace
    EOF
  8. Validate the Service Telemetry Operator and the dependent operators:

    $ oc get csv --namespace service-telemetry
    
    NAME                                          DISPLAY                                       VERSION          REPLACES                                      PHASE
    amq7-interconnect-operator.v1.10.13           Red Hat Integration - AMQ Interconnect        1.10.13          amq7-interconnect-operator.v1.10.4            Succeeded
    elasticsearch-eck-operator-certified.v2.5.0   Elasticsearch (ECK) Operator                  2.5.0            elasticsearch-eck-operator-certified.v2.4.0   Succeeded
    openshift-cert-manager.v1.7.1                 cert-manager Operator for Red Hat OpenShift   1.7.1-1                                                        Succeeded
    prometheusoperator.0.47.0                     Prometheus Operator                           0.47.0           prometheusoperator.0.37.0                     Succeeded
    service-telemetry-operator.v1.5.1669950702    Service Telemetry Operator                    1.5.1669950702                                                 Succeeded
    smart-gateway-operator.v5.0.1669950681        Smart Gateway Operator                        5.0.1669950681                                                 Succeeded

Verification

  • Verify that the Service Telemetry Operator has successfully reconciled.

    $ oc logs -f --selector=name=service-telemetry-operator
    
    [...]
    ----- Ansible Task Status Event StdOut (infra.watch/v1beta1, Kind=ServiceTelemetry, default/service-telemetry) -----
    
    
    PLAY RECAP *********************************************************************
    localhost                  : ok=115  changed=0    unreachable=0    failed=0    skipped=21   rescued=0    ignored=0
    
    
    $ oc get pods
    NAME                                                      READY   STATUS    RESTARTS        AGE
    alertmanager-default-0                                    3/3     Running   0               20h
    default-cloud1-ceil-event-smartgateway-6d57ffbbdc-5mrj8   2/2     Running   1 (3m42s ago)   4m21s
    default-cloud1-ceil-meter-smartgateway-67684d88c8-62mp7   3/3     Running   1 (3m43s ago)   4m20s
    default-cloud1-coll-event-smartgateway-66cddd5567-qb6p2   2/2     Running   1 (3m42s ago)   4m19s
    default-cloud1-coll-meter-smartgateway-76d5ff6db5-z5ch8   3/3     Running   0               4m20s
    default-cloud1-sens-meter-smartgateway-7b59669fdd-c42zg   3/3     Running   1 (3m43s ago)   4m20s
    default-interconnect-845c4b647c-wwfcm                     1/1     Running   0               4m10s
    elastic-operator-57b57964c5-6q88r                         1/1     Running   8 (17h ago)     20h
    elasticsearch-es-default-0                                1/1     Running   0               21h
    grafana-deployment-59c54f7d7c-zjnhm                       2/2     Running   0               20h
    interconnect-operator-848889bf8b-bq2zx                    1/1     Running   0               20h
    prometheus-default-0                                      3/3     Running   1 (20h ago)     20h
    prometheus-operator-5d7b69fd46-c2xlv                      1/1     Running   0               20h
    service-telemetry-operator-79fb664dfb-nj8jn               1/1     Running   0               5m11s
    smart-gateway-operator-79557664f8-ql7xr                   1/1     Running   0               5m7s