第 4 章 升级集群日志记录

在将 OpenShift Container Platform 集群从 4.1 升级到 4.2 后，必须将集群日志记录从 4.1 升级到 4.2。

注意

由于 默认的全局目录命名空间和 Catalog Source 发生变化，如果您根据 Elasticsearch 安装所述，从 YAML 文件手动创建 CatalogSourceConfig 和 Subscription 对象，您需要更新这些对象，以便在升级前指向新的目录命名空间和源，如下所示。

4.1. 更新集群日志记录

升级 OpenShift Container Platform 集群后，您可以通过更新 Elasticsearch Operator 和 Cluster Logging Operator 将集群日志记录从 4.1 升级到 4.2。

先决条件

  • 将集群从 4.1 升级到 4.2。

  • 确保集群日志记录健康：

    • 所有 Pod 都为 Ready 状态。
    • Elasticsearch 集群处于健康状态。

流程

  1. 编辑 CatalogSourceConfig (CSC) 和 Subscription 对象，以指向新的 Namespace 和 Cource:

    1. 通过 CLI，获取 Elasticsearch CSC 的名称： 

      $ oc get csc --all-namespaces
NAMESPACE               NAME                                 STATUS      MESSAGE                                       AGE
openshift-marketplace   certified-operators                  Succeeded   The object has been successfully reconciled   42m
openshift-marketplace   community-operators                  Succeeded   The object has been successfully reconciled   42m
openshift-marketplace   elasticsearch                        Succeeded   The object has been successfully reconciled   27m
openshift-marketplace   installed-redhat-default             Succeeded   The object has been successfully reconciled   26m
openshift-marketplace   installed-redhat-openshift-logging   Succeeded   The object has been successfully reconciled   18m
openshift-marketplace   redhat-operators                     Succeeded   The object has been successfully reconciled   42m

    2. 按如下方式编辑该文件： 

      $ oc edit csc elasticsearch -n openshift-marketplace

apiVersion: operators.coreos.com/v1
kind: CatalogSourceConfig
metadata:
  creationTimestamp: "2020-02-18T15:09:00Z"
  finalizers:
  - finalizer.catalogsourceconfigs.operators.coreos.com
  generation: 3
  name: elasticsearch
  namespace: openshift-marketplace
  resourceVersion: "17694"
  selfLink: /apis/operators.coreos.com/v1/namespaces/openshift-marketplace/catalogsourceconfigs/elasticsearch
  uid: 97c0cd55-5260-11ea-873c-02939b2f528f
spec:
  csDisplayName: Custom
  csPublisher: Custom
  packages: elasticsearch-operator
  targetNamespace: openshift-operators-redhat
  source: redhat-operators 1
      1
      将当前值改为 redhat-operators。

    3. 获取 Elasticsearch Subscription 对象的名称： 

      $ oc get sub

NAME                  PACKAGE                  SOURCE          CHANNEL
elasticsearch-pj7pf   elasticsearch-operator   elasticsearch   preview

    4. 按如下方式编辑该文件： 

      $ oc edit sub elasticsearch-pj7pf

apiVersion: operators.coreos.com/v1alpha1
kind: Subscription
metadata:
  creationTimestamp: "2020-02-17T17:51:18Z"
  generateName: elasticsearch-
  generation: 2
  name: elasticsearch-p5k7n
  namespace: openshift-operators-redhat
  resourceVersion: "38098"
  selfLink: /apis/operators.coreos.com/v1alpha1/namespaces/openshift-operators-redhat/subscriptions/elasticsearch-p5k7n
  uid: 19f6df33-51ae-11ea-82b9-027dfdb65ec2
spec:
  channel: "4.2"
  installPlanApproval: Automatic
  name: elasticsearch-operator
  source: redhat-operators 1
  sourceNamespace: openshift-marketplace 2
....
      1
      将当前值改为 redhat-operators。
      2
      将当前值改为 openshift-marketplace。

  2. 升级 Elasticsearch Operator：

    1. 在 Web 控制台中点击 Operator Management。
    2. 将项目更改为 all projects。
    3. 点与 Elasticsearch 订阅名称相同的 Elasticsearch Operator。
    4. 点 Subscription → Channel。
    5. 在 Change Subscription Update Channel 窗口中，选择 4.2 并点 Save。

    6. 等待几秒钟，然后点 Operators → Installed Operators。

      Elasticsearch Operator 显示为 4.2 版本。例如： 

      Elasticsearch Operator
4.2.0-201909201915 provided
by Red Hat, Inc

  3. 升级 Cluster Logging Operator：

    1. 在 Web 控制台中点击 Operator Management。
    2. 将项目更改为 all projects。
    3. 点 Cluster Logging Operator。
    4. 点 Subscription → Channel。
    5. 在 Change Subscription Update Channel 窗口中，选择 4.2 并点 Save。

    6. 等待几秒钟，然后点 Operators → Installed Operators。

      Cluster Logging Operator 显示为 4.2 版本。例如： 

      Cluster Logging
4.2.0-201909201915 provided
by Red Hat, Inc

  4. 检查日志记录组件：

    1. 确保 Elasticsearch Pod 使用的是 4.2 镜像： 

      $ oc get pod -o yaml -n openshift-logging --selector component=elasticsearch |grep 'image:'

image: registry.redhat.io/openshift4/ose-logging-elasticsearch5:v4.2.0-201909201915
image: registry.redhat.io/openshift4/ose-oauth-proxy:v4.2.0-201909201915
image: registry.redhat.io/openshift4/ose-logging-elasticsearch5:v4.2.0-201909201915
image: registry.redhat.io/openshift4/ose-oauth-proxy:v4.2.0-201909201915
image: registry.redhat.io/openshift4/ose-logging-elasticsearch5:v4.2.0-201909201915
image: registry.redhat.io/openshift4/ose-oauth-proxy:v4.2.0-201909201915
image: registry.redhat.io/openshift4/ose-logging-elasticsearch5:v4.2.0-201909201915
image: registry.redhat.io/openshift4/ose-oauth-proxy:v4.2.0-201909201915
image: registry.redhat.io/openshift4/ose-logging-elasticsearch5:v4.2.0-201909201915
image: registry.redhat.io/openshift4/ose-oauth-proxy:v4.2.0-201909201915
image: registry.redhat.io/openshift4/ose-logging-elasticsearch5:v4.2.0-201909201915
image: registry.redhat.io/openshift4/ose-oauth-proxy:v4.2.0-201909201915

    2. 确保所有 Elasticsearch Pod 都处于 Ready 状态： 

      $ oc get pod -n openshift-logging --selector component=elasticsearch

NAME                                            READY   STATUS    RESTARTS   AGE
elasticsearch-cdm-1pbrl44l-1-55b7546f4c-mshhk   2/2     Running   0          31m
elasticsearch-cdm-1pbrl44l-2-5c6d87589f-gx5hk   2/2     Running   0          30m
elasticsearch-cdm-1pbrl44l-3-88df5d47-m45jc     2/2     Running   0          29m

    3. 确保 Elasticsearch 集群健康： 

      oc exec -n openshift-logging -c elasticsearch elasticsearch-cdm-1pbrl44l-1-55b7546f4c-mshhk -- es_cluster_health

{
  "cluster_name" : "elasticsearch",
  "status" : "green",

....

    4. 确保日志记录收集器 Pod 使用的是 4.2 镜像： 

      $ oc get pod -n openshift-logging --selector logging-infra=fluentd -o yaml |grep 'image:'

image: registry.redhat.io/openshift4/ose-logging-fluentd:v4.2.0-201909201915
image: registry.redhat.io/openshift4/ose-logging-fluentd:v4.2.0-201909201915
image: registry.redhat.io/openshift4/ose-logging-fluentd:v4.2.0-201909201915
image: registry.redhat.io/openshift4/ose-logging-fluentd:v4.2.0-201909201915
image: registry.redhat.io/openshift4/ose-logging-fluentd:v4.2.0-201909201915
image: registry.redhat.io/openshift4/ose-logging-fluentd:v4.2.0-201909201915
image: registry.redhat.io/openshift4/ose-logging-fluentd:v4.2.0-201909201915
image: registry.redhat.io/openshift4/ose-logging-fluentd:v4.2.0-201909201915
image: registry.redhat.io/openshift4/ose-logging-fluentd:v4.2.0-201909201915
image: registry.redhat.io/openshift4/ose-logging-fluentd:v4.2.0-201909201915
image: registry.redhat.io/openshift4/ose-logging-fluentd:v4.2.0-201909201915
image: registry.redhat.io/openshift4/ose-logging-fluentd:v4.2.0-201909201915

    5. 确保 Kibana Pod 使用的是 4.2 镜像： 

      $ oc get pod -n openshift-logging --selector logging-infra=kibana -o yaml |grep 'image:'

image: registry.redhat.io/openshift4/ose-logging-kibana5:v4.2.0-201909210748
image: registry.redhat.io/openshift4/ose-oauth-proxy:v4.2.0-201909201915
image: registry.redhat.io/openshift4/ose-logging-kibana5:v4.2.0-201909210748
image: registry.redhat.io/openshift4/ose-oauth-proxy:v4.2.0-201909201915

    6. 确保 Curator CronJob 使用的是 4.2 镜像： 

      $ $ oc get CronJob curator -n openshift-logging -o yaml |grep 'image:'

image: registry.redhat.io/openshift4/ose-logging-curator5:v4.2.0-201909201915