4.8. 使用节点选择器移动日志记录子系统资源

您可以使用节点选择器将 Elasticsearch 和 Kibana Pod 部署到不同的节点上。

4.8.1. 移动 OpenShift Logging 资源

您可以配置 Cluster Logging Operator,以将用于日志记录子系统组件的 Pod(如 Elasticsearch 和 Kibana)部署到不同的节点上。您无法将 Cluster Logging Operator Pod 从其安装位置移走。

例如,您可以因为 CPU、内存和磁盘要求较高而将 Elasticsearch Pod 移到一个单独的节点上。

先决条件

  • 必须安装 Red Hat OpenShift Logging 和 Elasticsearch Operator。默认情况下没有安装这些功能。

流程

  1. 编辑 openshift-logging 项目中的 ClusterLogging 自定义资源(CR):

    $ oc edit ClusterLogging instance
    apiVersion: logging.openshift.io/v1
    kind: ClusterLogging
    
    ...
    
    spec:
      collection:
        logs:
          fluentd:
            resources: null
          type: fluentd
      logStore:
        elasticsearch:
          nodeCount: 3
          nodeSelector: 1
            node-role.kubernetes.io/infra: ''
          tolerations:
          - effect: NoSchedule
            key: node-role.kubernetes.io/infra
            value: reserved
          - effect: NoExecute
            key: node-role.kubernetes.io/infra
            value: reserved
          redundancyPolicy: SingleRedundancy
          resources:
            limits:
              cpu: 500m
              memory: 16Gi
            requests:
              cpu: 500m
              memory: 16Gi
          storage: {}
        type: elasticsearch
      managementState: Managed
      visualization:
        kibana:
          nodeSelector: 2
            node-role.kubernetes.io/infra: ''
          tolerations:
          - effect: NoSchedule
            key: node-role.kubernetes.io/infra
            value: reserved
          - effect: NoExecute
            key: node-role.kubernetes.io/infra
            value: reserved
          proxy:
            resources: null
          replicas: 1
          resources: null
        type: kibana
    
    ...
    1 2
    添加 nodeSelector 参数,并设为适用于您想要移动的组件的值。您可以根据为节点指定的值,按所示格式使用 nodeSelector 或使用 <key>: <value> 对。如果您在 infrasructure 节点中添加了污点,还要添加匹配的容限。

验证

要验证组件是否已移动,您可以使用 oc get pod -o wide 命令。

例如:

  • 您需要移动来自 ip-10-0-147-79.us-east-2.compute.internal 节点上的 Kibana pod:

    $ oc get pod kibana-5b8bdf44f9-ccpq9 -o wide

    输出示例

    NAME                      READY   STATUS    RESTARTS   AGE   IP            NODE                                        NOMINATED NODE   READINESS GATES
    kibana-5b8bdf44f9-ccpq9   2/2     Running   0          27s   10.129.2.18   ip-10-0-147-79.us-east-2.compute.internal   <none>           <none>

  • 您需要将 Kibana pod 移到 ip-10-0-139-48.us-east-2.compute.internal 节点,该节点是一个专用的基础架构节点:

    $ oc get nodes

    输出示例

    NAME                                         STATUS   ROLES          AGE   VERSION
    ip-10-0-133-216.us-east-2.compute.internal   Ready    master         60m   v1.22.1
    ip-10-0-139-146.us-east-2.compute.internal   Ready    master         60m   v1.22.1
    ip-10-0-139-192.us-east-2.compute.internal   Ready    worker         51m   v1.22.1
    ip-10-0-139-241.us-east-2.compute.internal   Ready    worker         51m   v1.22.1
    ip-10-0-147-79.us-east-2.compute.internal    Ready    worker         51m   v1.22.1
    ip-10-0-152-241.us-east-2.compute.internal   Ready    master         60m   v1.22.1
    ip-10-0-139-48.us-east-2.compute.internal    Ready    infra          51m   v1.22.1

    请注意,该节点具有 node-role.kubernetes.io/infra: " label:

    $ oc get node ip-10-0-139-48.us-east-2.compute.internal -o yaml

    输出示例

    kind: Node
    apiVersion: v1
    metadata:
      name: ip-10-0-139-48.us-east-2.compute.internal
      selfLink: /api/v1/nodes/ip-10-0-139-48.us-east-2.compute.internal
      uid: 62038aa9-661f-41d7-ba93-b5f1b6ef8751
      resourceVersion: '39083'
      creationTimestamp: '2020-04-13T19:07:55Z'
      labels:
        node-role.kubernetes.io/infra: ''
    ...

  • 要移动 Kibana pod,编辑 ClusterLogging CR 以添加节点选择器:

    apiVersion: logging.openshift.io/v1
    kind: ClusterLogging
    
    ...
    
    spec:
    
    ...
    
      visualization:
        kibana:
          nodeSelector: 1
            node-role.kubernetes.io/infra: ''
          proxy:
            resources: null
          replicas: 1
          resources: null
        type: kibana
    1
    添加节点选择器以匹配节点规格中的 label。
  • 保存 CR 后,当前 Kibana Pod 将被终止,新的 Pod 会被部署:

    $ oc get pods

    输出示例

    NAME                                            READY   STATUS        RESTARTS   AGE
    cluster-logging-operator-84d98649c4-zb9g7       1/1     Running       0          29m
    elasticsearch-cdm-hwv01pf7-1-56588f554f-kpmlg   2/2     Running       0          28m
    elasticsearch-cdm-hwv01pf7-2-84c877d75d-75wqj   2/2     Running       0          28m
    elasticsearch-cdm-hwv01pf7-3-f5d95b87b-4nx78    2/2     Running       0          28m
    fluentd-42dzz                                   1/1     Running       0          28m
    fluentd-d74rq                                   1/1     Running       0          28m
    fluentd-m5vr9                                   1/1     Running       0          28m
    fluentd-nkxl7                                   1/1     Running       0          28m
    fluentd-pdvqb                                   1/1     Running       0          28m
    fluentd-tflh6                                   1/1     Running       0          28m
    kibana-5b8bdf44f9-ccpq9                         2/2     Terminating   0          4m11s
    kibana-7d85dcffc8-bfpfp                         2/2     Running       0          33s

  • 新 pod 位于 ip-10-0-139-48.us-east-2.compute.internal 节点上 :

    $ oc get pod kibana-7d85dcffc8-bfpfp -o wide

    输出示例

    NAME                      READY   STATUS        RESTARTS   AGE   IP            NODE                                        NOMINATED NODE   READINESS GATES
    kibana-7d85dcffc8-bfpfp   2/2     Running       0          43s   10.131.0.22   ip-10-0-139-48.us-east-2.compute.internal   <none>           <none>

  • 片刻后,原始 Kibana Pod 将被删除。

    $ oc get pods

    输出示例

    NAME                                            READY   STATUS    RESTARTS   AGE
    cluster-logging-operator-84d98649c4-zb9g7       1/1     Running   0          30m
    elasticsearch-cdm-hwv01pf7-1-56588f554f-kpmlg   2/2     Running   0          29m
    elasticsearch-cdm-hwv01pf7-2-84c877d75d-75wqj   2/2     Running   0          29m
    elasticsearch-cdm-hwv01pf7-3-f5d95b87b-4nx78    2/2     Running   0          29m
    fluentd-42dzz                                   1/1     Running   0          29m
    fluentd-d74rq                                   1/1     Running   0          29m
    fluentd-m5vr9                                   1/1     Running   0          29m
    fluentd-nkxl7                                   1/1     Running   0          29m
    fluentd-pdvqb                                   1/1     Running   0          29m
    fluentd-tflh6                                   1/1     Running   0          29m
    kibana-7d85dcffc8-bfpfp                         2/2     Running   0          62s