After shutting down a node in the Openshift, the status of static and daemonset pods are not updated.

Solution Verified - Updated -

Issue

  • After shutting down a node in the Openshift, the static pod and daemonset pod still show the running state.
$ ssh core@master01.ocp4.example.net sudo shutdown -h now
$ ssh core@worker02.ocp4.example.net sudo shutdown -h now

$ oc get node
NAME                        STATUS     ROLES    AGE     VERSION
master01.ocp4.example.net   NotReady   master   12d     v1.23.5+3afdacb
master02.ocp4.example.net   Ready      master   12d     v1.23.5+3afdacb
master03.ocp4.example.net   Ready      master   12d     v1.23.5+3afdacb
worker01.ocp4.example.net   Ready      worker   44h     v1.23.5+3afdacb
worker02.ocp4.example.net   NotReady   worker   44h     v1.23.5+3afdacb
worker03.ocp4.example.net   Ready      worker   6d13h   v1.23.5+012e945

$ oc get po -o wide -n openshift-kube-controller-manager | grep -v Completed
NAME                                                      READY   STATUS      RESTARTS   AGE     IP              NODE                     
kube-controller-manager-guard-master01.ocp4.example.net   1/1     Running     0          6d11h   10.128.0.7      master01.ocp4.example.net
kube-controller-manager-guard-master02.ocp4.example.net   1/1     Running     0          6d11h   10.131.0.13     master02.ocp4.example.net
kube-controller-manager-guard-master03.ocp4.example.net   1/1     Running     0          6d11h   10.130.0.12     master03.ocp4.example.net
kube-controller-manager-master01.ocp4.example.net         4/4     Running     4          6d11h   10.74.253.204   master01.ocp4.example.net
kube-controller-manager-master02.ocp4.example.net         4/4     Running     4          6d11h   10.74.254.57    master02.ocp4.example.net
kube-controller-manager-master03.ocp4.example.net         4/4     Running     4          6d11h   10.74.250.166   master03.ocp4.example.net

$ oc get po -o wide -n openshift-etcd | grep -v Completed
NAME                                          READY   STATUS      RESTARTS   AGE     IP              NODE                     
etcd-master01.ocp4.example.net                4/4     Running     4          6d12h   10.74.253.204   master01.ocp4.example.net
etcd-master02.ocp4.example.net                4/4     Running     4          6d12h   10.74.254.57    master02.ocp4.example.net
etcd-master03.ocp4.example.net                4/4     Running     4          6d12h   10.74.250.166   master03.ocp4.example.net
etcd-quorum-guard-647bff9cdc-9nqnq            1/1     Running     0          6d11h   10.74.250.166   master03.ocp4.example.net
etcd-quorum-guard-647bff9cdc-wkrx5            1/1     Running     0          6d11h   10.74.253.204   master01.ocp4.example.net
etcd-quorum-guard-647bff9cdc-zxxq7            1/1     Running     0          6d11h   10.74.254.57    master02.ocp4.example.net

$ oc get po -o wide -n openshift-kube-apiserver | grep -v Completed
NAME                                             READY   STATUS      RESTARTS   AGE     IP              NODE                 
kube-apiserver-guard-master01.ocp4.example.net   1/1     Running     0          6d11h   10.128.0.12     master01.ocp4.example.net
kube-apiserver-guard-master02.ocp4.example.net   1/1     Running     0          6d11h   10.131.0.12     master02.ocp4.example.net
kube-apiserver-guard-master03.ocp4.example.net   1/1     Running     0          6d11h   10.130.0.11     master03.ocp4.example.net
kube-apiserver-master01.ocp4.example.net         5/5     Running     5          6d11h   10.74.253.204   master01.ocp4.example.net
kube-apiserver-master02.ocp4.example.net         5/5     Running     5          6d12h   10.74.254.57    master02.ocp4.example.net
kube-apiserver-master03.ocp4.example.net         5/5     Running     5          6d12h   10.74.250.166   master03.ocp4.example.net

$ oc get po -o wide -n openshift-kube-scheduler | grep -v Completed
NAME                                                       READY   STATUS      RESTARTS   AGE     IP              NODE                     
openshift-kube-scheduler-guard-master01.ocp4.example.net   1/1     Running     0          6d11h   10.128.0.9      master01.ocp4.example.net
openshift-kube-scheduler-guard-master02.ocp4.example.net   1/1     Running     0          6d11h   10.131.0.8      master02.ocp4.example.net
openshift-kube-scheduler-guard-master03.ocp4.example.net   1/1     Running     0          6d11h   10.130.0.13     master03.ocp4.example.net
openshift-kube-scheduler-master01.ocp4.example.net         3/3     Running     3          6d11h   10.74.253.204   master01.ocp4.example.net
openshift-kube-scheduler-master02.ocp4.example.net         3/3     Running     3          6d11h   10.74.254.57    master02.ocp4.example.net
openshift-kube-scheduler-master03.ocp4.example.net         3/3     Running     3          6d11h   10.74.250.166   master03.ocp4.example.net

$ oc get po -A -o wide  | grep worker02.ocp4.example.net | grep -v Terminating | grep -v Completed                                                
openshift-cluster-node-tuning-operator  tuned-sgkc6                          1/1   Running  1  44h  10.74.253.49   worker02.ocp4.example.net    
openshift-dns                           dns-default-nx74h                    2/2   Running  2  44h  10.129.0.2     worker02.ocp4.example.net    
openshift-dns                           node-resolver-fs4fb                  1/1   Running  1  44h  10.74.253.49   worker02.ocp4.example.net    
openshift-image-registry                node-ca-5hpnr                        1/1   Running  1  44h  10.74.253.49   worker02.ocp4.example.net    
openshift-ingress-canary                ingress-canary-96cld                 1/1   Running  1  44h  10.129.0.5     worker02.ocp4.example.net    
openshift-machine-config-operator       machine-config-daemon-s64hn          2/2   Running  2  44h  10.74.253.49   worker02.ocp4.example.net    
openshift-monitoring                    node-exporter-lnnzn                  2/2   Running  2  44h  10.74.253.49   worker02.ocp4.example.net    
openshift-multus                        multus-additional-cni-plugins-sjtt9  1/1   Running  1  44h  10.74.253.49   worker02.ocp4.example.net    
openshift-multus                        multus-zvqmx                         1/1   Running  1  44h  10.74.253.49   worker02.ocp4.example.net    
openshift-multus                        network-metrics-daemon-n2lnr         2/2   Running  2  44h  10.129.0.3     worker02.ocp4.example.net    
openshift-network-diagnostics           network-check-target-bxlgj           1/1   Running  1  44h  10.129.0.4     worker02.ocp4.example.net    
openshift-sdn                           sdn-vrkxh                            2/2   Running  2  44h  10.74.253.49   worker02.ocp4.example.net    

Environment

  • OpenShift Container Platform 4

Subscriber exclusive content

A Red Hat subscription provides unlimited access to our knowledgebase, tools, and much more.

Current Customers and Partners

Log in for full access

Log In

New to Red Hat?

Learn more about Red Hat subscriptions

Using a Red Hat product through a public cloud?

How to access this content