Rolling Update and Network Connectivity Issues in RHOCP 4

Issue

• During the cluster upgrade, many infra, worker nodes became NotReady. The long delay during the rolling update caused the issues with the master node upgrade process.

• The API VIP attachment issue and lack of ARP entries on the worker nodes created a network problem that prevented Kubelet from connecting to the API server, causing connectivity issues across the cluster.

  $ sudo journalctl -u kubelet -f
  
  E1226 07:34:00.858326    2955 kubelet_node_status.go:95] "Unable to register node with API server" err="Post "https://api-int.abc.def.com:6443/api/v1/nodes\": dial tcp 10.00.00.0:6443: connect: no route to host" node="gtnpvocthecl-pvw5v-master-1"
  Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.abcd.efg.com:6443/apis/storage.k8s.io/v1/csinodes/master-1": dial tcp 10.00.00.0:6443: connect: no route to host