Openshift control plane pods didn't come up

Latest response

I am trying to install open shift container platform for the development purpose. both master and node having RHEL 7.4 os . How ever completed the pre-requisite yml . But while installing deploy_cluster.yml getting below error. Need advise on this.

ASK [openshift_control_plane : Check status in the kube-system namespace] ***********************************************************************************
fatal: [SBSTPVMLX603]: FAILED! => {"changed": true, "cmd": ["oc", "status", "--config=/etc/origin/master/admin.kubeconfig", "-n", "kube-system"], "delta": "0:00:00.647370", "end": "2019-02-05 21:04:38.641597", "msg": "non-zero return code", "rc": 1, "start": "2019-02-05 21:04:37.994227", "stderr": "The connection to the server SBSTPVMLX603:8443 was refused - did you specify the right host or port?\nThe connection to the server SBSTPVMLX603:8443 was refused - did you specify the right host or port?\nThe connection to the server SBSTPVMLX603:8443 was refused - did you specify the right host or port?\nThe connection to the server SBSTPVMLX603:8443 was refused - did you specify the right host or port?\n

Responses

Having the same issue here. Anyone have any suggestions?

E0218 20:49:04.139528 1 reflector.go:136] k8s.io/client-go/informers/factory.go:130: Failed to list *v1.Service: Get https://osemaster02.local:8443/api/v1/services?limit=500&resourceVersion=0: dial tcp 11.11.11.11:8443: connect: connection refused

I am facing similar issue , any fix here

Failure summary:

  1. Hosts: master.example.com Play: Configure masters Task: Report control plane errors Message: Control plane pods didn't come up [root@master openshift-ansible]# journalctl -xe Apr 02 09:49:56 master.example.com atomic-openshift-node[70579]: E0402 09:49:56.497767 70579 kuberuntime_sandbox.go:56] CreatePodSandbox for pod "master-api-master.example.com_kube-system Apr 02 09:49:56 master.example.com atomic-openshift-node[70579]: E0402 09:49:56.497780 70579 kuberuntime_manager.go:646] createPodSandbox for pod "master-api-master.example.com_kube-syste Apr 02 09:49:56 master.example.com atomic-openshift-node[70579]: E0402 09:49:56.497827 70579 pod_workers.go:186] Error syncing pod 7fa21cbee331f510ca595c258cf8b075 ("master-api-master.exa Apr 02 09:49:56 master.example.com atomic-openshift-node[70579]: I0402 09:49:56.642459 70579 kubelet_node_status.go:269] Setting node annotation to enable volume controller attach/detach Apr 02 09:49:56 master.example.com atomic-openshift-node[70579]: I0402 09:49:56.652331 70579 kubelet_node_status.go:441] Recording NodeHasSufficientDisk event message for node master.exam Apr 02 09:49:56 master.example.com atomic-openshift-node[70579]: I0402 09:49:56.652358 70579 kubelet_node_status.go:441] Recording NodeHasSufficientMemory event message for node master.ex Apr 02 09:49:56 master.example.com atomic-openshift-node[70579]: I0402 09:49:56.652371 70579 kubelet_node_status.go:441] Recording NodeHasNoDiskPressure event message for node master.exam Apr 02 09:49:56 master.example.com atomic-openshift-node[70579]: I0402 09:49:56.652380 70579 kubelet_node_status.go:441] Recording NodeHasSufficientPID event message for node master.examp Apr 02 09:49:56 master.example.com atomic-openshift-node[70579]: I0402 09:49:56.652398 70579 kubelet_node_status.go:79] Attempting to register node master.example.com Apr 02 09:49:56 master.example.com atomic-openshift-node[70579]: E0402 09:49:56.653159 70579 kubelet_node_status.go:103] Unable to register node "master.example.com" with API server: Post Apr 02 09:49:57 master.example.com atomic-openshift-node[70579]: E0402 09:49:57.073032 70579 reflector.go:136] k8s.io/kubernetes/pkg/kubelet/kubelet.go:464: Failed to list *v1.Node: Get h Apr 02 09:49:57 master.example.com atomic-openshift-node[70579]: E0402 09:49:57.163560 70579 reflector.go:136] k8s.io/kubernetes/pkg/kubelet/config/apiserver.go:47: Failed to list *v1.Pod Apr 02 09:49:57 master.example.com atomic-openshift-node[70579]: E0402 09:49:57.214630 70579 reflector.go:136] k8s.io/kubernetes/pkg/kubelet/kubelet.go:455: Failed to list *v1.Service: Ge Apr 02 09:49:58 master.example.com atomic-openshift-node[70579]: E0402 09:49:58.076850 70579 reflector.go:136] k8s.io/kubernetes/pkg/kubelet/kubelet.go:464: Failed to list *v1.Node: Get h Apr 02 09:49:58 master.example.com atomic-openshift-node[70579]: E0402 09:49:58.165052 70579 reflector.go:136] k8s.io/kubernetes/pkg/kubelet/config/apiserver.go:47: Failed to list *v1.Pod Apr 02 09:49:58 master.example.com atomic-openshift-node[70579]: E0402 09:49:58.217896 70579 reflector.go:136] k8s.io/kubernetes/pkg/kubelet/kubelet.go:455: Failed to list *v1.Service: Ge Apr 02 09:49:58 master.example.com atomic-openshift-node[70579]: E0402 09:49:58.440528 70579 event.go:212] Unable to write event: 'Patch https://master.example.com:8443/api/v1/namespaces/ Apr 02 09:49:59 master.example.com atomic-openshift-node[70579]: E0402 09:49:59.078802 70579 reflector.go:136] k8s.io/kubernetes/pkg/kubelet/kubelet.go:464: Failed to list *v1.Node: Get h Apr 02 09:49:59 master.example.com atomic-openshift-node[70579]: E0402 09:49:59.169758 70579 reflector.go:136] k8s.io/kubernetes/pkg/kubelet/config/apiserver.go:47: Failed to list *v1.Pod Apr 02 09:49:59 master.example.com atomic-openshift-node[70579]: E0402 09:49:59.221433 70579 reflector.go:136] k8s.io/kubernetes/pkg/kubelet/kubelet.go:455: Failed to list *v1.Service: Ge Apr 02 09:50:00 master.example.com atomic-openshift-node[70579]: E0402 09:50:00.081681 70579 reflector.go:136] k8s.io/kubernetes/pkg/kubelet/kubelet.go:464: Failed to list *v1.Node: Get h Apr 02 09:50:00 master.example.com atomic-openshift-node[70579]: E0402 09:50:00.171305 70579 reflector.go:136] k8s.io/kubernetes/pkg/kubelet/config/apiserver.go:47: Failed to list *v1.Pod Apr 02 09:50:00 master.example.com atomic-openshift-node[70579]: E0402 09:50:00.227531 70579 reflector.go:136] k8s.io/kubernetes/pkg/kubelet/kubelet.go:455: Failed to list *v1.Service: Ge Apr 02 09:50:00 master.example.com atomic-openshift-node[70579]: W0402 09:50:00.378112 70579 cni.go:172] Unable to update cni config: No networks found in /etc/cni/net.d Apr 02 09:50:00 master.example.com atomic-openshift-node[70579]: E0402 09:50:00.378241 70579 kubelet.go:2101] Container runtime network not ready: NetworkReady=false reason:NetworkPluginN Apr 02 09:50:01 master.example.com atomic-openshift-node[70579]: E0402 09:50:01.086308 70579 reflector.go:136] k8s.io/kubernetes/pkg/kubelet/kubelet.go:464: Failed to list *v1.Node: Get h Apr 02 09:50:01 master.example.com atomic-openshift-node[70579]: E0402 09:50:01.174599 70579 reflector.go:136] k8s.io/kubernetes/pkg/kubelet/config/apiserver.go:47: Failed to list *v1.Pod Apr 02 09:50:01 master.example.com systemd[1]: Started Session 53 of user root. -- Subject: Unit session-53.scope has finished start-up -- Defined-By: systemd
-- Support: http://lists.freedesktop.org/mailman/listinfo/systemd-devel -- Unit session-53.scope has finished starting up.

-- The start-up result is done. Apr 02 09:50:01 master.example.com CROND[75108]: (root) CMD (/usr/lib64/sa/sa1 1 1) Apr 02 09:50:01 master.example.com atomic-openshift-node[70579]: E0402 09:50:01.228561 70579 reflector.go:136] k8s.io/kubernetes/pkg/kubelet/kubelet.go:455: Failed to list *v1.Service: Ge Apr 02 09:50:02 master.example.com atomic-openshift-node[70579]: E0402 09:50:02.089121 70579 reflector.go:136] k8s.io/kubernetes/pkg/kubelet/kubelet.go:464: Failed to list *v1.Node: Get h Apr 02 09:50:02 master.example.com atomic-openshift-node[70579]: E0402 09:50:02.177771 70579 reflector.go:136] k8s.io/kubernetes/pkg/kubelet/config/apiserver.go:47: Failed to list *v1.Pod Apr 02 09:50:02 master.example.com atomic-openshift-node[70579]: E0402 09:50:02.233038 70579 reflector.go:136] k8s.io/kubernetes/pkg/kubelet/kubelet.go:455: Failed to list *v1.Service: Ge Apr 02 09:50:02 master.example.com atomic-openshift-node[70579]: I0402 09:50:02.522926 70579 kubelet_node_status.go:269] Setting node annotation to enable volume controller attach/detach Apr 02 09:50:02 master.example.com atomic-openshift-node[70579]: I0402 09:50:02.537341 70579 kubelet_node_status.go:441] Recording NodeHasSufficientDisk event message for node master.exam Apr 02 09:50:02 master.example.com atomic-openshift-node[70579]: I0402 09:50:02.537382 70579 kubelet_node_status.go:441] Recording NodeHasSufficientMemory event message for node master.ex Apr 02 09:50:02 master.example.com atomic-openshift-node[70579]: I0402 09:50:02.537397 70579 kubelet_node_status.go:441] Recording NodeHasNoDiskPressure event message for node master.exam Apr 02 09:50:02 master.example.com atomic-openshift-node[70579]: I0402 09:50:02.537413 70579 kubelet_node_status.go:441] Recording NodeHasSufficientPID event message for node master.examp Apr 02 09:50:02 master.example.com atomic-openshift-node[70579]: I0402 09:50:02.839155 70579 kuberuntime_manager.go:385] No sandbox for pod "master-etcd-master.example.com_kube-system(21c Apr 02 09:50:02 master.example.com dockerd-current[41369]: time="2019-04-02T09:50:02.841017385-07:00" level=error msg="Handler for GET /v1.26/images/registry.redhat.io/openshift3/ose-pod:v3 Apr 02 09:50:02 master.example.com dockerd-current[41369]: time="2019-04-02T09:50:02.841462955-07:00" level=error msg="Handler for GET /v1.26/images/registry.redhat.io/openshift3/ose-pod:v3 [root@master openshift-ansible]#

Can you tell me a little bit more about your setup?

Do you have a loadbalancer in frond of the masters?

and do you have the following variable:

Set cluster_hostname to point at your load balancer

openshift_master_cluster_hostname: ocp-internal.example.cloud

greetings,

Jan Kappert

Hi Jan,

Please find my below request flow... https request (port 8443) --> Load balancer --> master 1, master 2 and master 3

do i want to add any additional variable in the inventory file or how do i resolve my issue. Waiting for your reply.