Pods fail to start in OpenShift 3.11 on OpenStack 13 using Kuryr CNI
Issue
- OpenShift on OpenShift using Kuryr (OSP 14 Image), the Kuryr CNI Daemon keeps failing with error below (pulled from
oc logs kuryr-cni-xxxxx
), prevents new Pods from getting IP, and prevents Pod from starting. The error below keeps repeating in the Kuryr CNI Pod logs:
2019-08-23 23:32:40.578 180 ERROR kuryr_kubernetes.cni.daemon.service [-] Error when processing addNetwork request. CNI Params: {'CNI_IFNAME': u'eth0', 'CNI_NETNS': u'/proc/20609/ns/net', 'C
NI_PATH': u'/opt/cni/bin', 'CNI_ARGS': u'IgnoreUnknown=1;K8S_POD_NAMESPACE=default;K8S_POD_NAME=docker-registry-1-deploy;K8S_POD_INFRA_CONTAINER_ID=e2df207e0dbb40479a331ced9fa7655952a73b7e8c
c3fd2e5e3759735b7ffd04', 'CNI_DAEMON': u'True', 'CNI_CONFIG_DIR_PATH': u'/etc/cni/net.d', 'CNI_COMMAND': u'ADD', 'CNI_BIN_DIR_PATH': u'/opt/cni/bin', 'CNI_CONTAINERID': u'e2df207e0dbb40479a3
31ced9fa7655952a73b7e8cc3fd2e5e3759735b7ffd04'}: RetryError: RetryError[Attempts: 61, Value: {'eth0': VIFVlanNested(active=False,address=fa:16:3e:1c:45:a0,has_traffic_filtering=False,id=f1be
ddb2-ab98-46b8-8812-305ae2fc9908,network=Network(bc0cbca2-fd44-46ae-89ea-074321b3f00a),plugin='noop',port_profile=<?>,preserve_on_delete=False,vif_name='tapf1beddb2-ab',vlan_id=3252)}]
2019-08-23 23:32:40.578 180 ERROR kuryr_kubernetes.cni.daemon.service Traceback (most recent call last):
2019-08-23 23:32:40.578 180 ERROR kuryr_kubernetes.cni.daemon.service File "/usr/lib/python2.7/site-packages/kuryr_kubernetes/cni/daemon/service.py", line 81, in add
2019-08-23 23:32:40.578 180 ERROR kuryr_kubernetes.cni.daemon.service vif = self.plugin.add(params)
2019-08-23 23:32:40.578 180 ERROR kuryr_kubernetes.cni.daemon.service File "/usr/lib/python2.7/site-packages/kuryr_kubernetes/cni/plugins/k8s_cni_registry.py", line 80, in add
2019-08-23 23:32:40.578 180 ERROR kuryr_kubernetes.cni.daemon.service vifs = wait_for_active(pod_name)
2019-08-23 23:32:40.578 180 ERROR kuryr_kubernetes.cni.daemon.service File "/usr/lib/python2.7/site-packages/retrying.py", line 68, in wrapped_f
2019-08-23 23:32:40.578 180 ERROR kuryr_kubernetes.cni.daemon.service return Retrying(*dargs, **dkw).call(f, *args, **kw)
2019-08-23 23:32:40.578 180 ERROR kuryr_kubernetes.cni.daemon.service File "/usr/lib/python2.7/site-packages/retrying.py", line 231, in call
2019-08-23 23:32:40.578 180 ERROR kuryr_kubernetes.cni.daemon.service raise RetryError(attempt)
2019-08-23 23:32:40.578 180 ERROR kuryr_kubernetes.cni.daemon.service RetryError: RetryError[Attempts: 61, Value: {'eth0': VIFVlanNested(active=False,address=fa:16:3e:1c:45:a0,has_traffic_filtering=False,id=f1beddb2-ab98-46b8-8812-305ae2fc9908,network=Network(bc0cbca2-fd44-46ae-89ea-074321b3f00a),plugin='noop',port_profile=<?>,preserve_on_delete=False,vif_name='tapf1beddb2-ab',vlan_id=3252)}]
2019-08-23 23:32:40.578 180 ERROR kuryr_kubernetes.cni.daemon.service
2019-08-23 23:32:40.581 180 INFO werkzeug [-] 127.0.0.1 - - [23/Aug/2019 23:32:40] "POST /addNetwork HTTP/1.1" 500 -
Environment
- Red Hat OpenShift Container Platform
- 3.11
- Red Hat OpenStack Platform 13
- OVN
- Kuryr CNI 0.5.4
Subscriber exclusive content
A Red Hat subscription provides unlimited access to our knowledgebase, tools, and much more.