OpenShift 4: IPV6 Packet handling rate limits and kubelet health probe failures on OVN
Issue
- IPV6 packets from many (thousands) of unique IP addresses making simultaneous requests against a service running on OpenShift can lead to instability on the host node, causing Kubelet to start failing health probes for all pods running on that target host.
- This behavior is NOT observed when the source IPs are using IPv4 traffic.
- Rate limitations and significant performance loss when traffic type is ipv6 compared to ipv4 on Openshift.
- All pods on a target host start failing health probes, CPU is running at 100% on the target host, OVS process is consuming enormous memory/cpu resourcing while under stress test/load from IPV6 traffic streamed from many different client IPs.
- If IPV6 traffic rate comes from single IP source or a few, the behavior is not present - observed when there are thousands of unique client IPv6 IPs making requests simultaneously. IPV4 traffic cannot replicate with a few clients or thousands of clients.
Environment
- Red Hat OpenShift Container Platform (RHOCP) 4.14+
- OpenShift-OVN-Kubernetes (OVN) ClusterNetworkInterface (CNI)
Subscriber exclusive content
A Red Hat subscription provides unlimited access to our knowledgebase, tools, and much more.