AWS Load Balancer Operator: Issue with Enabling IP Target Type in OpenShift

Root Cause

Disclaimer: Links contained herein to external website(s) are provided for convenience only. Red Hat has not reviewed the links and is not responsible for the content or its availability. The inclusion of any link to an external website does not imply endorsement by Red Hat of the website or their entities, products or services. You agree that Red Hat is not responsible or liable for any loss or expenses that may result due to your use of (or reliance on) the external site or content.

As per the OpenShift Container Platform 4.12 release notes, the AWS Load Balancer Operator sets the EnableIPTargetType feature gate to false. This means that the AWS Load Balancer controller has disabled support for services and ingress resources for target-type:ip.

The upstream PR titled "CFE-649: Disable ip target type on Service and Ingress resources" further explains the reason for this limitation. The use of target-type=ip on Service and Ingress type resources is not supported in OpenShift due to its dependency on AWS VPC CNI.

Extra:

The alb.ingress.kubernetes.io/target-type annotation in the AWS Load Balancer Controller for Kubernetes determines how traffic should be routed from the Application Load Balancer (ALB) to the backend targets within an Amazon EKS cluster. There are two possible values for this annotation: ip and instance. Here's a breakdown of the differences between the two:

1. alb.ingress.kubernetes.io/target-type: ip

• IP Mode: In this mode, the ALB routes traffic directly to the IP addresses of the pods behind the service or ingress.
• Pod-based Targeting: The ALB targets individual pods, regardless of which node they reside on.
• VPC Peering: This mode is useful when the EKS cluster is peered with another VPC, and you want to route traffic directly to pods even if they are in a different VPC.
• Scaling: As pods scale up or down, the ALB automatically updates its targets based on the pod IPs.
• Limitation: This mode requires the AWS VPC CNI plugin for networking and might not be supported in all Kubernetes distributions or setups.

2. alb.ingress.kubernetes.io/target-type: instance

• Instance Mode: In this mode, the ALB routes traffic to the nodes (EC2 instances) in the EKS cluster. The node then uses kube-proxy to route the traffic to the appropriate pod.
• NodePort Services: This mode works in conjunction with NodePort services. The ALB sends traffic to the NodePort, and then it's forwarded to the appropriate pod.
• Overhead: There's an additional hop in this mode since traffic goes to the node first and then to the pod. This can introduce a slight latency overhead.
• Node-based Targeting: The ALB targets EC2 instances. If a pod is rescheduled to a different node, the traffic will still reach the new pod as long as it's scheduled on a node that's a target of the ALB.
• Compatibility: This mode is more universally compatible since it doesn't rely on specific CNI plugins or direct pod IP targeting.

In Summary:
- The ip target type routes traffic directly to pod IPs, providing more direct routing but requires AWS CNI which Openshift does not support.
- The instance target type routes traffic to nodes (EC2 instances), which then forward the traffic to the appropriate pods, offering broader compatibility at the potential cost of an extra network hop.