Distributing incoming requests equally to available pod replicas

Solution Verified - Updated -

Environment

  • Red Hat OpenShift Container Platform (RHOCP)
    • 4
  • Red Hat OpenShift Service on AWS (ROSA)
    • 4
  • Red Hat OpenShift Dedicated 4 (OSD)
    • 4
  • Azure Red Hat OpenShift (ARO)
    • 4

Issue

  • Incoming requests are not getting distributed equally to all available pods.
  • How to balance the load among available pod replicas?
  • Is it possible to configure load balancing algorithm on router?

Resolution

When any request calls a pod externally, it goes from Router to Service to Pod. The default behavior of the HAProxy router is to use a cookie to ensure "sticky" routing. This enables sessions to remain with the same pod.

This behavior can be disabled by setting a haproxy.router.openshift.io/disable_cookies annotation on the route to true.

After disabling "sticky" routing with annotation above, it's possible to select a loadbalancing algorithm to balance the incoming requests. Use the annotation to select a loadbalancing algorithm with haproxy.router.openshift.io/balance as key and one of roundrobin, leastconn, source or random as value. The default value was set to random in NE-825.

How to choose between these load balancing algorithms?

Round-robin algorithm will distribute the requests evenly on pods and it works best when pods/servers have roughly identical computing capabilities and storage capacity. On the other hand, leastconn load balancing is a dynamic load balancing algorithm where client requests are distributed to the application server with the least number of active connections at the time the client request is received.

Available annotations for reference: Route-specific annotations

Root Cause

The default behavior of the HAProxy router is to use a cookie to ensure "sticky" routing. This enables sessions to remain with the same pod resulting in few pods getting overloaded where as few pods being less utilized.

This solution is part of Red Hat’s fast-track publication program, providing a huge library of solutions that Red Hat engineers have created while supporting our customers. To give you the knowledge you need the instant it becomes available, these articles may be presented in a raw and unedited form.

Comments