Route timeout configuration does not work properly in OCP 4, OSD and ROSA

Solution Verified - Updated -

Environment

  • Red Hat OpenShift Container Platform (RHOCP)
    • 4
  • Red Hat OpenShift Service on AWS (ROSA)
    • 4
  • Red Hat OpenShift Dedicated (OSD)
    • 4
  • AWS
    • AWS CLB

Issue

  • Route timeout configuration does not work properly in OpenShift cluster in AWS.
  • Application times out after 60 seconds in OSD/ROSA.
  • There is an issue with a timeout of calling the public route on OSD or ROSA. After changing the timeout of the router to 70 seconds, there is still a timeout after 60 seconds.

Resolution

Disclaimer: Links contained herein to external website(s) are provided for convenience only. Red Hat has not reviewed the links and is not responsible for the content or its availability. The inclusion of any link to an external website does not imply endorsement by Red Hat of the website or their entities, products or services. You agree that Red Hat is not responsible or liable for any loss or expenses that may result due to your use of (or reliance on) the external site or content.

For OCP 4

In order to get a higher timeout on the router, the annotation haproxy.router.openshift.io/timeout=<timeout><time_unit> needs to be added to the route. Refer to Configuring route timeouts for additional information.

To configure additional timeouts in OCP 4, please refer to Change or modify default timeout in HAProxy configurations in Openshift 4.

Note: Starting with OCP 4.11, it's possible to configure idle connection timeouts for AWS CLB directly in the IngressController as per Configuring Classic Load Balancer timeouts on AWS.

For OSD 4 and ROSA 4

The SRE Team has implemented an ELB timeout of 30 minutes across OSD v4 and ROSA clusters:

However, the default route timeout is currently 30 seconds. Refer to the ROSA documentation to configure the route timeouts with the annotation haproxy.router.openshift.io/timeout in the route resources. The same annotation works also for OSD: Configuring route timeouts.

Note: The timeout of the default IngressController in ROSA STS/PrivateLink clusters is 60 seconds and cannot be changed as explained in Customize the ingresscontroller in OSD and ROSA. The supported configuration to increase the timeout in ROSA STS clusters is using the CDO.

Root Cause

The default timeout on the AWS ELB is 60 seconds. Starting with OCP 4.11, it's possible to configure idle connection timeouts for AWS CLB in the IngressController.

Diagnostic Steps

Check the configuration of the route:

$ oc get route [route_name] -n [namespace_name] -o yaml

Check the configuration of the ingresscontroller:

$ oc get ingresscontroller default-n openshift-ingress-operator

This solution is part of Red Hat’s fast-track publication program, providing a huge library of solutions that Red Hat engineers have created while supporting our customers. To give you the knowledge you need the instant it becomes available, these articles may be presented in a raw and unedited form.

Comments