Cluster installation failed with error code OCM3008 : AWSAPIRateLimitExceeded

Solution Verified - Updated -

Environment

  • Red Hat Openshift Container Platform (OCP 4)
  • Red Hat OpenShift Service on AWS (ROSA 4)
  • Red Hat Openshift Dedicated 4 (OSD 4)

Issue

  • Cluster installation failed with error code OCM3008 : AWSAPIRateLimitExceeded
  • Following error log is observed in the install logs:
level=error\nlevel=error msg=Error: error waiting for Route53 Hosted Zone .* creation: timeout while waiting for state to become 'INSYNC' (last state: 'PENDING', timeout: 15m0s)

Resolution

This error code suggests that AWS API rate limit exceeded waiting for Route53. Our advice to the customer is to reattempt the installation.

Root Cause

This issue is caused by AWS throttling the CreateHostedZone request. The terraform provider is not properly handling the throttling response and gets stuck in a state where it does not retry the request. Eventually, the terraform provider times out claiming that it is waiting for the hosted zone to be INSYNC.

This solution is part of Red Hat’s fast-track publication program, providing a huge library of solutions that Red Hat engineers have created while supporting our customers. To give you the knowledge you need the instant it becomes available, these articles may be presented in a raw and unedited form.

Comments