CanaryChecksRepetitiveFailures: Canary route checks for the default ingress controller are failing in OpenShift 4.17

Issue

• Canary route health check failing with HTTP 503.

• The ingress cluster operator is degraded with the below message:

  The "default" ingress controller reports Degraded=True: DegradedConditions: One or more other status conditions indicate a degraded state: CanaryChecksSucceeding=False (CanaryChecksRepetitiveFailures: Canary route checks for the default ingress controller are failing.
  

  error sending canary HTTP request to "canary-openshift-ingress-canary.apps.xyz.redhat.com" [...] read: connection reset by peer
  

  expected canary request body to contain "Healthcheck requested"
  

• The ingress-operator pod logs shows errors like the following ones:

  ERROR    operator.ingress_controller    controller/controller.go:114    got retryable error; requeueing    {"after": "1m0s", "error": "IngressController is degraded: CanaryChecksSucceeding=False (CanaryChecksRepetitiveFailures: Canary route checks for the default ingress controller are failing. Last 1 error messages:\nexpected canary request body to contain \"Healthcheck requested\" (x2971 over 49h35m19s))"}
  

  ERROR   operator.ingress_controller     controller/controller.go:114    got retryable error; requeueing {"after": "1m0s", "error": "IngressController is degraded: CanaryChecksSucceeding=False (CanaryChecksRepetitiveFailures: Canary route checks for the default ingress controller are failing. Last 3 error messages:\nerror sending canary HTTP request to \"canary-openshift-ingress-canary.apps.xyz.redhat.com\"