Ingress Operator degraded due to CanaryChecksRepetitiveFailures failures intermittently in OCP4 IPI cluster

Issue

• Ingress , Console and Authentication cluster operators intermittently gets degraded:

  NAME                                      VERSION  AVAILABLE  PROGRESSING  DEGRADED  SINCE
  authentication                            4.x      False      False        True      7m18s
  console                                   4.x      False      False        True      6m43s
  ingress                                   4.x      True       False        True      3m45s
  

• Ingress Operator throws the below error:

    - lastTransitionTime: 'Date'
      message: 'The "default" ingress controller reports Degraded=True: DegradedConditions:
        One or more other status conditions indicate a degraded state: CanaryChecksSucceeding=False
        (CanaryChecksRepetitiveFailures: Canary route checks for the default ingress
        controller are failing. Last 1 error messages:
  
        error sending canary HTTP Request: Timeout: Get "https://canary-openshift-ingress-canary.apps.example.com":
        context deadline exceeded (Client.Timeout exceeded while awaiting headers) (x9
        over 8m0s))'
      reason: IngressDegraded
      status: 'True'