Cloud-Credential-Operator in degraded state with CredentialsFailing and DeleteConflict RHOCP 4

Solution Verified - Updated -

Issue

  • cloud-credential-operator (CCO) failed to sync some of the CredentialsRequest and threw a DeleteConflict error:

    2022-06-07T07:00:12.540335561Z time="2022-06-07T07:00:12Z" level=error msg="DeleteConflict: Cannot delete entity, must remove users from group first.\n\tstatus code: 409, request id: 123x456-789y-0123z
    
  • After rotating the access keys and secret keys stored in aws-creds in the kube-system namespace and deleting all the CredentialsRequests for them to pick up the new credentials there are some CredentialRequests, which are not getting synced with, for example (openshift-machine-api-aws, openshift-ingress,openshift-image-registry, etc..) are failing to sync, when checking the status of these it shows following deprovisioning error:

    - lastProbeTime: "2022-06-07T05:02:41Z"
      lastTransitionTime: "2022-06-07T05:02:41Z"
      message: 'failed to deprovision resource: AWS Error: DeleteConflict: Cannot delete
        entity, must remove users from group first., status code: 409'
      reason: CloudCredDeprovisionFailure
      status: "True"
      type: CredentialsDeprovisionFailure
    

Environment

  • Red Hat OpenShift Container Platform (RHOCP)
    • 4
  • Cloud Providers like (AWS, Azure, GCP, etc..)

Subscriber exclusive content

A Red Hat subscription provides unlimited access to our knowledgebase, tools, and much more.

Current Customers and Partners

Log in for full access

Log In

New to Red Hat?

Learn more about Red Hat subscriptions

Using a Red Hat product through a public cloud?

How to access this content