Loki ingester couldn't transmit logs because of NoCredentialProviders error

Solution Verified - Updated -

Issue

  • Logs are not transmitted to AWS S3 from LokiStack.
  • Loki ingester pods are streaming below logs indefinitely:

    $ oc logs logging-loki-ingester-0 -n openshift-logging | grep NoCredentialProviders
    .........
    level=error ts=2024-06-26T13:19:57.606576446Z caller=flush.go:143 org_id=infrastructure msg="failed to flush" err="failed to flush chunks: store put chunk: NoCredentialProviders: no valid providers in chain. Deprecated.\n\tFor verbose messaging see aws.Config.CredentialsChainVerboseErrors, num_chunks: 1, labels: {kubernetes_container_name=\"osd-delete-backplane-serviceaccounts\", kubernetes_host=\"ip-xx-xx-xx-xx.<region>.compute.internal\", kubernetes_namespace_name=\"openshift-backplane\", kubernetes_pod_name=\"osd-delete-backplane-serviceaccounts-xxxxx-xxxxx\", log_type=\"infrastructure\"}"
    

Environment

  • Red Hat OpenShift Service on AWS (ROSA)
    • 4.14+
  • Red Hat OpenShift Logging (RHOL)
    • 5.9+
  • LokiStack

Subscriber exclusive content

A Red Hat subscription provides unlimited access to our knowledgebase, tools, and much more.

Current Customers and Partners

Log in for full access

Log In

New to Red Hat?

Learn more about Red Hat subscriptions

Using a Red Hat product through a public cloud?

How to access this content