Fluentd is in CrashLoopBackOff state after RHOCP upgrade

Solution Verified - Updated -

Issue

  • Fluentd is stuck in CrashLoopBackOff after cluster upgrade.
$ oc get pods -l component=fluentd -n openshift-logging 
NAME                                            READY   STATUS             RESTARTS   AGE
fluentd-xxxxx                                   0/1     CrashLoopBackOff   63         5h6m
fluentd-xxxxx                                   0/1     CrashLoopBackOff   63         5h6m
fluentd-xxxxx                                   0/1     CrashLoopBackOff   63         5h6m
fluentd-xxxxx                                   0/1     CrashLoopBackOff   63         5h6m
fluentd-xxxxx                                   0/1     CrashLoopBackOff   63         5h6m
fluentd-xxxxx                                   0/1     CrashLoopBackOff   63         5h6m
fluentd-xxxxx                                   0/1     CrashLoopBackOff   63         5h6m
  • Below logs are observed in fluentd pods.
2021-01-19T06:11:46.340504814Z Setting each total_size_limit for 2 buffers to 10265531289 bytes
2021-01-19T06:11:46.357700887Z Setting queued_chunks_limit_size for each buffer to 1223
2021-01-19T06:11:46.357700887Z Setting chunk_limit_size for each buffer to 8388608
2021-01-19T06:11:48.965670188Z 2021-01-19 06:11:48 +0000 [error]: config error file="/etc/fluent/fluent.conf" error_class=Fluent::ConfigError error="Invalid Kubernetes API v1 endpoint https://kubernetes.default.svc: Unauthorized"

Environment

  • Red Hat OpenShift Container Platform (RHOCP)
    • 4.5 to 4.6

Subscriber exclusive content

A Red Hat subscription provides unlimited access to our knowledgebase, tools, and much more.

Current Customers and Partners

Log in for full access

Log In

New to Red Hat?

Learn more about Red Hat subscriptions

Using a Red Hat product through a public cloud?

How to access this content