Fluentd pod keeps restarting and stuck in CrashLoopBackOff
Issue
- All the
logging-fluentd-xxxxxpods inopenshift-loggingnamespace keeps crashing.
logging-fluentd-jj77j 0/1 CrashLoopBackOff 1131 4d 10.0.0.1 node02.example.com <none>
logging-fluentd-lwp6z 0/1 CrashLoopBackOff 549 4d 10.0.0.2 node01.example.com <none>
logging-fluentd-s9nsp 0/1 CrashLoopBackOff 834 4d 10.0.0.3 node03.example.com <none>
Dockerservice logs on the node show below error messages:
Aug 22 02:03:17 node1.example.com dockerd-current[17000]: time="2020-08-22T02:03:17.722980022-04:00" level=error msg="Handler for POST /v1.26/containers/create?name=k8s_fluentd-elasticsearch_logging-fluentd-mksxs_openshift-logging_bc70e819-ec22-11e9-b59b-566f2fcb01c7_6332 returned error: Conflict. The container name \"/k8s_fluentd-elasticsearch_logging-fluentd-mksxs_openshift-logging_bc70e819-ec22-11e9-b59b-566f2fcb01c7_6332\" is already in use by container 05a8e05ccbac065ff191ae840f07a21918b26afe30047008eb7dea4d8feef06c. You have to remove (or rename) that container to be able to reuse that name."
- Fluentd pod logs show below error messages:
2020-08-23 11:14:43 -0400 [error]: unexpected error error="getaddrinfo: Name or service not known"
2020-08-23 11:14:43 -0400 [error]: /usr/share/gems/gems/fluentd-0.12.43/lib/fluent/plugin/out_forward.rb:508:in `getaddrinfo'
2020-08-23 11:14:43 -0400 [error]: /usr/share/gems/gems/fluentd-0.12.43/lib/fluent/plugin/out_forward.rb:508:in `resolve_dns!'
2020-08-23 11:14:43 -0400 [error]: /usr/share/gems/gems/fluentd-0.12.43/lib/fluent/plugin/out_forward.rb:494:in `resolved_host'
- Restarting of fluentd pods manually does not help.
Environment
- Red Hat OpenShift Container Platform
- 3.11
Subscriber exclusive content
A Red Hat subscription provides unlimited access to our knowledgebase, tools, and much more.