Why does starting td-agent fails with an error "error=#<Errno::ENOENT: No such file or directory @ rb_sysopen - /var/log/td-agent/tmp/fluentd-docker.pos>" ?
Issue
-
When combining https://access.redhat.com/documentation/en/openshift-enterprise/version-3.0/openshift-enterprise-30-administrator-guide/chapter-5-aggregating-container-logs with http://www.fluentd.org/guides/recipes/graylog2 to forward OpenShift log messages into a graylog instance the td-agent fails.
The servers don't have internet access to download rubygems so I downloaded the current version from https://rubygems.org manually. The versions seem to be incomaptible -
Error shown in the td-agent.log-
2015-08-24 13:39:51 +0200 [info]: listening dRuby uri="druby://127.0.0.1:24230" object="Engine"
2015-08-24 13:39:51 +0200 [error]: unexpected error error_class=Errno::ENOENT error=#<Errno::ENOENT: No such file or directory @ rb_sysopen - /var/log/td-agent/tmp/fluentd-docker.pos>
2015-08-24 13:39:51 +0200 [error]: /opt/td-agent/embedded/lib/ruby/gems/2.1.0/gems/fluentd-0.12.7/lib/fluent/plugin/in_tail.rb:78:in `initialize'
2015-08-24 13:39:51 +0200 [error]: /opt/td-agent/embedded/lib/ruby/gems/2.1.0/gems/fluentd-0.12.7/lib/fluent/plugin/in_tail.rb:78:in `open'
2015-08-24 13:39:51 +0200 [error]: /opt/td-agent/embedded/lib/ruby/gems/2.1.0/gems/fluentd-0.12.7/lib/fluent/plugin/in_tail.rb:78:in `start'
2015-08-24 13:39:51 +0200 [error]: /opt/td-agent/embedded/lib/ruby/gems/2.1.0/gems/fluentd-0.12.7/lib/fluent/root_agent.rb:111:in `block in start'
2015-08-24 13:39:51 +0200 [error]: /opt/td-agent/embedded/lib/ruby/gems/2.1.0/gems/fluentd-0.12.7/lib/fluent/root_agent.rb:110:in `each'
2015-08-24 13:39:51 +0200 [error]: /opt/td-agent/embedded/lib/ruby/gems/2.1.0/gems/fluentd-0.12.7/lib/fluent/root_agent.rb:110:in `start'
2015-08-24 13:39:51 +0200 [error]: /opt/td-agent/embedded/lib/ruby/gems/2.1.0/gems/fluentd-0.12.7/lib/fluent/engine.rb:201:in `start'
2015-08-24 13:39:51 +0200 [error]: /opt/td-agent/embedded/lib/ruby/gems/2.1.0/gems/fluentd-0.12.7/lib/fluent/engine.rb:151:in `run'
2015-08-24 13:39:51 +0200 [error]: /opt/td-agent/embedded/lib/ruby/gems/2.1.0/gems/fluentd-0.12.7/lib/fluent/supervisor.rb:481:in `run_engine'
2015-08-24 13:39:51 +0200 [error]: /opt/td-agent/embedded/lib/ruby/gems/2.1.0/gems/fluentd-0.12.7/lib/fluent/supervisor.rb:140:in `block in start'
2015-08-24 13:39:51 +0200 [error]: /opt/td-agent/embedded/lib/ruby/gems/2.1.0/gems/fluentd-0.12.7/lib/fluent/supervisor.rb:266:in `call'
2015-08-24 13:39:51 +0200 [error]: /opt/td-agent/embedded/lib/ruby/gems/2.1.0/gems/fluentd-0.12.7/lib/fluent/supervisor.rb:266:in `main_process'
2015-08-24 13:39:51 +0200 [error]: /opt/td-agent/embedded/lib/ruby/gems/2.1.0/gems/fluentd-0.12.7/lib/fluent/supervisor.rb:241:in `block in supervise'
2015-08-24 13:39:51 +0200 [error]: /opt/td-agent/embedded/lib/ruby/gems/2.1.0/gems/fluentd-0.12.7/lib/fluent/supervisor.rb:240:in `fork'
2015-08-24 13:39:51 +0200 [error]: /opt/td-agent/embedded/lib/ruby/gems/2.1.0/gems/fluentd-0.12.7/lib/fluent/supervisor.rb:240:in `supervise'
2015-08-24 13:39:51 +0200 [error]: /opt/td-agent/embedded/lib/ruby/gems/2.1.0/gems/fluentd-0.12.7/lib/fluent/supervisor.rb:134:in `start'
2015-08-24 13:39:51 +0200 [error]: /opt/td-agent/embedded/lib/ruby/gems/2.1.0/gems/fluentd-0.12.7/lib/fluent/command/fluentd.rb:167:in `<top (required)>'
2015-08-24 13:39:51 +0200 [error]: /opt/td-agent/embedded/lib/ruby/site_ruby/2.1.0/rubygems/core_ext/kernel_require.rb:73:in `require'
2015-08-24 13:39:51 +0200 [error]: /opt/td-agent/embedded/lib/ruby/site_ruby/2.1.0/rubygems/core_ext/kernel_require.rb:73:in `require'
2015-08-24 13:39:51 +0200 [error]: /opt/td-agent/embedded/lib/ruby/gems/2.1.0/gems/fluentd-0.12.7/bin/fluentd:6:in `<top (required)>'
2015-08-24 13:39:51 +0200 [error]: /opt/td-agent/embedded/bin/fluentd:23:in `load'
2015-08-24 13:39:51 +0200 [error]: /opt/td-agent/embedded/bin/fluentd:23:in `<top (required)>'
2015-08-24 13:39:51 +0200 [error]: /usr/sbin/td-agent:7:in `load'
2015-08-24 13:39:51 +0200 [error]: /usr/sbin/td-agent:7:in `<main>'
2015-08-24 13:39:51 +0200 [info]: shutting down fluentd
2015-08-24 13:39:51 +0200 [warn]: unexpected error while shutting down input plugin plugin=Fluent::ForwardInput plugin_id="object:3feb10f69088" error_class=RuntimeError error=#<RuntimeError: loop not running>
2015-08-24 13:39:51 +0200 [warn]: unexpected error while shutting down input plugin plugin=Fluent::HttpInput plugin_id="object:3feb10eb158c" error_class=RuntimeError error=#<RuntimeError: loop not running>
2015-08-24 13:39:51 +0200 [error]: unexpected error error="no watchers for this loop"
2015-08-24 13:39:51 +0200 [error]: unexpected error error=#<RuntimeError: no watchers for this loop> error_class=RuntimeError
2015-08-24 13:39:51 +0200 [warn]: /opt/td-agent/embedded/lib/ruby/gems/2.1.0/gems/cool.io-1.3.0/lib/cool.io/loop.rb:95:in `stop'
2015-08-24 13:39:51 +0200 [warn]: /opt/td-agent/embedded/lib/ruby/gems/2.1.0/gems/cool.io-1.3.0/lib/cool.io/loop.rb:95:in `stop'
2015-08-24 13:39:51 +0200 [warn]: /opt/td-agent/embedded/lib/ruby/gems/2.1.0/gems/fluentd-0.12.7/lib/fluent/plugin/in_http.rb:107:in `shutdown'
2015-08-24 13:39:51 +0200 [warn]: /opt/td-agent/embedded/lib/ruby/gems/2.1.0/gems/fluentd-0.12.7/lib/fluent/root_agent.rb:121:in `block (2 levels) in shutdown'
2015-08-24 13:39:51 +0200 [warn]: /opt/td-agent/embedded/lib/ruby/gems/2.1.0/gems/fluentd-0.12.7/lib/fluent/plugin/in_forward.rb:67:in `shutdown'
2015-08-24 13:39:51 +0200 [warn]: /opt/td-agent/embedded/lib/ruby/gems/2.1.0/gems/fluentd-0.12.7/lib/fluent/root_agent.rb:121:in `block (2 levels) in shutdown'
2015-08-24 13:39:51 +0200 [error]: /opt/td-agent/embedded/lib/ruby/gems/2.1.0/gems/cool.io-1.3.0/lib/cool.io/loop.rb:84:in `run'
2015-08-24 13:39:51 +0200 [error]: /opt/td-agent/embedded/lib/ruby/gems/2.1.0/gems/fluentd-0.12.7/lib/fluent/plugin/in_http.rb:113:in `run'
2015-08-24 13:39:51 +0200 [error]: /opt/td-agent/embedded/lib/ruby/gems/2.1.0/gems/cool.io-1.3.0/lib/cool.io/loop.rb:84:in `run'
2015-08-24 13:39:51 +0200 [error]: /opt/td-agent/embedded/lib/ruby/gems/2.1.0/gems/fluentd-0.12.7/lib/fluent/plugin/in_forward.rb:91:in `run'
2015-08-24 13:39:51 +0200 [info]: process finished code=0
2015-08-24 13:39:51 +0200 [warn]: process died within 1 second. exit.
- Steps executed:
yum -y localinstall td-agent-2.2.0-0.x86_64.rpm
/opt/td-agent/embedded/bin/gem install fluent-plugin-kubernetes-0.3.1.gem docker-api-1.22.2.gem excon-0.45.4.gem gelf-1.4.0.gem
mkdir -p /etc/td-agent/config.d
chown td-agent:td-agent /etc/td-agent/config.d
echo 'DAEMON_ARGS=' >> /etc/sysconfig/td-agent
echo 'TD_AGENT_ARGS="/usr/sbin/td-agent --log /var/log/td-agent/td-agent.log --use-v1-config"' >> /etc/sysconfig/td-agent
echo "@include config.d/*.conf" >> /etc/td-agent/td-agent.conf
cp out_gelf.rb /etc/td-agent/plugin/
systemctl enable td-agent
systemctl start td-agent
Environment
- Openshift 3.0
Subscriber exclusive content
A Red Hat subscription provides unlimited access to our knowledgebase of over 48,000 articles and solutions.
Welcome! Check out the Getting Started with Red Hat page for quick tours and guides for common tasks.
