Fluentd fails to send logs with unknown property

Solution Verified - Updated -

Issue

  • Fluentd fills up with illegal argument exceptions due to "id" or other custom properties
2018-06-14 14:33:21 +0200 [debug]: Elasticsearch errors returned, retrying:  {"took"=>2, "errors"=>true, "items"=>[{"create"=>{"_index"=>"project.test-prod.286e429c-8efd-11e6-b4e5-0050569a3c52.2018.06.14", "_type"=>"com.redhat.viaq.common", "_id"=>"ZmU3YTE5NzYtMzIwZi00Njg1LTlmZTMtOTViN2Q1YjU4NzVi", "status"=>400, "error"=>{"type"=>"mapper_parsing_exception", "reason"=>"failed to parse [res.body]", "caused_by"=>{"type"=>"illegal_argument_exception", "reason"=>"unknown property [id]"}}}}, . . . "status"=>409, "error"=>{"type"=>"document_already_exists_exception", "reason"=>"[com.redhat.viaq.common][NTdhMGVhY2QtYTVmYy00MzQyLTk3YTQtZjJjMWU2YzQxYjBi]: document already exists", "shard"=>"0", . . .

2018-06-14 14:33:21 +0200 [warn]: temporarily failed to flush the buffer. next_retry=2018-06-14 14:38:21 +0200 error_class="Fluent::ElasticsearchErrorHandler::ElasticsearchError" error="Elasticsearch returned errors, retrying. Add '@log_level debug' to your config to see the full response" plugin_id="object:3ff3d4816f0c"
  2018-06-14 14:33:21 +0200 [warn]: suppressed same stacktrace

Environment

  • Red Hat OpenShift Container Platform
    • 3.X

Subscriber exclusive content

A Red Hat subscription provides unlimited access to our knowledgebase, tools, and much more.

Current Customers and Partners

Log in for full access

Log In

New to Red Hat?

Learn more about Red Hat subscriptions

Using a Red Hat product through a public cloud?

How to access this content