record cannot use elasticsearch index name type project_full: record is missing kubernetes field
Issue
- The operation logs, are written well, however, we didn't see any application log.
Reading the fluent logs, we see that this message is repeated a lot of times and all the messages are like this :
[error]: record cannot use elasticsearch index name type project_full: record is missing kubernetes field
- Lots of orphaned indices
health status index pri rep docs.count docs.deleted store.size pri.store.size
green open .orphaned.2018.03.11 1 0 2318 0 0 0
green open .kibana.f06c2de9f8edacbf8d8c80d7c5124d865484d701 1 0 2 0 0 0
green open .orphaned.2018.03.10 1 0 3259 0 0 0
green open .orphaned.2018.03.15 1 0 9938436 0 5391 5391
green open .orphaned.2018.03.14 1 0 9159668 0 6411 6411
green open .orphaned.2018.03.13 1 0 17209 0 3 3
green open .orphaned.2018.03.12 1 0 20202 0 3 3
green open .searchguard.logging-es-data-master-lui0t6o3 1 1 5 1 0 0
green open fluentd 1 0 34750 0 25 25
green open .kibana.d8c8066e1bac5ae4c4e8677a7c09c49cedb12e7a 1 0 2 0 0 0
green open .searchguard.logging-es-data-master-5rpso23p 1 2 5 1 0 0
green open .searchguard.logging-es-data-master-w28w7g3s 1 1 5 1 0 0
green open .kibana 1 0 1 0 0 0
green open .orphaned.2018.03.19 1 0 3590307 0 810 810
green open .orphaned.2018.03.18 1 0 5058668 0 1106 1106
green open .orphaned.2018.03.17 1 0 5115571 0 1132 1132
green open .orphaned.2018.03.16 1 0 7506136 0 2845 2845
green open .orphaned.2018.03.09 1 0 3520 0 0 0
- Fluentd index causing strange message with a lot of slashes or backslashes
record cannot use elasticsearch index name type project_full: record is missing kubernetes field: {"message"=>"\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\...
Environment
- Red Hat OpenShift Container Platform
- 3.6
Subscriber exclusive content
A Red Hat subscription provides unlimited access to our knowledgebase, tools, and much more.