How to disable forwarding of infrastructure logs to the internal ElasticSearch instance of OpenShift Container Platform 4.x
Issue
The ElasticSearch indices starting with infra-...
are constantly increasing and cause the internal ElasticSearch pods to run out of storage. How can one disable the forwarding of infrastructure logs to the internal ElasticSearch instance of OpenShift Container Platform 4.x?
Example to verify this. Below, assume that the infra-...
indices consumed a lot of storage:
$ pod=$(oc get pods --selector component=elasticsearch -o name | head -1)
$ oc exec $pod -- /bin/bash -c 'date; indices; shards --params="h=index,shard,prirep,state,docs,store,ip,node,unassigned.reason"'
Defaulting container name to elasticsearch.
Use 'oc describe pod/elasticsearch-cdm-l60tbpbc-1-6555f6df9b-zsx8q -n openshift-logging' to see all of the containers in this pod.
Wed Jul 21 13:43:52 UTC 2021
Wed Jul 21 13:43:52 UTC 2021
health status index uuid pri rep docs.count docs.deleted store.size pri.store.size
green open .security SFkK8N71ShOI-9zcvFYcEg 1 1 6 0 0 0
green open app-000002 Is5-Bf8eSf-uCuz0PCXrrQ 3 1 0 0 0 0
green open app-000001 JD0MvHUwSuiR4yT27FF69A 3 1 0 0 0 0
green open .kibana_1 yuc_93spQqi80WqyvzOyFQ 1 1 0 0 0 0
green open audit-000001 yBoq0Z5IR46osXfAyR69Pw 3 1 564317 0 937 470
green open infra-000001 0KQu4I2aS6yLf5UiTGAqSw 3 1 1840985 0 2601 1309
green open app-000003 HQcU5XnYRRSPz0zqAY0L6w 3 1 0 0 0 0
.kibana_1 0 r STARTED 0 261b 10.128.4.7 elasticsearch-cdm-l60tbpbc-3
.kibana_1 0 p STARTED 0 261b 10.130.2.18 elasticsearch-cdm-l60tbpbc-1
.security 0 p STARTED 6 32.3kb 10.128.4.7 elasticsearch-cdm-l60tbpbc-3
.security 0 r STARTED 6 32.3kb 10.131.2.7 elasticsearch-cdm-l60tbpbc-2
app-000001 1 p STARTED 0 261b 10.128.4.7 elasticsearch-cdm-l60tbpbc-3
app-000001 1 r STARTED 0 261b 10.131.2.7 elasticsearch-cdm-l60tbpbc-2
app-000001 2 p STARTED 0 261b 10.131.2.7 elasticsearch-cdm-l60tbpbc-2
app-000001 2 r STARTED 0 261b 10.130.2.18 elasticsearch-cdm-l60tbpbc-1
app-000001 0 r STARTED 0 261b 10.128.4.7 elasticsearch-cdm-l60tbpbc-3
app-000001 0 p STARTED 0 261b 10.130.2.18 elasticsearch-cdm-l60tbpbc-1
audit-000001 1 p STARTED 188080 157.4mb 10.128.4.7 elasticsearch-cdm-l60tbpbc-3
audit-000001 1 r STARTED 188080 156.5mb 10.131.2.7 elasticsearch-cdm-l60tbpbc-2
audit-000001 2 p STARTED 188020 156mb 10.131.2.7 elasticsearch-cdm-l60tbpbc-2
audit-000001 2 r STARTED 188020 156.2mb 10.130.2.18 elasticsearch-cdm-l60tbpbc-1
audit-000001 0 r STARTED 188217 153.6mb 10.128.4.7 elasticsearch-cdm-l60tbpbc-3
audit-000001 0 p STARTED 188217 157.3mb 10.130.2.18 elasticsearch-cdm-l60tbpbc-1
app-000003 1 r STARTED 0 261b 10.131.2.7 elasticsearch-cdm-l60tbpbc-2
app-000003 1 p STARTED 0 261b 10.130.2.18 elasticsearch-cdm-l60tbpbc-1
app-000003 2 p STARTED 0 261b 10.128.4.7 elasticsearch-cdm-l60tbpbc-3
app-000003 2 r STARTED 0 261b 10.131.2.7 elasticsearch-cdm-l60tbpbc-2
app-000003 0 p STARTED 0 261b 10.128.4.7 elasticsearch-cdm-l60tbpbc-3
app-000003 0 r STARTED 0 261b 10.130.2.18 elasticsearch-cdm-l60tbpbc-1
app-000002 1 r STARTED 0 261b 10.131.2.7 elasticsearch-cdm-l60tbpbc-2
app-000002 1 p STARTED 0 261b 10.130.2.18 elasticsearch-cdm-l60tbpbc-1
app-000002 2 p STARTED 0 261b 10.128.4.7 elasticsearch-cdm-l60tbpbc-3
app-000002 2 r STARTED 0 261b 10.131.2.7 elasticsearch-cdm-l60tbpbc-2
app-000002 0 p STARTED 0 261b 10.128.4.7 elasticsearch-cdm-l60tbpbc-3
app-000002 0 r STARTED 0 261b 10.130.2.18 elasticsearch-cdm-l60tbpbc-1
infra-000001 1 p STARTED 614865 444.4mb 10.128.4.7 elasticsearch-cdm-l60tbpbc-3
infra-000001 1 r STARTED 614520 437.8mb 10.131.2.7 elasticsearch-cdm-l60tbpbc-2
infra-000001 2 p STARTED 613356 429.5mb 10.131.2.7 elasticsearch-cdm-l60tbpbc-2
infra-000001 2 r STARTED 613338 416.7mb 10.130.2.18 elasticsearch-cdm-l60tbpbc-1
infra-000001 0 r STARTED 613108 437.3mb 10.128.4.7 elasticsearch-cdm-l60tbpbc-3
infra-000001 0 p STARTED 612764 435.8mb 10.130.2.18 elasticsearch-cdm-l60tbpbc-1
Environment
OpenShift Container Platform 4.x
Cluster Logging Operator
Subscriber exclusive content
A Red Hat subscription provides unlimited access to our knowledgebase, tools, and much more.