How to disable forwarding of infrastructure logs to the internal ElasticSearch instance of OpenShift Container Platform 4.x

Solution In Progress - Updated -

Issue

The ElasticSearch indices starting with infra-... are constantly increasing and cause the internal ElasticSearch pods to run out of storage. How can one disable the forwarding of infrastructure logs to the internal ElasticSearch instance of OpenShift Container Platform 4.x?

Example to verify this. Below, assume that the infra-... indices consumed a lot of storage:

$ pod=$(oc get pods --selector component=elasticsearch -o name | head -1)
$ oc exec $pod -- /bin/bash -c 'date; indices; shards --params="h=index,shard,prirep,state,docs,store,ip,node,unassigned.reason"'
Defaulting container name to elasticsearch.
Use 'oc describe pod/elasticsearch-cdm-l60tbpbc-1-6555f6df9b-zsx8q -n openshift-logging' to see all of the containers in this pod.
Wed Jul 21 13:43:52 UTC 2021
Wed Jul 21 13:43:52 UTC 2021
health status index        uuid                   pri rep docs.count docs.deleted store.size pri.store.size
green  open   .security    SFkK8N71ShOI-9zcvFYcEg   1   1          6            0          0              0
green  open   app-000002   Is5-Bf8eSf-uCuz0PCXrrQ   3   1          0            0          0              0
green  open   app-000001   JD0MvHUwSuiR4yT27FF69A   3   1          0            0          0              0
green  open   .kibana_1    yuc_93spQqi80WqyvzOyFQ   1   1          0            0          0              0
green  open   audit-000001 yBoq0Z5IR46osXfAyR69Pw   3   1     564317            0        937            470
green  open   infra-000001 0KQu4I2aS6yLf5UiTGAqSw   3   1    1840985            0       2601           1309
green  open   app-000003   HQcU5XnYRRSPz0zqAY0L6w   3   1          0            0          0              0
.kibana_1    0 r STARTED      0    261b 10.128.4.7  elasticsearch-cdm-l60tbpbc-3 
.kibana_1    0 p STARTED      0    261b 10.130.2.18 elasticsearch-cdm-l60tbpbc-1 
.security    0 p STARTED      6  32.3kb 10.128.4.7  elasticsearch-cdm-l60tbpbc-3 
.security    0 r STARTED      6  32.3kb 10.131.2.7  elasticsearch-cdm-l60tbpbc-2 
app-000001   1 p STARTED      0    261b 10.128.4.7  elasticsearch-cdm-l60tbpbc-3 
app-000001   1 r STARTED      0    261b 10.131.2.7  elasticsearch-cdm-l60tbpbc-2 
app-000001   2 p STARTED      0    261b 10.131.2.7  elasticsearch-cdm-l60tbpbc-2 
app-000001   2 r STARTED      0    261b 10.130.2.18 elasticsearch-cdm-l60tbpbc-1 
app-000001   0 r STARTED      0    261b 10.128.4.7  elasticsearch-cdm-l60tbpbc-3 
app-000001   0 p STARTED      0    261b 10.130.2.18 elasticsearch-cdm-l60tbpbc-1 
audit-000001 1 p STARTED 188080 157.4mb 10.128.4.7  elasticsearch-cdm-l60tbpbc-3 
audit-000001 1 r STARTED 188080 156.5mb 10.131.2.7  elasticsearch-cdm-l60tbpbc-2 
audit-000001 2 p STARTED 188020   156mb 10.131.2.7  elasticsearch-cdm-l60tbpbc-2 
audit-000001 2 r STARTED 188020 156.2mb 10.130.2.18 elasticsearch-cdm-l60tbpbc-1 
audit-000001 0 r STARTED 188217 153.6mb 10.128.4.7  elasticsearch-cdm-l60tbpbc-3 
audit-000001 0 p STARTED 188217 157.3mb 10.130.2.18 elasticsearch-cdm-l60tbpbc-1 
app-000003   1 r STARTED      0    261b 10.131.2.7  elasticsearch-cdm-l60tbpbc-2 
app-000003   1 p STARTED      0    261b 10.130.2.18 elasticsearch-cdm-l60tbpbc-1 
app-000003   2 p STARTED      0    261b 10.128.4.7  elasticsearch-cdm-l60tbpbc-3 
app-000003   2 r STARTED      0    261b 10.131.2.7  elasticsearch-cdm-l60tbpbc-2 
app-000003   0 p STARTED      0    261b 10.128.4.7  elasticsearch-cdm-l60tbpbc-3 
app-000003   0 r STARTED      0    261b 10.130.2.18 elasticsearch-cdm-l60tbpbc-1 
app-000002   1 r STARTED      0    261b 10.131.2.7  elasticsearch-cdm-l60tbpbc-2 
app-000002   1 p STARTED      0    261b 10.130.2.18 elasticsearch-cdm-l60tbpbc-1 
app-000002   2 p STARTED      0    261b 10.128.4.7  elasticsearch-cdm-l60tbpbc-3 
app-000002   2 r STARTED      0    261b 10.131.2.7  elasticsearch-cdm-l60tbpbc-2 
app-000002   0 p STARTED      0    261b 10.128.4.7  elasticsearch-cdm-l60tbpbc-3 
app-000002   0 r STARTED      0    261b 10.130.2.18 elasticsearch-cdm-l60tbpbc-1 
infra-000001 1 p STARTED 614865 444.4mb 10.128.4.7  elasticsearch-cdm-l60tbpbc-3 
infra-000001 1 r STARTED 614520 437.8mb 10.131.2.7  elasticsearch-cdm-l60tbpbc-2 
infra-000001 2 p STARTED 613356 429.5mb 10.131.2.7  elasticsearch-cdm-l60tbpbc-2 
infra-000001 2 r STARTED 613338 416.7mb 10.130.2.18 elasticsearch-cdm-l60tbpbc-1 
infra-000001 0 r STARTED 613108 437.3mb 10.128.4.7  elasticsearch-cdm-l60tbpbc-3 
infra-000001 0 p STARTED 612764 435.8mb 10.130.2.18 elasticsearch-cdm-l60tbpbc-1 

Environment

OpenShift Container Platform 4.x
Cluster Logging Operator

Subscriber exclusive content

A Red Hat subscription provides unlimited access to our knowledgebase, tools, and much more.

Current Customers and Partners

Log in for full access

Log In