Elasticsearch pod went into CrashLoopBackOff with with 'java.io.IOException: No space left on device' in OCP 4.5

Solution Verified - Updated -

Issue

  • Elasticsearch pod went into CrashLoopBackOff with java.io.IOException: No space left on device.
elasticsearch-cdm-wx1qj2o2-1-8cbb6d8f-6s6rq     1/2     CrashLoopBackOff   163        3d10h   10.152.X.Y     node1   <none>           <none>
elasticsearch-cdm-wx1qj2o2-2-6dd654bbf6-6bc2b   2/2     Running            0          3d10h   10.152.X.Y     node2   <none>           <none>
elasticsearch-cdm-wx1qj2o2-3-f8fbdf769-wcrz2    2/2     Running            0          3d10h   10.152.X.Y     node3   <none>           <none>
  • Elasticsearch Logs shows below message:
at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:84) ~[elasticsearch-5.6.16.redhat-2.jar:5.6.16.redhat-2]
') error: org.apache.logging.log4j.core.appender.AppenderLoggingException: Error writing to stream /elasticsearch/persistent/elasticsearch/logs/elasticsearch.log org.apache.logging.log4j.core.appender.AppenderLoggingException: Error writing to stream /elasticsearch/persistent/elasticsearch/logs/elasticsearch.log
        at org.apache.logging.log4j.core.appender.OutputStreamManager.writeToDestination(OutputStreamManager.java:263)
        at org.apache.logging.log4j.core.appender.FileManager.writeToDestination(FileManager.java:261)
        at org.apache.logging.log4j.core.appender.rolling.RollingFileManager.writeToDestination(RollingFileManager.java:219)
        at org.apache.logging.log4j.core.appender.OutputStreamManager.flushBuffer(OutputStreamManager.java:293)
                                                       ... Output Ommitted ...
         at java.lang.Thread.dispatchUncaughtException(Thread.java:1959)
Caused by: java.io.IOException: No space left on device
        at java.io.FileOutputStream.writeBytes(Native Method)
        at java.io.FileOutputStream.write(FileOutputStream.java:326)
        at org.apache.logging.log4j.core.appender.OutputStreamManager.writeToDestination(OutputStreamManager.java:261)
        ... 39 more

Environment

  • Red Hat OpenShift Container Platform
    • 4.5

Subscriber exclusive content

A Red Hat subscription provides unlimited access to our knowledgebase, tools, and much more.

Current Customers and Partners

Log in for full access

Log In

New to Red Hat?

Learn more about Red Hat subscriptions

Using a Red Hat product through a public cloud?

How to access this content