The Elasticsearch pod went into CrashLoopBackoff state after upgrade.

Solution Verified - Updated -

Issue

  • The Elasticsearch pod went into CrashLoopBackoff state after upgrading the cluster to 4.4.X.
# oc get pods
NAME                                            READY   STATUS             RESTARTS   AGE
elasticsearch-cdm-h853uucb-1-b8cc6b896-m5jtl    1/2     CrashLoopBackOff   0          6h22m
elasticsearch-cdm-h853uucb-2-6847cfcc75-zjvhf   1/2     CrashLoopBackOff   0          6h22m
elasticsearch-cdm-h853uucb-3-7bf8745969-hqf75   1/2     CrashLoopBackOff   0          6h22m
  • The Elasticsearch pod went into CrashLoopBackoff state and pod logs show "Could not resolve placeholder 'POD_IP'".
# oc logs elasticsearch-cdm-xxxxxx-1-xxxxxx -c elasticsearch
````
Warning:
The JKS keystore uses a proprietary format. It is recommended to migrate to PKCS12 which is an industry standard format using "keytool -importkeystore -srckeystore /etc/elasticsearch/secret/logging-es.jks -destkeystore /etc/elasticsearch/secret/logging-es.jks -deststoretype pkcs12".
Certificate was added to keystore

Warning:
The JKS keystore uses a proprietary format. It is recommended to migrate to PKCS12 which is an industry standard format using "keytool -importkeystore -srckeystore /etc/elasticsearch/secret/logging-es.jks -destkeystore /etc/elasticsearch/secret/logging-es.jks -deststoretype pkcs12".
Certificate was added to keystore
Certificate was added to keystore
[2020-04-15 02:51:00,248][INFO ][container.run            ] Setting heap dump location /elasticsearch/persistent/heapdump.hprof
[2020-04-15 02:51:00,249][INFO ][container.run            ] ES_JAVA_OPTS: ' -Xms1024m -Xmx1024m -XX:HeapDumpPath=/elasticsearch/persistent/heapdump.hprof -Dsg.display_lic_none=false -Dio.netty.recycler.maxCapacityPerThread=0 -Dio.netty.allocator.type=unpooled'
[2020-04-15 02:51:00,249][INFO ][container.run            ] Checking if Elasticsearch is ready
OpenJDK 64-Bit Server VM warning: If the number of processors is expected to increase from one, then you should configure the number of parallel GC threads appropriately using -XX:ParallelGCThreads=N
Exception in thread "main" java.lang.IllegalArgumentException: Could not resolve placeholder 'POD_IP'
        at org.elasticsearch.common.settings.PropertyPlaceholder.parseStringValue(PropertyPlaceholder.java:116)
        at org.elasticsearch.common.settings.PropertyPlaceholder.replacePlaceholders(PropertyPlaceholder.java:69)
        at org.elasticsearch.common.settings.Settings$Builder.replacePropertyPlaceholders(Settings.java:1109)
        at org.elasticsearch.common.settings.Settings$Builder.replacePropertyPlaceholders(Settings.java:1069)
        at org.elasticsearch.node.InternalSettingsPreparer.initializeSettings(InternalSettingsPreparer.java:137)
        at org.elasticsearch.node.InternalSettingsPreparer.prepareEnvironment(InternalSettingsPreparer.java:115)
        at org.elasticsearch.cli.EnvironmentAwareCommand.createEnv(EnvironmentAwareCommand.java:75)
        at org.elasticsearch.cli.EnvironmentAwareCommand.execute(EnvironmentAwareCommand.java:70)
        at org.elasticsearch.cli.Command.mainWithoutErrorHandling(Command.java:134)
        at org.elasticsearch.cli.Command.main(Command.java:90)
        at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:91)
        at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:84)

Environment

  • Red Hat OpenShift Container Platform
    • 4.4.X

Subscriber exclusive content

A Red Hat subscription provides unlimited access to our knowledgebase, tools, and much more.

Current Customers and Partners

Log in for full access

Log In

New to Red Hat?

Learn more about Red Hat subscriptions

Using a Red Hat product through a public cloud?

How to access this content