Missing Deployment and Failed Pods for ElasticSearch After RHOCP 4 Cluster Upgrade
Issue
- ElasticSearch Deployment missing after RHOCP Cluster upgrade
- ElasticSearch Pod fails to start after upgrade
$ oc get pods
elasticsearch-cdm-xxxxxxxx-1-xxxxxxxxx-xxxxxx 0/2 Pending 0 4s
elasticsearch-cdm-xxxxxxxx-2-xxxxxxxxx-xxxxxx 1/2 Running 0 21h
$ oc get deployment
NAME READY UP-TO-DATE AVAILABLE AGE
cluster-logging-operator 1/1 1 1 21h
elasticsearch-cdm-xxxxxxxx-1 ?? 1 ?? 1d
elasticsearch-cdm-xxxxxxxx-2 ?? 1 ?? 1d
kibana 1/1 1 1 1d
Logs of ElasctiSearch
2022-07-19T17:05:17.680297853Z [2022-07-19T17:05:17,680][WARN ][o.e.d.z.ZenDiscovery ] [elasticsearch-cdm-xxxxxxxx-2] not enough master nodes discovered during pinging (found [[Candidate{node={elasticsearch-cdm-xxxxxxxx-2}{XXXXXXXXXXXXXXXXXXX}{XXXXXXXXXXXXXXXXXXX}{10.0.0.0}{10.0.0.0:9300}, clusterStateVersion=-1}]], but needed [2]), pinging again
2022-07-19T17:05:20.682835005Z [2022-07-19T17:05:20,682][WARN ][o.e.d.z.ZenDiscovery ] [elasticsearch-cdm-xxxxxxxx-2] not enough master nodes discovered during pinging (found [[Candidate{node={elasticsearch-cdm-xxxxxxxx-2}{XXXXXXXXXXXXXXXXXXX}{XXXXXXXXXXXXXXXXXXX}{10.0.0.0}{10.0.0.0:9300}, clusterStateVersion=-1}]], but needed [2]), pinging again
2022-07-19T17:05:22.178699330Z [2022-07-19T17:05:22,178][ERROR][c.a.o.s.a.BackendRegistry] [elasticsearch-cdm-xxxxxxxx-2] Not yet initialized
2022-07-19T17:05:23.683644884Z [2022-07-19T17:05:23,683][WARN ][o.e.d.z.ZenDiscovery ] [elasticsearch-cdm-xxxxxxxx-2] not enough master nodes discovered during pinging (found [[Candidate{node={elasticsearch-cdm-xxxxxxxx-2}{XXXXXXXXXXXXXXXXXXX}{XXXXXXXXXXXXXXXXXXX}{10.0.0.0}{10.0.0.0:9300}, clusterStateVersion=-1}]], but needed [2]), pinging again
2022-07-19T17:05:26.684508897Z [2022-07-19T17:05:26,684][WARN ][o.e.d.z.ZenDiscovery ] [elasticsearch-cdm-xxxxxxxx-2] not enough master nodes discovered during pinging (found [[Candidate{node={elasticsearch-cdm-xxxxxxxx-2}{XXXXXXXXXXXXXXXXXXX}{XXXXXXXXXXXXXXXXXXX}{10.0.0.0}{10.0.0.0:9300}, clusterStateVersion=-1}]], but needed [2]), pinging again
Environment
- Red Hat OpenShift Container Platform (RHOCP)
- 4
- Red Hat OpenShift Logging (RHOL) 5
Subscriber exclusive content
A Red Hat subscription provides unlimited access to our knowledgebase, tools, and much more.