Kibana "Tenant indices migration failed" issue after migration in RHOCP 4
Issue
- After upgrading to RHOL 5.4 Kibana is in Yellow status throwing error
plugin:opendistro_security@6.8.1 Tenant indices migration failed. Workaround in articles Kibana status is Yellow in OpenShift 4 and Kibana in yellow state with error "plugin:opendistro_security@6.8.1 Setting up index template" in OpenShift Container Platform 4 don't work -
Kibana UI shows:
Kibana status is `Yellow` Plugin Status: plugin:opendistro_security@6.8.1 Tenant indices migration failed -
Kibana pod logs show
Authorization exceptionerror withstatusCode: 403$ oc -n openshift-logging logs $(oc -n openshift-logging get pods -l component=kibana -o NAME) -c kibana|grep "Authorization Exception:.*statusCode.*403" {"type":"error","@timestamp":"2022-04-26T19:04:42Z","tags":["error","migration"],"pid":117,"level":"error","error":{"message":"Authorization Exception: Authorization Exception","name":"Error","stack":"Authorization Exception :: {\"path\":\"/_opendistro/_security/tenantinfo\",\"query\":{},\"statusCode\":403,\"response\":\"\"}\n at respond (/opt/app-root/src/node_modules/elasticsearch/src/lib/transport.js:308:15)\n at checkRespForFailure (/opt/app-root/src/node_modules/elasticsearch/src/lib/transport.js:267:7)\n at HttpConnector.<anonymous> (/opt/app-root/src/node_modules/elasticsearch/src/lib/connectors/http.js:166:7)\n at IncomingMessage.wrapper (/opt/app-root/src/node_modules/elasticsearch/node_modules/lodash/lodash.js:4935:19)\n at IncomingMessage.emit (events.js:203:15)\n at endReadableNT (_stream_readable.js:1145:12)\n at process._tickCallback (internal/process/next_tick.js:63:19)"},"message":"Authorization Exception: Authorization Exception"} - Kibana UI shows
Kibana server is not ready yet - Not able to view logs in kibana as kibana shows status is Yellow.
- Kibana is in Yellow state but elasticsearch cluster is healthy.
Environment
- Red Hat OpenShift Container Platform (RHOCP)
- 4
- Red Hat OpenShift Logging (RHOL)
- 5.4
Subscriber exclusive content
A Red Hat subscription provides unlimited access to our knowledgebase, tools, and much more.