Logs missing when a Loki ingester pod is down in RHOCP4
Issue
- Log is missing during 1 node down when using the LokiStack
- One of the Loki ingesters is not working and logs are not visible in the OCP Console
-
The Loki distributor pod is throwing the error:
2024-11-04T09:36:34.132274806Z level=warn ts=2024-11-04T09:36:34.052327523Z caller=logging.go:126 orgID=application msg="POST /loki/api/v1/push (500) 252.549µs Response: \"at least 2 live replicas required, could only find 1\\n\" ws: false; Accept-Encoding: identity; Content-Encoding: snappy; Content-Length: 2745; Content-Type: application/x-protobuf; User-Agent: Vector/0.34.1 (x86_64-unknown-linux-gnu); X-Forwarded-For: 10.129.3.4; X-Forwarded-Prefix: /api/logs/v1/application; X-Scope-Orgid: application; "
Environment
- Red Hat OpenShift Container Platform (RHOCP)
- 4
- Red Hat OpenShift Logging (RHOL)
- 5
- 6
- Red Hat Loki Operator
- 5
- 6
- LokiStack
Subscriber exclusive content
A Red Hat subscription provides unlimited access to our knowledgebase, tools, and much more.