Thanos compactor halted

Issue

The Thanos Compactor is an essential component for the stability of the Observability functionality in Advanced Cluster Management(ACM). The Compactors job is to:

• Delete blocks in object storage when they fall outside the configured metrics retention period.
• Down-sampling blocks to improve query latency

A non-working or halted compactor could have the following negative impacts on the Observability functionality:

• Object storage disk space usage always increasing.
• Increased metric query latency, including dashboards slowly loading in Grafana.
• Increased disk space usage on the Compactor and Store Gateway PVCs.
• Alert ACMThanosCompactHalted firing on the Hub (ACM 2.9+).