"ACMThanosCompactHalted" alert is firing

Issue

• After upgrading from Red Hat Advanced Cluster Management for Kubernetes (RHACM) 2.8 to 2.9, the alert ACMThanosCompactHalted is firing.

• The log of the Thanos Compactor in the namespace open-cluster-management-observability is showing the following error:

  $ oc logs observability-thanos-compact-0
  [..]
  ts=2024-01-24T15:34:51.948653839Z caller=compact.go:491 level=error msg="critical error detected; halting" err="compaction: group 0@15699422364132557315: compact blocks [/var/thanos/compact/compact/0@15699422364132557315/01HKZGQGJCKQWF3XMA8EXAMPLE /var/thanos/compact/compact/0@15699422364132557315/01HKZQK7TD06J2XWGR5EXAMPLE /var/thanos/compact/compact/0@15699422364132557315/01HKZYEZ2DVDQXF1STVEXAMPLE /var/thanos/compact/compact/0@15699422364132557315/01HM05APAHXBQSNC0N5EXAMPLE]: populate block: chunk iter: cannot populate chunk 8 from block 01HKZYEZ2DVDQXF1STVEXAMPLE: segment index 0 out of range"
  [..]