Thanos error connection reset by peer, observability-thanos-compact pod stay in CrashLoopBackOff

Issue

• We setup the observability service, at Red Hat Advanced Cluster Management for Kubernetes. As in the document :
  enabling-observability

• Observability-thanos-store-shard-1 and observability-thanos-compact pods do not work properly and stay in CrashLoopBackOff status.

• Remote source is failed. :
  observability-thanos-store-shard-2-0_open-cluster-management-observability(1c358b60-2650-4862-bbc4-b4e7f2095e27) CrashLoopBackOff relate to errors in thanos-store .

in pods/observability-thanos-store-shard-0-0/thanos-store/thanos-store

2022-11-28T08:46:31.416004136Z level=warn ts=2022-11-28T08:46:31.415958836Z caller=intrumentation.go:67 msg="changing probe status" status=not-ready reason="bucket store initial sync: sync block: BaseFetcher: iter bucket: Get \"http://s3.openshift-storage.svc:443/observability-963e0566-9497-4ffa-a1b8-03f46e8154b8/?location=\": 

Connection closed by foreign host http://s3.openshift-storage.svc:443/observability-963e0566-9497-4ffa-a1b8-03f46e8154b8/?location=. Retry again."

2022-11-28T08:46:31.416212189Z level=info ts=2022-11-28T08:46:31.416196066Z caller=grpc.go:138 service=gRPC/server component=store msg="internal server is shutting down" err="bucket store initial sync: sync
 block: BaseFetcher: iter bucket: Get \"http://s3.openshift-storage.svc:443/observability-963e0566-9497-4ffa-a1b8-03f46e8154b8/?location=\": Connection closed by foreign host http://s3.openshift-storage.svc:443/observability-963e0566-9497-4ffa-a1b8-03f46e8154b8/?location=. Retry again."
2022-11-28T08:46:31.416235516Z level=info ts=2022-11-28T08:46:31.416215011Z caller=grpc.go:151 service=gRPC/server component=store msg="gracefully stopping internal server"
2022-11-28T08:46:31.416288403Z level=info ts=2022-11-28T08:46:31.416266948Z caller=grpc.go:164 service=gRPC/server component=store msg="internal server is shutdown gracefully" err="bucket store initial sync: sync block: BaseFetcher: iter bucket: Get \"http://s3.openshift-storage.svc:443/observability-963e0566-9497-4ffa-a1b8-03f46e8154b8/?location=\": 
Connection closed by foreign host http://s3.openshift-storage.svc:443/observability-963e0566-9497-4ffa-a1b8-03f46e8154b8/?location=. Retry again."