PrometheusScrapeBodySizeLimitHit alert troubleshooting in OpenShift Container Platform 4

Issue

The alert PrometheusScrapeBodySizeLimitHit is triggered when at least one of Prometheus' scrape targets replies with a response body larger than the configured body_size_limit. By default, there is no limit on the body size of the scraped targets. When set, this limit prevents Prometheus from consuming excessive amounts of memory when scraped targets return a response that is deemed too large.

Metrics coming from targets responding with a body exceeding the configured size limit aren't ingested by Prometheus. The targets will be considered as being down and they will have their up metric set to 0 which may also trigger the TargetDown alert.