Clair reports "NoSuchKey" error for the empty layer which leaves images in queued status in Quay

Issue

• When a new image is pushed to Quay, it stays in queued scanning status indefinitely and results are never reported or available.

• Clair reports a NoSuchKey for a specific, so-called "empty layer" in its logs:

  {"level":"error", "request_id":"7fd1eda4a4f5586d", "component":"indexer/controller/Controller.Index", "manifest":"sha256:50fcf1a51771f4aacebcab2b7f17eb395b12e25f7b35104b10473ad554d34edc", "state":"FetchLayers","error":"failed to fetch layers: encountered error while fetching a layer: error realizing layer sha256:a3ed95caeb02ffe68cdd9fd84406680ae93d633cb16422d00e8a7c22955b46d4: fetcher: unexpected status code: 404 Not Found (body starts: \"<?xml version=\\\"1.0\\\" encoding=\\\"UTF-8\\\"?><Error><Code>NoSuchKey</Code><BucketName>quay-registry-quay-datasto-0621b784-4fbd-452c-8499-151d3754a182</BucketName><RequestId>tx00000b9c999fc5e15843f-0067473a3e-30d31a4-ocs-storagecluster-cephobjectstore</RequestId><\")", "time":"2024-11-27T15:26:54Z", "message":"error during scan"}
  

• Empty layer is a layer with the cannonical SHA digest sha256:a3ed95caeb02ffe68cdd9fd84406680ae93d633cb16422d00e8a7c22955b46d4.