Prometheus: Prometheus: Opening storage failed find blocks

Solution Verified - Updated -


  • Prometheus pod is in CrashLoopBack state with the error below:

    $ oc logs <prometheus pod> -c prometheus
    level=info ts=2020-01-20T11:31:51.966720313Z caller=web.go:415 component=web msg="Start listening for connections" address=
    level=info ts=2020-01-20T11:31:51.966169539Z caller=main.go:533 msg="Starting TSDB ..."
    level=info ts=2020-01-20T11:31:51.967487728Z caller=repair.go:39 component=tsdb msg="found healthy block" mint=1577880000000 maxt=1577944800000 ulid=01DXJQWTJAHCNPEZYJN3QH92FJ
    level=info ts=2020-01-20T11:31:51.972484595Z caller=repair.go:39 component=tsdb msg="found healthy block" mint=1579219200000 maxt=1579226400000 ulid=01DYRQ2X1AM517EJM4ST1CN8Q1
    level=info ts=2020-01-20T11:31:53.396927938Z caller=main.go:402 msg="Stopping scrape discovery manager..."
    level=info ts=2020-01-20T11:31:53.396990862Z caller=main.go:416 msg="Stopping notify discovery manager..."
    level=error ts=2020-01-20T11:31:53.396990862Z caller=main.go:596 err="Opening storage failed find blocks: lstat /prometheus/01DTYBMKE9QFNRW5M5B2KRD6CJ.tmp: stale NFS file handle"


  • OpenShift Container Platform 3.x

Subscriber exclusive content

A Red Hat subscription provides unlimited access to our knowledgebase of over 48,000 articles and solutions.

Current Customers and Partners

Log in for full access

Log In