Metrics upgrade from 3.5 to 3.6.1 failed initially with 'imagepullbackoff' and later failed to create heapster-certs
Issue
- After upgrade to 3.6 and deploying metrics, containers are stuck deploying in
imagepullbackoff.
openshift_metrics_install_metrics=true
openshift_metrics_cassandra_storage_type=emptydir
openshift_hosted_metrics_deploy=true
openshift_hosted_metrics_deployer_prefix=registry.access.redhat.com/openshift3/
openshift_hosted_metrics_deployer_version=v3.6.0
- Using this, the below error is seen.
16:04:30 Warning Failed Failed to pull image "registry.access.redhat.com/openshift3/metrics-hawkular-metrics:v3.6.0": rpc error: code = 2 desc = error parsing HTTP 404 response body: invalid character 'F' looking for beginning of value: "File not found.\""
- Also with this, below error is seen.
8:16:05 PM Warning Failed mount Unable to mount volumes for pod "heapster-4rxxt_openshift-infra(4fc95bdb-d17d-11e7-8a5b-005056b434b3)": timeout expired waiting for volumes to attach/mount for pod "openshift-infra"/"heapster-4rxxt". list of unattached/unmounted volumes=[heapster-certs]
3 times in the last 5 minutes
8:16:05 PM Warning Failed sync Error syncing pod
3 times in the last 5 minutes
8:15:48 PM Warning Failed mount MountVolume.SetUp failed for volume "kubernetes.io/secret/4fc95bdb-d17d-11e7-8a5b-005056b434b3-heapster-certs" (spec.Name: "heapster-certs") pod "4fc95bdb-d17d-11e7-8a5b-005056b434b3" (UID: "4fc95bdb-d17d-11e7-8a5b-005056b434b3") with: secrets "heapster-certs" not found
11 times in the last 7 minutes
Environment
- OpenShift Container Platform (OCP)
- 3.5
- 3.6
Subscriber exclusive content
A Red Hat subscription provides unlimited access to our knowledgebase, tools, and much more.