Autoscaler pod in CrashLoopBackOff state after recent OpenShift Container Platform update
Issue
-
After updating OpenShift to 4.14.17 we are observing
autoscaler-default
pod inCrashLoopBackOff
state with the below error being reported.I0327 10:37:39.166179 1 node_instances_cache.go:168] Refresh cloud provider node instances cache finished, refresh took 31.131576ms panic: interface conversion: interface {} is nil, not string goroutine 128 [running]: k8s.io/autoscaler/cluster-autoscaler/cloudprovider/clusterapi.unstructuredToTaint(...) /go/src/k8s.io/autoscaler/cluster-autoscaler/cloudprovider/clusterapi/clusterapi_unstructured.go:246 k8s.io/autoscaler/cluster-autoscaler/cloudprovider/clusterapi.unstructuredScalableResource.Taints({0xc000ba8b60?, 0xc000126f00?, 0xc0009daf80?, 0x2?}) /go/src/k8s.io/autoscaler/cluster-autoscaler/cloudprovider/clusterapi/clusterapi_unstructured.go:214 +0x8a5 k8s.io/autoscaler/cluster-autoscaler/cloudprovider/clusterapi.(*nodegroup).TemplateNodeInfo(0xc00074be20) /go/src/k8s.io/autoscaler/cluster-autoscaler/cloudprovider/clusterapi/clusterapi_nodegroup.go:266 +0x2ea k8s.io/autoscaler/cluster-autoscaler/core/utils.GetNodeInfoFromTemplate({0x26db450, 0xc00074be20}, {0xc0047e2100, 0x16, 0x20}, {0xc003b1d2f0?, 0xc003b1d320?}) /go/src/k8s.io/autoscaler/cluster-autoscaler/core/utils/utils.go:41 +0x9d k8s.io/autoscaler/cluster-autoscaler/processors/nodeinfosprovider.(*MixedTemplateNodeInfoProvider).Process(0xc000959440, 0xc003cd8400, {0xc0065b1740, 0x8, 0x69561a?}, {0xc0047e2100, 0x16, 0x20}, {0xc003b1d2f0, 0xc003b1d320}, ...) /go/src/k8s.io/autoscaler/cluster-autoscaler/processors/nodeinfosprovider/mixed_nodeinfos_processor.go:155 +0x599 k8s.io/autoscaler/cluster-autoscaler/core.(*StaticAutoscaler).RunOnce(0xc0006c6160, {0x4?, 0x0?, 0x3b8baa0?}) /go/src/k8s.io/autoscaler/cluster-autoscaler/core/static_autoscaler.go:352 +0xc8a main.run(0xc0000d7000?, {0x26d1f18, 0xc000742270}) /go/src/k8s.io/autoscaler/cluster-autoscaler/main.go:489 +0x2cd main.main.func2({0x931de6?, 0xc000137380?}) /go/src/k8s.io/autoscaler/cluster-autoscaler/main.go:577 +0x25 created by k8s.io/client-go/tools/leaderelection.(*LeaderElector).Run /go/src/k8s.io/autoscaler/cluster-autoscaler/vendor/k8s.io/client-go/tools/leaderelection/leaderelection.go:208 +0x105
-
We are seeing
autoscaler-default
inCrashLoopBackOff
state after recent update of OpenShift.
Environment
- Red Hat OpenShift Container Platform (RHOCP)
- 4.13.38 and later
- 4.14.17 and later
- 4.15.0 and later
Subscriber exclusive content
A Red Hat subscription provides unlimited access to our knowledgebase, tools, and much more.