catalog-operator repeatedly crashes with "runtime error: index out of range [0] with length 0"

Solution Verified - Updated -

Issue

  • After updating to OpenShift Container Platform 4.6.4, the catalog-operator in the namespace 'openshift-operator-lifecycle-manager' will stay in state 'CrashLoopBackOff'. The following error messages are shown:

    [..]
    time="2020-11-11T09:41:45Z" level=info msg=syncing event=update reconciling="*v1alpha1.Subscription" selflink=/apis/operators.coreos.com/v1alpha1/namespaces/openshift-operators/subscriptions/abc-def-operator
    time="2020-11-11T09:41:45Z" level=info msg=syncing id=VyvNB ip=install-7k27n namespace=customer-example phase=Complete
    time="2020-11-11T09:41:45Z" level=info msg=syncing event=update reconciling="*v1alpha1.Subscription" selflink=/apis/operators.coreos.com/v1alpha1/namespaces/example-argocd/subscriptions/argocd-operator
    time="2020-11-11T09:41:45Z" level=info msg=syncing id=0tbFL ip=install-nskss namespace=customer-example phase=Complete
    time="2020-11-11T09:41:45Z" level=info msg=syncing event=update reconciling="*v1alpha1.Subscription" selflink=/apis/operators.coreos.com/v1alpha1/namespaces/example-argocd/subscriptions/argocd-operator
    time="2020-11-11T09:41:45Z" level=info msg="state.Key.Namespace=openshift-marketplace state.Key.Name=certified-operators state.State=CONNECTING"
    panic: runtime error: index out of range [0] with length 0
    
    goroutine 632 [running]:
    github.com/operator-framework/operator-lifecycle-manager/pkg/controller/registry/reconciler.imageID(...)
    /build/pkg/controller/registry/reconciler/grpc.go:313
    github.com/operator-framework/operator-lifecycle-manager/pkg/controller/registry/reconciler.imageChanged(0xc002f34500, 0xc00a6eeeb8, 0x1, 0x1, 0x0)
    /build/pkg/controller/registry/reconciler/grpc.go:300 +0x32d
    github.com/operator-framework/operator-lifecycle-manager/pkg/controller/registry/reconciler.(*GrpcRegistryReconciler).ensureUpdatePod(0xc00aba8570, 0xc000237200, 0xc00a6eee00, 0x0)
    /build/pkg/controller/registry/reconciler/grpc.go:238 +0x3ca
    github.com/operator-framework/operator-lifecycle-manager/pkg/controller/registry/reconciler.(*GrpcRegistryReconciler).EnsureRegistryServer(0xc00aba8570, 0xc000237200, 0xc00b73b230, 0x1)
    /build/pkg/controller/registry/reconciler/grpc.go:158 +0xc7
    github.com/operator-framework/operator-lifecycle-manager/pkg/controller/operators/catalog.(*Operator).syncRegistryServer(0xc000639cc0, 0xc00b765880, 0xc00052db00, 0xc00052db00, 0xc0004edc01, 0x0, 0x0)
    /build/pkg/controller/operators/catalog/operator.go:625 +0x331
    github.com/operator-framework/operator-lifecycle-manager/pkg/controller/operators/catalog.(*Operator).syncCatalogSources.func1(0xc00052db00, 0xc00a1d8100, 0x4, 0x4, 0x1, 0x4800c5, 0x3)
    /build/pkg/controller/operators/catalog/operator.go:741 +0x79
    github.com/operator-framework/operator-lifecycle-manager/pkg/controller/operators/catalog.(*Operator).syncCatalogSources(0xc000639cc0, 0x1c96800, 0xc000363720, 0xc00b765730, 0x1d9fba8)
    /build/pkg/controller/operators/catalog/operator.go:788 +0x409
    github.com/operator-framework/operator-lifecycle-manager/pkg/lib/queueinformer.LegacySyncHandler.ToSyncerWithDelete.func1(0x1f4eb00, 0xc0007be740, 0x1f1dd40, 0xc00a1d80c0, 0xc00a1d80c0, 0x1b4aa00)
    /build/pkg/lib/queueinformer/queueinformer.go:183 +0x26e
    github.com/operator-framework/operator-lifecycle-manager/pkg/lib/kubestate.SyncFunc.Sync(0xc000466fa0, 0x1f4eb00, 0xc0007be740, 0x1f1dd40, 0xc00a1d80c0, 0xc00b765501, 0x0)
    /build/pkg/lib/kubestate/kubestate.go:184 +0x4e
    github.com/operator-framework/operator-lifecycle-manager/pkg/lib/queueinformer.(*QueueInformer).Sync(...)
    /build/pkg/lib/queueinformer/queueinformer.go:36
    github.com/operator-framework/operator-lifecycle-manager/pkg/lib/queueinformer.(*operator).processNextWorkItem(0xc000151ef0, 0x1f4eb00, 0xc0007be740, 0xc000cad3e0, 0x0)
    /build/pkg/lib/queueinformer/queueinformer_operator.go:287 +0x348
    github.com/operator-framework/operator-lifecycle-manager/pkg/lib/queueinformer.(*operator).worker(0xc000151ef0, 0x1f4eb00, 0xc0007be740, 0xc000cad3e0)
    /build/pkg/lib/queueinformer/queueinformer_operator.go:231 +0x49
    created by github.com/operator-framework/operator-lifecycle-manager/pkg/lib/queueinformer.(*operator).start
    /build/pkg/lib/queueinformer/queueinformer_operator.go:221 +0x45d
    

Environment

  • Red Hat OpenShift Container Platform (OCP) 4.6

Subscriber exclusive content

A Red Hat subscription provides unlimited access to our knowledgebase, tools, and much more.

Current Customers and Partners

Log in for full access

Log In

New to Red Hat?

Learn more about Red Hat subscriptions

Using a Red Hat product through a public cloud?

How to access this content