RHODS Dashboard fails with warning FailedToRetrieveImagePullSecret post upgrade from RHOCP 4.14 to 4.16

Solution Verified - Updated -

Environment

  • Red Hat OpenShift AI (RHOAI) 2
  • Red Hat OpenShift Container Platform (RHOCP) 4

Issue

  • The rhods-dashboard pod is unable to retrieve some image pull secrets addon-managed-odh-pullsecret with a FailedToRetrieveImagePullSecret warning.

Resolution

  • Download Pull Secret from Red Hat Console and create a secret manually using below command:
$ oc create secret generic addon-managed-odh-pullsecret -n redhat-ods-applications --type=kubernetes.io/dockerconfigjson --from-file=.dockerconfigjson=</path/to/downloaded/pull-secret.txt>
  • To create a pod service account or a namespace, wait until the service account is provisioned with a docker pull secret.

Root Cause

  • The addon-managed-odh-pullsecret secret is not present in redhat-ods-applications namespace.
  • This warning for search-pull-secret occurs when creating a pod before its service account is fully provisioned, the pod fails to access the OpenShift image registry.

Diagnostic Steps

  • Check if the below warning are present in the for rhods-dashboard pods in redhat-ods-applications namespace:
$ oc get events
1m57s       Warning   FailedToRetrieveImagePullSecret   pod/rhods-dashboard-abc   Unable to retrieve some image pull secrets (addon-managed-odh-pullsecret); attempting to pull the image may not succeed.
2m53s       Warning   FailedToRetrieveImagePullSecret   pod/rhods-dashboard-xyz   Unable to retrieve some image pull secrets (addon-managed-odh-pullsecret); attempting to pull the image may not succeed.
  • Check if addon-managed-odh-pullsecret secret present:
$ oc get secret addon-managed-odh-pullsecret -o yaml
Error from server (NotFound): secrets "addon-managed-odh-pullsecret" not found

This solution is part of Red Hat’s fast-track publication program, providing a huge library of solutions that Red Hat engineers have created while supporting our customers. To give you the knowledge you need the instant it becomes available, these articles may be presented in a raw and unedited form.

Comments