ECR Image pull fails in-spite of attaching AmazonEC2ContainerRegistryReadOnly policy to the worker nodes in OpenShift Container Platform 4.

Issue

Attaching AmazonEC2ContainerRegistryReadOnly policy to the worker nodes, has no effect on ECR Image pull. The user gets an authentication error. Attaching the policy ideally should avoid the need to provide an image-pull-secret. However, the error is resolved only if the user also provides an image-pull-secret.

AWSECRLegacyCredProvider: AWS clusters that use AmazonEC2ContainerRegistryReadOnly node policies to access ECR are unable to pull images from ECR after updating to exposed 4.14.z. Customers that have a container pulling from ECR, during an upgrade to 4.14 could be affected.

• With PDB & ImagePullPolicy is Always:

  • The first pod in the set will be moved, enter ImagePullBackOff and the cluster upgrade will halt. Worker MachineConfigPool will never reach an upgraded state.

• With PDB & ImagePullPolicy is IfNotPresent:

  • Undefined behavior, the image may or may not be present on workers so users may be lucky, or they may end up in the same situation as the Always ImagePullPolicy case.

• Without PDB & ImagePullPolicy is Always:

  • All pods that require pulling from an ECR private registry, will enter ImagePullBackOff during the upgrade.

• Without PDB & ImagePullPolicy is IfNotPresent:

  • As above, users may end up in ImagePullBackOff issues.