machine config operator reports pending config post node upgrade

Issue

• The MCO is stuck on all our machine config pools after upgrading the cluster to 4.5.x, it appears the nodes deployed the new configs related to 4.5.x but did not update the current config names, so now its stuck trying to get to the 4.5.x desired rendered config.

• The machine config daemon on the respective node reports the following message even post the upgrade completed.

I0828 16:45:09.278661    2332 daemon.go:771] Current config: rendered-worker-3b9d0c44e1f3c81e505b86d2ce5c4428
I0828 16:45:09.278678    2332 daemon.go:772] Desired config: rendered-worker-268565e91e2a80762715929e58bc11e6
I0828 16:45:09.278682    2332 daemon.go:786] Pending config: rendered-worker-3b9d0c44e1f3c81e505b86d2ce5c4428
I0828 16:45:09.285337    2332 update.go:1404] Disk currentConfig rendered-worker-268565e91e2a80762715929e58bc11e6 overrides node annotation rendered-kafka-3b9d0c44e1f3c81e505b86d2ce5c4428
I0828 16:45:09.287841    2332 daemon.go:1014] Validating against pending config rendered-worker-3b9d0c44e1f3c81e505b86d2ce5c4428
E0828 16:45:09.287895    2332 daemon.go:1244] expected target osImageURL quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:511ea3dbbb6e6059a52251f6291305f2a9ba5f98192cfb92989fda88960fcf82
E0828 16:45:09.287904    2332 writer.go:135] Marking Degraded due to: unexpected on-disk state validating against rendered-worker-3b9d0c44e1f3c81e505b86d2ce5c4428

• The upgrade of the node looks fine but somehow the machine-config-daemon still takes the pending config in consideration marking the machine-config cluster operator degraded.