Machine Config Pools degraded with "unexpected on-disk state" on OpenShift 4

Issue

• After performing an update to a newer version of OpenShift Container Platform, not all nodes are upgraded. For example:

  worker-2.ocp.example.net  Ready, SchedulingDisabled   worker  34d  v1.17.1+912792b         <----------
  

• After performing an update to a newer version of OpenShift Container Platform, the MachineConfigOperator is reporting degraded pools:

  'Failed to resync $VERSION because: error during syncRequiredMachineConfigPools:
        [timed out waiting for the condition, error pool $POOL is not ready, retrying.
        Status: (pool degraded: true total: x, ready y, updated: y, unavailable: 1)]'
  

• A machine config pool is degraded, and in the MachineConfigOperator clusteroperator extensions, we see an error similar to:

  worker: 'pool is degraded because nodes fail with "1 nodes are reporting degraded
        status on sync": "Node worker0 is reporting: \"unexpected
        on-disk state validating against rendered-worker-xxxx:
        expected target osImageURL \\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:xxx\\\",
        have \\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:yyy\\\"
        (\\\"zzz\\\")\""'
  

• The machine-config-daemon pod logs show messages like:

  Marking Degraded due to: unexpected on-disk state validating against rendered-master- 
  xxxxxx: content mismatch for file "/usr/local/bin/configure-ovs.sh"
  

  Marking Degraded due to: "unexpected on-disk state validating against rendered-master-xxxxxx: expected target osImageURL \"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:xxxxxxx\", have \"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:yyyyyyy\""