Issue

• Upgrade from osp9 to osp10 fails
• OSP9 upgrade fails
• OSP9 undercloud upgrade fails
• Undercloud upgrade from OSP9 to OSP10 fails with following errors :

2016-12-20 14:47:19 - ESC[mNotice: /Stage[main]/Nova::Db::Sync/Exec[nova-db-sync]/returns: WARNING: cell0 mapping not found - not syncing cell0.ESC[0m
2016-12-20 14:47:19 - ESC[mNotice: /Stage[main]/Nova::Db::Sync/Exec[nova-db-sync]/returns: error: There are still 32 unmigrated records in the compute_nodes table. Migration cannot continue until all records have been migrated.ESC[0m
2016-12-20 14:47:19 - ESC[1;31mError: /usr/bin/nova-manage  db sync returned 1 instead of one of [0]ESC[0m
2016-12-20 14:47:19 - ESC[1;31mError: /Stage[main]/Nova::Db::Sync/Exec[nova-db-sync]/returns: change from notrun to 0 failed: /usr/bin/nova-manage  db sync returned 1 instead of one of [0]ESC[0m
2016-12-20 14:47:19 - ESC[mNotice: /Stage[main]/Nova::Deps/Anchor[nova::dbsync::end]: Dependency Exec[nova-db-sync] has failures: true
ESC[0m
2016-12-20 14:47:19 - ESC[1;31mWarning: /Stage[main]/Nova::Deps/Anchor[nova::dbsync::end]: Skipping because of failed dependenciesESC[0m
2016-12-20 14:47:19 - ESC[mNotice: /Stage[main]/Nova::Cron::Archive_deleted_rows/Cron[nova-manage db archive_deleted_rows]: Dependency Exec[nova-db-sync] has failures: trueESC[0m
2016-12-20 14:47:19 - ESC[1;31mWarning: /Stage[main]/Nova::Cron::Archive_deleted_rows/Cron[nova-manage db archive_deleted_rows]: Skipping because of failed dependenciesESC[0m
2016-12-20 14:47:19 - ESC[mNotice: /Stage[main]/Nova::Deps/Anchor[nova::dbsync_api::begin]: Dependency Exec[nova-db-sync] has failures: trueESC[0m
2016-12-20 14:47:19 - ESC[1;31mWarning: /Stage[main]/Nova::Deps/Anchor[nova::dbsync_api::begin]: Skipping because of failed dependenciesESC[0m
2016-12-20 14:47:19 - ESC[mNotice: /Stage[main]/Nova::Db::Sync_api/Exec[nova-db-sync-api]: Dependency Exec[nova-db-sync] has failures: trueESC[0m
2016-12-20 14:47:19 - ESC[1;31mWarning: /Stage[main]/Nova::Db::Sync_api/Exec[nova-db-sync-api]: Skipping because of failed dependencies
ESC[0m
2016-12-20 14:47:19 - ESC[mNotice: /Stage[main]/Nova::Deps/Anchor[nova::dbsync_api::end]: Dependency Exec[nova-db-sync] has failures: trueESC[0m
2016-12-20 14:47:19 - ESC[1;31mWarning: /Stage[main]/Nova::Deps/Anchor[nova::dbsync_api::end]: Skipping because of failed dependencies
ESC[0m
2016-12-20 14:47:19 - ESC[mNotice: /Stage[main]/Nova::Deps/Anchor[nova::service::begin]: Dependency Exec[nova-db-sync] has failures: trueESC[0m
2016-12-20 14:47:19 - ESC[1;31mWarning: /Stage[main]/Nova::Deps/Anchor[nova::service::begin]: Skipping because of failed dependenciesESC[0m
2016-12-20 14:47:19 - ESC[mNotice: /Stage[main]/Nova::Cert/Nova::Generic_service[cert]/Service[nova-cert]: Dependency Exec[nova-db-sync] has failures: trueESC[0m
2016-12-20 14:47:19 - ESC[1;31mWarning: /Stage[main]/Nova::Cert/Nova::Generic_service[cert]/Service[nova-cert]: Skipping because of failed dependenciesESC[0m
2016-12-20 14:47:19 - ESC[mNotice: /Stage[main]/Nova::Scheduler/Nova::Generic_service[scheduler]/Service[nova-scheduler]: Dependency Exec[nova-db-sync] has failures: trueESC[0m
2016-12-20 14:47:19 - ESC[1;31mWarning: /Stage[main]/Nova::Scheduler/Nova::Generic_service[scheduler]/Service[nova-scheduler]: Skipping because of failed dependenciesESC[0m

=========== messages truncated ===============

2016-12-20 14:51:39 - ESC[mNotice: Finished catalog run in 359.76 secondsESC[0m
+ rc=6
+ set -e
+ echo 'puppet apply exited with exit code 6'
puppet apply exited with exit code 6
+ '[' 6 '!=' 2 -a 6 '!=' 0 ']'
+ exit 6
[2016-12-20 14:51:44,303] (os-refresh-config) [ERROR] during configure phase. [Command '['dib-run-parts', '/usr/libexec/os-refresh-config/configure.d']' returned non-zero exit status 1]

[2016-12-20 14:51:44,303] (os-refresh-config) [ERROR] Aborting...
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/usr/lib/python2.7/site-packages/instack_undercloud/undercloud.py", line 1206, in install
    _run_orc(instack_env)
  File "/usr/lib/python2.7/site-packages/instack_undercloud/undercloud.py", line 996, in _run_orc
    _run_live_command(args, instack_env, 'os-refresh-config')
  File "/usr/lib/python2.7/site-packages/instack_undercloud/undercloud.py", line 519, in _run_live_command
    raise RuntimeError('%s failed. See log for details.' % name)
RuntimeError: os-refresh-config failed. See log for details.
Command 'instack-install-undercloud' returned non-zero exit status 1
(END)

• Running dib-run-parts manually showing following error :

$ dib-run-parts /usr/libexec/os-refresh-config/configure.d
dib-run-parts Tue Dec 20 16:03:35 CET 2016 Running /usr/libexec/os-refresh-config/configure.d/20-os-apply-config
[2016/12/20 04:03:35 PM] [WARNING] DEPRECATED: falling back to /var/run/os-collect-config/os_config_files.json
[2016/12/20 04:03:35 PM] [ERROR] [Errno 13] Permission denied: '/usr/libexec/os-apply-config/templates/etc/os-collect-config.conf'
[2016/12/20 04:03:35 PM] [ERROR] ('could not render moustache template /usr/libexec/os-apply-config/templates/etc/os-collect-config.conf', '/usr/libexec/os-apply-config/templates/etc/os-collect-config.conf')