overcloud deploy fails with unable to push cib errors

Issue

note only 1 of the 3 controllers had the unable to push cib errors
- journalctl -u os-collect-config

nError: unable to push cib\nCall cib_apply_diff failed (-203): Update does not conform to the configured schema\n<cib crm_feature_set=\"3.0.14\" validate-with=\"pacemaker-2.10\" epoch=\"7\" num_updates=\"0\" admin_epoch=\"0\" cib-last-written=\"Fri Jul 13 18:13:04 2018\" update-origin=\"overcloud-controller-2\" update-client=\"cibadmin\" update-user=\"root\" have-quorum=\"1\" dc-uuid=\"2\">\n  <configuration>\n    <crm_config>
\n      <cluster_property_set id=\"cib-bootstrap-options\">\n        <nvpair id=\"cib-bootstrap-options-have-watchdog\" name=\"have-watchdog\" value=\"false\"
></nvpair>\n        <nvpair id=\"cib-bootstrap-options-dc-version\" name=\"dc-version\" value=\"1.1.18-11.el7_5.3-2b07d5c5a9\"></nvpair>\n        <nvpair id=\"cib-bootstrap-o
ptions-cluster-infrastructure\" name=\"cluster-infrastructure\" value=\"corosync\"></nvpair>\n        <nvpair id=\"cib-bootstrap-options-cluster-name\" name=\"cluster
-name\" value=\"tripleo_cluster\"></nvpair>\n        <nvpair id=\"cib-bootstrap-options-stonith-enabled\" name=\"stonith-enabled\" value=\"false\"></nvpair>\n        <nvpair
id=\"cib-bootstrap-options-stonith-enabled\" name=\"stonith-enabled\" value=\"false\"></nvpair>\n      </cluster_property_set>\n    </crm_config>\n    <nodes>\n
<node id=\"1\" uname=\"overcloud-controller-0\"></node>\n      <node id=\"2\" uname=\"overcloud-controller-1\"></node>\n      <node id=\"3\" uname=\"overcloud-controller-
2\"></node>\n    </nodes>\n    <resources></resources>\n    <constraints></constraints>\n  </configuration>\n  <status>\n    <node_state id=\"2\" uname=\"overcloud-controller-1\" in_ccm=\"true\" crmd=\"online\" crm-debug-origin=\"do_state_transition\" join=\"member\" expected=\"member\">\n      <lrm id=\"2\">\n        <lrm_resources></lrm_resources>\n      </lrm>\n    </node_state>\n    <node_state id=\"3\" uname=\"overcloud-controller-2\" in_ccm=\"true\" crmd=\"online\" crm-debug-origin=\"do_state_transition\" join=\"member\" expected=\"member\">\n      <lrm id=\"3\">\n        <lrm_resources></lrm_resources>\n      </lrm>\n    </node_state>\n    <node_state id=\"1\" uname=\"overcloud-controller-0\" in_ccm=\"true\" crmd=\"online\" crm-debug-origin=\"do_state_transition\" join=\"member\" expected=\"member\">\n      <lrm id=\"1\">\n        <lrm_resources></lrm_resources>\n      </lrm>\n    </node_state>\n  </status>\n</cib>\n\n", "deploy_status_code": 0}

• heat stack-list --show-nested|grep -iv compl

+--------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------+-----------------+----------------------+----------------------+--------------------------------------+
| id                                   | stack_name                                                                                                                            | stack_status    | creation_time        | updated_time         | parent                               |
+--------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------+-----------------+----------------------+----------------------+--------------------------------------+
| c7213b12-efc9-4783-8b14-00a2dc689b90 | overcloud                                                                                                                             | UPDATE_FAILED   | 2018-07-13T19:37:09Z | 2018-07-13T19:41:09Z | None                                 |
| 48115d3b-9947-4f48-92c9-f5a5a7fb61c7 | overcloud-AllNodesDeploySteps-3os4k5m2cdtq                                                                                            | CREATE_FAILED   | 2018-07-13T20:01:20Z | None                 | c7213b12-efc9-4783-8b14-00a2dc689b90 |
| ecfdb082-8177-47ae-b3d4-77f3e7903e3a | overcloud-AllNodesDeploySteps-3os4k5m2cdtq-ControllerDeployment_Step2-fve2trl2ru2e                                                    | CREATE_FAILED   | 2018-07-13T20:04:45Z | None                 | 48115d3b-9947-4f48-92c9-f5a5a7fb61c7 |
+--------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------+-----------------+----------------------+----------------------+--------------------------------------+
[stack@osp10-director- ~]$ heat resource-list ecfdb082-8177-47ae-b3d4-77f3e7903e3a
WARNING (shell) "heat resource-list" is deprecated, please use "openstack stack resource list" instead
+---------------+--------------------------------------+--------------------------------+-----------------+----------------------+
| resource_name | physical_resource_id                 | resource_type                  | resource_status | updated_time         |
+---------------+--------------------------------------+--------------------------------+-----------------+----------------------+
| 0             | ecaa85af-6c0f-49fa-8bcf-13207044934f | OS::Heat::StructuredDeployment | CREATE_FAILED   | 2018-07-13T20:04:45Z |
| 1             | db28fbc6-6c3e-46ba-a264-af8d0a17a3f5 | OS::Heat::StructuredDeployment | CREATE_FAILED   | 2018-07-13T20:04:45Z |
| 2             | d2d5b52f-0bab-4a42-9280-2d73e9f1b84a | OS::Heat::StructuredDeployment | CREATE_FAILED   | 2018-07-13T20:04:45Z |
+---------------+--------------------------------------+--------------------------------+-----------------+----------------------+
[stack@osp10-director- ~]$


heat deployment-show d2d5b52f-0bab-4a42-9280-2d73e9f1b84a 
   "deploy_stderr": "exception: connect failed\n\u001b[1;31mWarning: Scope(Class[Mongodb::Server]): Replset specified, but no replset_members or replset_config provided.\u001b[0m\n\u001b[1;31mWarning: Scope(Haproxy::Config[haproxy]): haproxy: The $merge_options parameter will default to true in the next major release. Please review the documentation regarding the implications.\u001b[0m\n\u001b[1;31mError: /Stage[main]/Tripleo::Profile::Base::Rabbitmq/Rabbitmq_user[guest]: Could not evaluate: Command is still failing after 180 seconds expired!\u001b[0m\n",

• corosync.log

IPaddr2(ip-10.12.143.201)[52045]:    ERROR: Unable to find nic or netmask.