overcloud deploy fails with unable to push cib errors
Issue
note only 1 of the 3 controllers had the unable to push cib errors
- journalctl -u os-collect-config
nError: unable to push cib\nCall cib_apply_diff failed (-203): Update does not conform to the configured schema\n<cib crm_feature_set=\"3.0.14\" validate-with=\"pacemaker-2.10\" epoch=\"7\" num_updates=\"0\" admin_epoch=\"0\" cib-last-written=\"Fri Jul 13 18:13:04 2018\" update-origin=\"overcloud-controller-2\" update-client=\"cibadmin\" update-user=\"root\" have-quorum=\"1\" dc-uuid=\"2\">\n <configuration>\n <crm_config>
\n <cluster_property_set id=\"cib-bootstrap-options\">\n <nvpair id=\"cib-bootstrap-options-have-watchdog\" name=\"have-watchdog\" value=\"false\"
></nvpair>\n <nvpair id=\"cib-bootstrap-options-dc-version\" name=\"dc-version\" value=\"1.1.18-11.el7_5.3-2b07d5c5a9\"></nvpair>\n <nvpair id=\"cib-bootstrap-o
ptions-cluster-infrastructure\" name=\"cluster-infrastructure\" value=\"corosync\"></nvpair>\n <nvpair id=\"cib-bootstrap-options-cluster-name\" name=\"cluster
-name\" value=\"tripleo_cluster\"></nvpair>\n <nvpair id=\"cib-bootstrap-options-stonith-enabled\" name=\"stonith-enabled\" value=\"false\"></nvpair>\n <nvpair
id=\"cib-bootstrap-options-stonith-enabled\" name=\"stonith-enabled\" value=\"false\"></nvpair>\n </cluster_property_set>\n </crm_config>\n <nodes>\n
<node id=\"1\" uname=\"overcloud-controller-0\"></node>\n <node id=\"2\" uname=\"overcloud-controller-1\"></node>\n <node id=\"3\" uname=\"overcloud-controller-
2\"></node>\n </nodes>\n <resources></resources>\n <constraints></constraints>\n </configuration>\n <status>\n <node_state id=\"2\" uname=\"overcloud-controller-1\" in_ccm=\"true\" crmd=\"online\" crm-debug-origin=\"do_state_transition\" join=\"member\" expected=\"member\">\n <lrm id=\"2\">\n <lrm_resources></lrm_resources>\n </lrm>\n </node_state>\n <node_state id=\"3\" uname=\"overcloud-controller-2\" in_ccm=\"true\" crmd=\"online\" crm-debug-origin=\"do_state_transition\" join=\"member\" expected=\"member\">\n <lrm id=\"3\">\n <lrm_resources></lrm_resources>\n </lrm>\n </node_state>\n <node_state id=\"1\" uname=\"overcloud-controller-0\" in_ccm=\"true\" crmd=\"online\" crm-debug-origin=\"do_state_transition\" join=\"member\" expected=\"member\">\n <lrm id=\"1\">\n <lrm_resources></lrm_resources>\n </lrm>\n </node_state>\n </status>\n</cib>\n\n", "deploy_status_code": 0}
- heat stack-list --show-nested|grep -iv compl
+--------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------+-----------------+----------------------+----------------------+--------------------------------------+
| id | stack_name | stack_status | creation_time | updated_time | parent |
+--------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------+-----------------+----------------------+----------------------+--------------------------------------+
| c7213b12-efc9-4783-8b14-00a2dc689b90 | overcloud | UPDATE_FAILED | 2018-07-13T19:37:09Z | 2018-07-13T19:41:09Z | None |
| 48115d3b-9947-4f48-92c9-f5a5a7fb61c7 | overcloud-AllNodesDeploySteps-3os4k5m2cdtq | CREATE_FAILED | 2018-07-13T20:01:20Z | None | c7213b12-efc9-4783-8b14-00a2dc689b90 |
| ecfdb082-8177-47ae-b3d4-77f3e7903e3a | overcloud-AllNodesDeploySteps-3os4k5m2cdtq-ControllerDeployment_Step2-fve2trl2ru2e | CREATE_FAILED | 2018-07-13T20:04:45Z | None | 48115d3b-9947-4f48-92c9-f5a5a7fb61c7 |
+--------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------+-----------------+----------------------+----------------------+--------------------------------------+
[stack@osp10-director- ~]$ heat resource-list ecfdb082-8177-47ae-b3d4-77f3e7903e3a
WARNING (shell) "heat resource-list" is deprecated, please use "openstack stack resource list" instead
+---------------+--------------------------------------+--------------------------------+-----------------+----------------------+
| resource_name | physical_resource_id | resource_type | resource_status | updated_time |
+---------------+--------------------------------------+--------------------------------+-----------------+----------------------+
| 0 | ecaa85af-6c0f-49fa-8bcf-13207044934f | OS::Heat::StructuredDeployment | CREATE_FAILED | 2018-07-13T20:04:45Z |
| 1 | db28fbc6-6c3e-46ba-a264-af8d0a17a3f5 | OS::Heat::StructuredDeployment | CREATE_FAILED | 2018-07-13T20:04:45Z |
| 2 | d2d5b52f-0bab-4a42-9280-2d73e9f1b84a | OS::Heat::StructuredDeployment | CREATE_FAILED | 2018-07-13T20:04:45Z |
+---------------+--------------------------------------+--------------------------------+-----------------+----------------------+
[stack@osp10-director- ~]$
heat deployment-show d2d5b52f-0bab-4a42-9280-2d73e9f1b84a
"deploy_stderr": "exception: connect failed\n\u001b[1;31mWarning: Scope(Class[Mongodb::Server]): Replset specified, but no replset_members or replset_config provided.\u001b[0m\n\u001b[1;31mWarning: Scope(Haproxy::Config[haproxy]): haproxy: The $merge_options parameter will default to true in the next major release. Please review the documentation regarding the implications.\u001b[0m\n\u001b[1;31mError: /Stage[main]/Tripleo::Profile::Base::Rabbitmq/Rabbitmq_user[guest]: Could not evaluate: Command is still failing after 180 seconds expired!\u001b[0m\n",
- corosync.log
IPaddr2(ip-10.12.143.201)[52045]: ERROR: Unable to find nic or netmask.
Environment
- Red Hat OpenStack Platform 10.0
Subscriber exclusive content
A Red Hat subscription provides unlimited access to our knowledgebase of over 48,000 articles and solutions.
Welcome! Check out the Getting Started with Red Hat page for quick tours and guides for common tasks.
