Overcloud update fails during puppet host config step 1
Issue
During an openstack overcloud update, the update fails with errors similar to the following:
2020-12-09 02:40:00Z [overcloud]: UPDATE_FAILED Resource UPDATE failed: Error: resources.AllNodesDeploySteps.resources.ControllerDeployment_Step1.resources[0]: Deployment to server failed: deploy_status_code: Deployment exited with non-zero status code: 2
Stack overcloud UPDATE_FAILED
overcloud.AllNodesDeploySteps.ControllerDeployment_Step1.0:
resource_type: OS::Heat::StructuredDeployment
physical_resource_id: 9da545d9-28dc-4fc4-8b76-2d358a60abcd
status: UPDATE_FAILED
status_reason: |
Error: resources[0]: Deployment to server failed: deploy_status_code : Deployment exited with non-zero status code: 2
deploy_stdout: |
...
"Warning: /Stage[main]/Tripleo::Firewall/Exec[nonpersistent_v6_rules_cleanup]: Skipping because of failed dependencies",
"Warning: /Stage[main]/Tripleo::Firewall/Exec[nonpersistent_ironic_inspector_pxe_filter_v4_rules_cleanup]: Skipping because of failed dependencies",
"Warning: /Stage[main]/Tripleo::Firewall/Exec[nonpersistent_ironic_inspector_pxe_filter_v6_rules_cleanup]: Skipping because of failed dependencies"
]
}
to retry, use: --limit @/var/lib/heat-config/heat-config-ansible/3ad89b15-6425-415b-bd90-45d9f627abcd_playbook.retry
PLAY RECAP *********************************************************************
localhost : ok=27 changed=10 unreachable=0 failed=1
(truncated, view all with --long)
deploy_stderr: |
Heat Stack update failed.
Heat Stack update failed.
And also
TASK [Debug output for task: Run puppet host configuration for step 1] *********
fatal: [localhost]: FAILED! => {
"failed_when_result": true,
"outputs.stdout_lines|default([])|union(outputs.stderr_lines|default([]))": [
"Notice: hiera(): Cannot load backend module_data: cannot load such file -- hiera/backend/module_data_backend",
"Notice: Scope(Class[Tripleo::Firewall::Post]): At this stage, all network traffic is blocked.",
"Notice: Compiled catalog for ospcontrol-ctc-a107d.ctc.pa.tesprod.comcast.net in environment production in 3.18 seconds",
"Notice: /Stage[main]/Tripleo::Profile::Base::Kernel/Kmod::Load[nf_conntrack_proto_sctp]/Exec[modprobe nf_conntrack_proto_sctp]/returns: executed successfully",
"Notice: /Stage[main]/Tripleo::Profile::Base::Docker/Service[docker]: Dependency Augeas[docker-daemon.json-debug] has failures: true",
"Notice: /Stage[main]/Pacemaker::Stonith/Pacemaker::Property[Disable STONITH]/Pcmk_property[property--stonith-enabled]/ensure: created",
"Notice: /Stage[main]/Ssh::Server::Config/Concat[/etc/ssh/sshd_config]/File[/etc/ssh/sshd_config]/content: content changed '{md5}f8044ed53107372d76ab3978d355de0f' to '{md5}3534841fdb8db5b58d66600a60bf3759'",
"Notice: /Stage[main]/Ssh::Server::Service/Service[sshd]: Triggered 'refresh' from 2 events",
"Notice: /Stage[main]/Tripleo::Firewall::Post/Tripleo::Firewall::Rule[998 log all]/Firewall[998 log all ipv4]: Dependency Augeas[docker-daemon.json-debug] has failures: true",
"Notice: /Stage[main]/Tripleo::Firewall::Post/Tripleo::Firewall::Rule[998 log all]/Firewall[998 log all ipv6]: Dependency Augeas[docker-daemon.json-debug] has failures: true",
"Notice: /Stage[main]/Tripleo::Firewall::Post/Tripleo::Firewall::Rule[999 drop all]/Firewall[999 drop all ipv4]: Dependency Augeas[docker-daemon.json-debug] has failures: true",
"Notice: /Stage[main]/Tripleo::Firewall::Post/Tripleo::Firewall::Rule[999 drop all]/Firewall[999 drop all ipv6]: Dependency Augeas[docker-daemon.json-debug] has failures: true",
"Notice: /Stage[main]/Firewall::Linux::Redhat/File[/etc/sysconfig/iptables]: Dependency Augeas[docker-daemon.json-debug] has failures: true",
"Notice: /Stage[main]/Firewall::Linux::Redhat/File[/etc/sysconfig/ip6tables]: Dependency Augeas[docker-daemon.json-debug] has failures: true",
"Notice: /Stage[main]/Tripleo::Firewall/Exec[nonpersistent_v4_rules_cleanup]: Dependency Augeas[docker-daemon.json-debug] has failures: true",
"Notice: /Stage[main]/Tripleo::Firewall/Exec[nonpersistent_v6_rules_cleanup]: Dependency Augeas[docker-daemon.json-debug] has failures: true",
"Notice: /Stage[main]/Tripleo::Firewall/Exec[nonpersistent_ironic_inspector_pxe_filter_v4_rules_cleanup]: Dependency Augeas[docker-daemon.json-debug] has failures: true",
"Notice: /Stage[main]/Tripleo::Firewall/Exec[nonpersistent_ironic_inspector_pxe_filter_v6_rules_cleanup]: Dependency Augeas[docker-daemon.json-debug] has failures: true",
Environment
- Red Hat OpenStack Platform 13
Subscriber exclusive content
A Red Hat subscription provides unlimited access to our knowledgebase, tools, and much more.