- Posted In
- Red Hat OpenStack Platform
- Tags
- Ceph
Error: /Stage[main]/Ceph::Osds/Ceph::Osd[/srv/data]/Exec[ceph-osd-activate-/srv/data]/returns: change from notrun to 0 failed: Command exceeded timeout (truncated, view all with --long)
I was trying to deploy the RHOSP 11 with Ceph storage,
I have successfull heat templates which was successfully deploying cloud with ceph integration ( Hyper-Converged ) still last week. Deployment step got failed in starting of mistral and found Redhat have updated the code, found it through the following link and updated the code.
https://access.redhat.com/solutions/3459241
Followed by, I get error in step 3.1 Ceph Disk activation .
I found the exact error through the command.
"openstack stack failures list overcloud"
overcloud.AllNodesDeploySteps.ComputeDeployment_Step3.1:
resource_type: OS::Heat::StructuredDeployment
physical_resource_id: 12bfa168-8779-431e-a807-3ab4619ea169
status: CREATE_FAILED
status_reason: |
Error: resources[1]: Deployment to server failed: deploy_status_code : Deployment exited with non-zero status code: 6
deploy_stdout: |
Error: Command exceeded timeout
Error: /Stage[main]/Ceph::Osds/Ceph::Osd[/srv/data]/Exec[ceph-osd-activate-/srv/data]/returns: change from notrun to 0 failed: Command exceeded timeout
(truncated, view all with --long)
Is there anything I have to change in the new code or any configuration mismatches.
Any help would be greatly appreciated.
Thanks in advance!!
Responses
Might need to include --long to see what caused the error:
$ openstack stack failures list overcloud --long
Please find the error log, I was not able to see anything that indicated the error or problem directly.
[stack@dir]$ openstack stack failures list overcloud --long
overcloud.AllNodesDeploySteps.ComputeDeployment_Step3.1:
resource_type: OS::Heat::StructuredDeployment
physical_resource_id: 844fa8a7-6550-459e-bdaa-05a5b6a25745
status: CREATE_FAILED
status_reason: |
Error: resources[1]: Deployment to server failed: deploy_status_code : Deployment exited with non-zero status code: 6
deploy_stdout: |
Notice: hiera(): Cannot load backend module_data: cannot load such file -- hiera/backend/module_data_backend
Notice: Scope(Class[Tripleo::Firewall::Post]): At this stage, all network traffic is blocked.
Notice: Compiled catalog for overcloud-compute-1.localdomain in environment production in 3.98 seconds
Notice: /Stage[main]/Main/Package_manifest[/var/lib/tripleo/installed-packages/overcloud_compute3]/ensure: created
Notice: /Stage[main]/Neutron/Neutron_config[DEFAULT/auth_strategy]/ensure: created
Notice: /Stage[main]/Neutron/Neutron_config[DEFAULT/core_plugin]/ensure: created
Notice: /Stage[main]/Neutron/Neutron_config[DEFAULT/host]/ensure: created
Notice: /Stage[main]/Neutron/Neutron_config[DEFAULT/dhcp_agents_per_network]/ensure: created
Notice: /Stage[main]/Neutron/Neutron_config[DEFAULT/allow_overlapping_ips]/ensure: created
Notice: /Stage[main]/Neutron/Neutron_config[DEFAULT/global_physnet_mtu]/ensure: created
Notice: /Stage[main]/Neutron/Neutron_config[agent/root_helper]/ensure: created
Notice: /Stage[main]/Neutron/Neutron_config[DEFAULT/service_plugins]/ensure: created
Notice: /Stage[main]/Ceilometer/Ceilometer_config[DEFAULT/http_timeout]/ensure: created
Notice: /Stage[main]/Ceilometer/Ceilometer_config[DEFAULT/host]/ensure: created
Notice: /Stage[main]/Ceilometer/Ceilometer_config[publisher/telemetry_secret]/ensure: created
Notice: /Stage[main]/Ceilometer/Ceilometer_config[database/event_time_to_live]/ensure: created
Notice: /Stage[main]/Ceilometer/Ceilometer_config[database/metering_time_to_live]/ensure: created
Notice: /Stage[main]/Tripleo::Profile::Base::Kernel/Kmod::Load[nf_conntrack_proto_sctp]/Exec[modprobe nf_conntrack_proto_sctp]/returns: executed successfully
Notice: /Stage[main]/Neutron::Logging/Oslo::Log[neutron_config]/Neutron_config[DEFAULT/log_dir]/ensure: created
Notice: /Stage[main]/Neutron/Oslo::Messaging::Default[neutron_config]/Neutron_config[DEFAULT/control_exchange]/ensure: created
Notice: /Stage[main]/Neutron/Oslo::Concurrency[neutron_config]/Neutron_config[oslo_concurrency/lock_path]/ensure: created
Notice: /Stage[main]/Neutron/Oslo::Messaging::Rabbit[neutron_config]/Neutron_config[oslo_messaging_rabbit/heartbeat_timeout_threshold]/ensure: created
Notice: /Stage[main]/Neutron/Oslo::Messaging::Rabbit[neutron_config]/Neutron_config[oslo_messaging_rabbit/rabbit_password]/ensure: created
Notice: /Stage[main]/Neutron/Oslo::Messaging::Rabbit[neutron_config]/Neutron_config[oslo_messaging_rabbit/rabbit_use_ssl]/ensure: created
Notice: /Stage[main]/Neutron/Oslo::Messaging::Rabbit[neutron_config]/Neutron_config[oslo_messaging_rabbit/rabbit_userid]/ensure: created
Notice: /Stage[main]/Neutron/Oslo::Messaging::Rabbit[neutron_config]/Neutron_config[oslo_messaging_rabbit/rabbit_hosts]/ensure: created
Notice: /Stage[main]/Neutron/Oslo::Messaging::Rabbit[neutron_config]/Neutron_config[oslo_messaging_rabbit/rabbit_ha_queues]/ensure: created
Notice: /Stage[main]/Neutron/Oslo::Messaging::Rabbit[neutron_config]/Neutron_config[DEFAULT/rpc_backend]/ensure: created
Notice: /Stage[main]/Neutron::Deps/Anchor[neutron::config::end]: Triggered 'refresh' from 18 events
Notice: /Stage[main]/Neutron::Deps/Anchor[neutron::service::begin]: Triggered 'refresh' from 1 events
Notice: /Stage[main]/Ceilometer::Logging/Oslo::Log[ceilometer_config]/Ceilometer_config[DEFAULT/log_dir]/ensure: created
Notice: /Stage[main]/Ceilometer/Oslo::Messaging::Rabbit[ceilometer_config]/Ceilometer_config[oslo_messaging_rabbit/heartbeat_timeout_threshold]/ensure: created
Notice: /Stage[main]/Ceilometer/Oslo::Messaging::Rabbit[ceilometer_config]/Ceilometer_config[oslo_messaging_rabbit/rabbit_password]/ensure: created
Notice: /Stage[main]/Ceilometer/Oslo::Messaging::Rabbit[ceilometer_config]/Ceilometer_config[oslo_messaging_rabbit/rabbit_use_ssl]/ensure: created
Notice: /Stage[main]/Ceilometer/Oslo::Messaging::Rabbit[ceilometer_config]/Ceilometer_config[oslo_messaging_rabbit/rabbit_userid]/ensure: created
Notice: /Stage[main]/Ceilometer/Oslo::Messaging::Rabbit[ceilometer_config]/Ceilometer_config[oslo_messaging_rabbit/rabbit_hosts]/ensure: created
Notice: /Stage[main]/Ceilometer/Oslo::Messaging::Rabbit[ceilometer_config]/Ceilometer_config[oslo_messaging_rabbit/rabbit_ha_queues]/ensure: created
Notice: /Stage[main]/Ceilometer/Oslo::Messaging::Rabbit[ceilometer_config]/Ceilometer_config[DEFAULT/rpc_backend]/ensure: created
Notice: /Stage[main]/Ceilometer/Oslo::Messaging::Notifications[ceilometer_config]/Ceilometer_config[oslo_messaging_notifications/topics]/ensure: created
Notice: /Stage[main]/Ceilometer::Deps/Anchor[ceilometer::config::end]: Triggered 'refresh' from 14 events
Notice: /Stage[main]/Ceilometer::Deps/Anchor[ceilometer::service::begin]: Triggered 'refresh' from 1 events
Notice: /Stage[main]/Ceph::Osds/Ceph::Osd[/srv/data]/Exec[ceph-osd-prepare-/srv/data]/returns: ++ readlink -f /srv/data
Notice: /Stage[main]/Ceph::Osds/Ceph::Osd[/srv/data]/Exec[ceph-osd-prepare-/srv/data]/returns: + disk=/srv/data
Notice: /Stage[main]/Ceph::Osds/Ceph::Osd[/srv/data]/Exec[ceph-osd-prepare-/srv/data]/returns: + test -b /srv/data
Notice: /Stage[main]/Ceph::Osds/Ceph::Osd[/srv/data]/Exec[ceph-osd-prepare-/srv/data]/returns: + echo /srv/data
Notice: /Stage[main]/Ceph::Osds/Ceph::Osd[/srv/data]/Exec[ceph-osd-prepare-/srv/data]/returns: + egrep -e '^/dev' -q -v
Notice: /Stage[main]/Ceph::Osds/Ceph::Osd[/srv/data]/Exec[ceph-osd-prepare-/srv/data]/returns: + mkdir -p /srv/data
Notice: /Stage[main]/Ceph::Osds/Ceph::Osd[/srv/data]/Exec[ceph-osd-prepare-/srv/data]/returns: + getent passwd ceph
Notice: /Stage[main]/Ceph::Osds/Ceph::Osd[/srv/data]/Exec[ceph-osd-prepare-/srv/data]/returns: + chown -h ceph:ceph /srv/data
Notice: /Stage[main]/Ceph::Osds/Ceph::Osd[/srv/data]/Exec[ceph-osd-prepare-/srv/data]/returns: ++ readlink -f /srv/data
Notice: /Stage[main]/Ceph::Osds/Ceph::Osd[/srv/data]/Exec[ceph-osd-prepare-/srv/data]/returns: ++ readlink -f ''
Notice: /Stage[main]/Ceph::Osds/Ceph::Osd[/srv/data]/Exec[ceph-osd-prepare-/srv/data]/returns: + ceph-disk prepare --cluster ceph --cluster-uuid eb2bb192-b1c9-11e6-9205-525400330666 /srv/data
Notice: /Stage[main]/Ceph::Osds/Ceph::Osd[/srv/data]/Exec[ceph-osd-prepare-/srv/data]/returns: + udevadm settle
Notice: /Stage[main]/Ceph::Osds/Ceph::Osd[/srv/data]/Exec[ceph-osd-prepare-/srv/data]/returns: executed successfully
Notice: /Stage[main]/Ceph::Osds/Ceph::Osd[/srv/data]/Exec[fcontext_/srv/data]/returns: executed successfully
Notice: /Stage[main]/Tripleo::Firewall/Exec[nonpersistent_v4_rules_cleanup]/returns: executed successfully
Notice: /Stage[main]/Tripleo::Firewall/Exec[reload_iptables]: Triggered 'refresh' from 1 events
Notice: /Stage[main]/Tripleo::Firewall/Exec[nonpersistent_v6_rules_cleanup]/returns: executed successfully
Notice: /Stage[main]/Tripleo::Firewall/Exec[reload_ip6tables]: Triggered 'refresh' from 1 events
Notice: Applied catalog in 306.67 seconds
deploy_stderr: |
exception: connect failed
Warning: This method is deprecated, please use match expressions with Stdlib::Compat::Array instead. They are described at https://docs.puppet.com/puppet/latest/reference/lang_data_type.html#match-expressions. at ["/var/lib/heat-config/heat-config-puppet/3f729bd7-2e89-46dd-8ffd-9db4ad9b08dc.pp", 32]:["/etc/puppet/modules/tripleo/manifests/profile/base/ceph/client.pp", 29]
(at /etc/puppet/modules/stdlib/lib/puppet/functions/deprecation.rb:25:in deprecation')
Warning: This method is deprecated, please use match expressions with Stdlib::Compat::Ipv6 instead. They are described at https://docs.puppet.com/puppet/latest/reference/lang_data_type.html#match-expressions. at ["/var/lib/heat-config/heat-config-puppet/3f729bd7-2e89-46dd-8ffd-9db4ad9b08dc.pp", 32]:["/etc/puppet/modules/tripleo/manifests/profile/base/ceph/client.pp", 29]
(at /etc/puppet/modules/stdlib/lib/puppet/functions/deprecation.rb:25:indeprecation')
Warning: ModuleLoader: module 'ceph' has unresolved dependencies - it will only see those that are resolved. Use 'puppet module list --tree' to see information about modules
(file & line not available)
Warning: This method is deprecated, please use the stdlib validate_legacy function, with Stdlib::Compat::Hash. There is further documentation for validate_legacy function in the README. at ["/var/lib/heat-config/heat-config-puppet/3f729bd7-2e89-46dd-8ffd-9db4ad9b08dc.pp", 32]:["/etc/puppet/modules/tripleo/manifests/profile/base/ceph/client.pp", 29]
(at /etc/puppet/modules/stdlib/lib/puppet/functions/deprecation.rb:25:in deprecation')
Warning: ModuleLoader: module 'timezone' has unresolved dependencies - it will only see those that are resolved. Use 'puppet module list --tree' to see information about modules
(file & line not available)
Warning: This method is deprecated, please use the stdlib validate_legacy function, with Stdlib::Compat::Bool. There is further documentation for validate_legacy function in the README. at ["/var/lib/heat-config/heat-config-puppet/3f729bd7-2e89-46dd-8ffd-9db4ad9b08dc.pp", 34]:
(at /etc/puppet/modules/stdlib/lib/puppet/functions/deprecation.rb:25:indeprecation')
Warning: This method is deprecated, please use the stdlib validate_legacy function, with Stdlib::Compat::Absolute_Path. There is further documentation for validate_legacy function in the README. at ["/var/lib/heat-config/heat-config-puppet/3f729bd7-2e89-46dd-8ffd-9db4ad9b08dc.pp", 35]:["/etc/puppet/modules/tripleo/manifests/profile/base/time/ntp.pp", 29]
(at /etc/puppet/modules/stdlib/lib/puppet/functions/deprecation.rb:25:in deprecation')
Warning: This method is deprecated, please use the stdlib validate_legacy function, with Stdlib::Compat::String. There is further documentation for validate_legacy function in the README. at ["/var/lib/heat-config/heat-config-puppet/3f729bd7-2e89-46dd-8ffd-9db4ad9b08dc.pp", 35]:["/etc/puppet/modules/tripleo/manifests/profile/base/time/ntp.pp", 29]
(at /etc/puppet/modules/stdlib/lib/puppet/functions/deprecation.rb:25:indeprecation')
Warning: This method is deprecated, please use the stdlib validate_legacy function, with Stdlib::Compat::Array. There is further documentation for validate_legacy function in the README. at ["/var/lib/heat-config/heat-config-puppet/3f729bd7-2e89-46dd-8ffd-9db4ad9b08dc.pp", 35]:["/etc/puppet/modules/tripleo/manifests/profile/base/time/ntp.pp", 29]
(at /etc/puppet/modules/stdlib/lib/puppet/functions/deprecation.rb:25:in deprecation')
Warning: This method is deprecated, please use the stdlib validate_legacy function, with Pattern[]. There is further documentation for validate_legacy function in the README. at ["/var/lib/heat-config/heat-config-puppet/3f729bd7-2e89-46dd-8ffd-9db4ad9b08dc.pp", 35]:["/etc/puppet/modules/tripleo/manifests/profile/base/time/ntp.pp", 29]
(at /etc/puppet/modules/stdlib/lib/puppet/functions/deprecation.rb:25:indeprecation')
Warning: This method is deprecated, please use the stdlib validate_legacy function, with Stdlib::Compat::Numeric. There is further documentation for validate_legacy function in the README. at ["/var/lib/heat-config/heat-config-puppet/3f729bd7-2e89-46dd-8ffd-9db4ad9b08dc.pp", 35]:["/etc/puppet/modules/tripleo/manifests/profile/base/time/ntp.pp", 29]
(at /etc/puppet/modules/stdlib/lib/puppet/functions/deprecation.rb:25:in deprecation')
Warning: ModuleLoader: module 'openstacklib' has unresolved dependencies - it will only see those that are resolved. Use 'puppet module list --tree' to see information about modules
(file & line not available)
Warning: Scope(Class[Neutron]): neutron::rabbit_host, neutron::rabbit_hosts, neutron::rabbit_password, neutron::rabbit_port, neutron::rabbit_user and neutron::rabbit_virtual_host are deprecated. Please use neutron::default_transport_url instead.
Warning: Scope(Class[Ceilometer]): ceilometer::rabbit_host, ceilometer::rabbit_hosts, ceilometer::rabbit_password, ceilometer::rabbit_port, ceilometer::rabbit_userid and ceilometer::rabbit_virtual_host are deprecated. Please use ceilometer::default_transport_url instead.
Warning: This method is deprecated, please use match expressions with Stdlib::Compat::String instead. They are described at https://docs.puppet.com/puppet/latest/reference/lang_data_type.html#match-expressions. at :
(at /etc/puppet/modules/stdlib/lib/puppet/functions/deprecation.rb:25:indeprecation')
Warning: This method is deprecated, please use match expressions with Stdlib::Compat::Bool instead. They are described at https://docs.puppet.com/puppet/latest/reference/lang_data_type.html#match-expressions. at :
(at /etc/puppet/modules/stdlib/lib/puppet/functions/deprecation.rb:25:in deprecation')
Warning: Scope(Oslo::Messaging::Rabbit[neutron_config]): The oslo_messaging rabbit_host, rabbit_hosts, rabbit_port, rabbit_userid, rabbit_password, rabbit_virtual_host parameters have been deprecated by the [DEFAULT]\transport_url. Please use oslo::messaging::default::transport_url instead.
Warning: Scope(Oslo::Messaging::Rabbit[ceilometer_config]): The oslo_messaging rabbit_host, rabbit_hosts, rabbit_port, rabbit_userid, rabbit_password, rabbit_virtual_host parameters have been deprecated by the [DEFAULT]\transport_url. Please use oslo::messaging::default::transport_url instead.
Error: Command exceeded timeout
Error: /Stage[main]/Ceph::Osds/Ceph::Osd[/srv/data]/Exec[ceph-osd-activate-/srv/data]/returns: change from notrun to 0 failed: Command exceeded timeout
overcloud.AllNodesDeploySteps.ComputeDeployment_Step3.0:
resource_type: OS::Heat::StructuredDeployment
physical_resource_id: f7e0be0e-021c-43c9-be02-89cceaabd28e
status: CREATE_FAILED
status_reason: |
Error: resources[0]: Deployment to server failed: deploy_status_code : Deployment exited with non-zero status code: 6
deploy_stdout: |
Notice: hiera(): Cannot load backend module_data: cannot load such file -- hiera/backend/module_data_backend
Notice: Scope(Class[Tripleo::Firewall::Post]): At this stage, all network traffic is blocked.
Notice: Compiled catalog for overcloud-compute-0.localdomain in environment production in 4.17 seconds
Notice: /Stage[main]/Main/Package_manifest[/var/lib/tripleo/installed-packages/overcloud_compute3]/ensure: created
Notice: /Stage[main]/Neutron/Neutron_config[DEFAULT/auth_strategy]/ensure: created
Notice: /Stage[main]/Neutron/Neutron_config[DEFAULT/core_plugin]/ensure: created
Notice: /Stage[main]/Neutron/Neutron_config[DEFAULT/host]/ensure: created
Notice: /Stage[main]/Neutron/Neutron_config[DEFAULT/dhcp_agents_per_network]/ensure: created
Notice: /Stage[main]/Neutron/Neutron_config[DEFAULT/allow_overlapping_ips]/ensure: created
Notice: /Stage[main]/Neutron/Neutron_config[DEFAULT/global_physnet_mtu]/ensure: created
Notice: /Stage[main]/Neutron/Neutron_config[agent/root_helper]/ensure: created
Notice: /Stage[main]/Neutron/Neutron_config[DEFAULT/service_plugins]/ensure: created
Notice: /Stage[main]/Neutron::Plugins::Ml2/File[/etc/neutron/plugin.ini]/ensure: created
Notice: /Stage[main]/Neutron::Plugins::Ml2/File[/etc/default/neutron-server]/ensure: created
Notice: /Stage[main]/Neutron::Plugins::Ml2/Neutron_plugin_ml2[ml2/type_drivers]/ensure: created
Notice: /Stage[main]/Neutron::Plugins::Ml2/Neutron_plugin_ml2[ml2/tenant_network_types]/ensure: created
Notice: /Stage[main]/Neutron::Plugins::Ml2/Neutron_plugin_ml2[ml2/mechanism_drivers]/ensure: created
Notice: /Stage[main]/Neutron::Plugins::Ml2/Neutron_plugin_ml2[ml2/path_mtu]/ensure: created
Notice: /Stage[main]/Neutron::Plugins::Ml2/Neutron_plugin_ml2[ml2/extension_drivers]/ensure: created
Notice: /Stage[main]/Neutron::Plugins::Ml2/Neutron_plugin_ml2[securitygroup/firewall_driver]/ensure: created
Notice: /Stage[main]/Ceilometer/Ceilometer_config[DEFAULT/http_timeout]/ensure: created
Notice: /Stage[main]/Ceilometer/Ceilometer_config[DEFAULT/host]/ensure: created
Notice: /Stage[main]/Ceilometer/Ceilometer_config[publisher/telemetry_secret]/ensure: created
Notice: /Stage[main]/Ceilometer/Ceilometer_config[database/event_time_to_live]/ensure: created
Notice: /Stage[main]/Ceilometer/Ceilometer_config[database/metering_time_to_live]/ensure: created
Notice: /Stage[main]/Tripleo::Profile::Base::Kernel/Kmod::Load[nf_conntrack_proto_sctp]/Exec[modprobe nf_conntrack_proto_sctp]/returns: executed successfully
Notice: /Stage[main]/Neutron::Logging/Oslo::Log[neutron_config]/Neutron_config[DEFAULT/log_dir]/ensure: created
Notice: /Stage[main]/Neutron/Oslo::Messaging::Default[neutron_config]/Neutron_config[DEFAULT/control_exchange]/ensure: created
Notice: /Stage[main]/Neutron/Oslo::Concurrency[neutron_config]/Neutron_config[oslo_concurrency/lock_path]/ensure: created
Notice: /Stage[main]/Neutron/Oslo::Messaging::Rabbit[neutron_config]/Neutron_config[oslo_messaging_rabbit/heartbeat_timeout_threshold]/ensure: created
Notice: /Stage[main]/Neutron/Oslo::Messaging::Rabbit[neutron_config]/Neutron_config[oslo_messaging_rabbit/rabbit_password]/ensure: created
Notice: /Stage[main]/Neutron/Oslo::Messaging::Rabbit[neutron_config]/Neutron_config[oslo_messaging_rabbit/rabbit_use_ssl]/ensure: created
Notice: /Stage[main]/Neutron/Oslo::Messaging::Rabbit[neutron_config]/Neutron_config[oslo_messaging_rabbit/rabbit_userid]/ensure: created
Notice: /Stage[main]/Neutron/Oslo::Messaging::Rabbit[neutron_config]/Neutron_config[oslo_messaging_rabbit/rabbit_hosts]/ensure: created
Notice: /Stage[main]/Neutron/Oslo::Messaging::Rabbit[neutron_config]/Neutron_config[oslo_messaging_rabbit/rabbit_ha_queues]/ensure: created
Notice: /Stage[main]/Neutron/Oslo::Messaging::Rabbit[neutron_config]/Neutron_config[DEFAULT/rpc_backend]/ensure: created
Notice: /Stage[main]/Neutron::Plugins::Ml2/Neutron::Plugins::Ml2::Type_driver[vxlan]/Neutron_plugin_ml2[ml2_type_vxlan/vxlan_group]/ensure: created
Notice: /Stage[main]/Neutron::Plugins::Ml2/Neutron::Plugins::Ml2::Type_driver[vxlan]/Neutron_plugin_ml2[ml2_type_vxlan/vni_ranges]/ensure: created
Notice: /Stage[main]/Neutron::Plugins::Ml2/Neutron::Plugins::Ml2::Type_driver[vlan]/Neutron_plugin_ml2[ml2_type_vlan/network_vlan_ranges]/ensure: created
Notice: /Stage[main]/Neutron::Plugins::Ml2/Neutron::Plugins::Ml2::Type_driver[flat]/Neutron_plugin_ml2[ml2_type_flat/flat_networks]/ensure: created
Notice: /Stage[main]/Neutron::Plugins::Ml2/Neutron::Plugins::Ml2::Type_driver[gre]/Neutron_plugin_ml2[ml2_type_gre/tunnel_id_ranges]/ensure: created
Notice: /Stage[main]/Neutron::Deps/Anchor[neutron::config::end]: Triggered 'refresh' from 31 events
Notice: /Stage[main]/Neutron::Deps/Anchor[neutron::service::begin]: Triggered 'refresh' from 1 events
Notice: /Stage[main]/Ceilometer::Logging/Oslo::Log[ceilometer_config]/Ceilometer_config[DEFAULT/log_dir]/ensure: created
Notice: /Stage[main]/Ceilometer/Oslo::Messaging::Rabbit[ceilometer_config]/Ceilometer_config[oslo_messaging_rabbit/heartbeat_timeout_threshold]/ensure: created
Notice: /Stage[main]/Ceilometer/Oslo::Messaging::Rabbit[ceilometer_config]/Ceilometer_config[oslo_messaging_rabbit/rabbit_password]/ensure: created
Notice: /Stage[main]/Ceilometer/Oslo::Messaging::Rabbit[ceilometer_config]/Ceilometer_config[oslo_messaging_rabbit/rabbit_use_ssl]/ensure: created
Notice: /Stage[main]/Ceilometer/Oslo::Messaging::Rabbit[ceilometer_config]/Ceilometer_config[oslo_messaging_rabbit/rabbit_userid]/ensure: created
Notice: /Stage[main]/Ceilometer/Oslo::Messaging::Rabbit[ceilometer_config]/Ceilometer_config[oslo_messaging_rabbit/rabbit_hosts]/ensure: created
Notice: /Stage[main]/Ceilometer/Oslo::Messaging::Rabbit[ceilometer_config]/Ceilometer_config[oslo_messaging_rabbit/rabbit_ha_queues]/ensure: created
Notice: /Stage[main]/Ceilometer/Oslo::Messaging::Rabbit[ceilometer_config]/Ceilometer_config[DEFAULT/rpc_backend]/ensure: created
Notice: /Stage[main]/Ceilometer/Oslo::Messaging::Notifications[ceilometer_config]/Ceilometer_config[oslo_messaging_notifications/topics]/ensure: created
Notice: /Stage[main]/Ceilometer::Deps/Anchor[ceilometer::config::end]: Triggered 'refresh' from 14 events
Notice: /Stage[main]/Ceilometer::Deps/Anchor[ceilometer::service::begin]: Triggered 'refresh' from 1 events
Notice: /Stage[main]/Ceph::Osds/Ceph::Osd[/srv/data]/Exec[ceph-osd-prepare-/srv/data]/returns: ++ readlink -f /srv/data
Notice: /Stage[main]/Ceph::Osds/Ceph::Osd[/srv/data]/Exec[ceph-osd-prepare-/srv/data]/returns: + disk=/srv/data
Notice: /Stage[main]/Ceph::Osds/Ceph::Osd[/srv/data]/Exec[ceph-osd-prepare-/srv/data]/returns: + test -b /srv/data
Notice: /Stage[main]/Ceph::Osds/Ceph::Osd[/srv/data]/Exec[ceph-osd-prepare-/srv/data]/returns: + echo /srv/data
Notice: /Stage[main]/Ceph::Osds/Ceph::Osd[/srv/data]/Exec[ceph-osd-prepare-/srv/data]/returns: + egrep -e '^/dev' -q -v
Notice: /Stage[main]/Ceph::Osds/Ceph::Osd[/srv/data]/Exec[ceph-osd-prepare-/srv/data]/returns: + mkdir -p /srv/data
Notice: /Stage[main]/Ceph::Osds/Ceph::Osd[/srv/data]/Exec[ceph-osd-prepare-/srv/data]/returns: + getent passwd ceph
Notice: /Stage[main]/Ceph::Osds/Ceph::Osd[/srv/data]/Exec[ceph-osd-prepare-/srv/data]/returns: + chown -h ceph:ceph /srv/data
Notice: /Stage[main]/Ceph::Osds/Ceph::Osd[/srv/data]/Exec[ceph-osd-prepare-/srv/data]/returns: ++ readlink -f /srv/data
Notice: /Stage[main]/Ceph::Osds/Ceph::Osd[/srv/data]/Exec[ceph-osd-prepare-/srv/data]/returns: ++ readlink -f ''
Notice: /Stage[main]/Ceph::Osds/Ceph::Osd[/srv/data]/Exec[ceph-osd-prepare-/srv/data]/returns: + ceph-disk prepare --cluster ceph --cluster-uuid eb2bb192-b1c9-11e6-9205-525400330666 /srv/data
Notice: /Stage[main]/Ceph::Osds/Ceph::Osd[/srv/data]/Exec[ceph-osd-prepare-/srv/data]/returns: + udevadm settle
Notice: /Stage[main]/Ceph::Osds/Ceph::Osd[/srv/data]/Exec[ceph-osd-prepare-/srv/data]/returns: executed successfully
Notice: /Stage[main]/Ceph::Osds/Ceph::Osd[/srv/data]/Exec[fcontext_/srv/data]/returns: executed successfully
Notice: /Stage[main]/Tripleo::Firewall/Exec[nonpersistent_v4_rules_cleanup]/returns: executed successfully
Notice: /Stage[main]/Tripleo::Firewall/Exec[reload_iptables]: Triggered 'refresh' from 1 events
Notice: /Stage[main]/Tripleo::Firewall/Exec[nonpersistent_v6_rules_cleanup]/returns: executed successfully
Notice: /Stage[main]/Tripleo::Firewall/Exec[reload_ip6tables]: Triggered 'refresh' from 1 events
Notice: Applied catalog in 306.83 seconds
deploy_stderr: |
exception: connect failed
Warning: This method is deprecated, please use match expressions with Stdlib::Compat::Array instead. They are described at https://docs.puppet.com/puppet/latest/reference/lang_data_type.html#match-expressions. at ["/var/lib/heat-config/heat-config-puppet/944cf9fa-229d-4b1f-8d75-49ea31a4de30.pp", 32]:["/etc/puppet/modules/tripleo/manifests/profile/base/ceph/client.pp", 29]
(at /etc/puppet/modules/stdlib/lib/puppet/functions/deprecation.rb:25:indeprecation')
Warning: This method is deprecated, please use match expressions with Stdlib::Compat::Ipv6 instead. They are described at https://docs.puppet.com/puppet/latest/reference/lang_data_type.html#match-expressions. at ["/var/lib/heat-config/heat-config-puppet/944cf9fa-229d-4b1f-8d75-49ea31a4de30.pp", 32]:["/etc/puppet/modules/tripleo/manifests/profile/base/ceph/client.pp", 29]
(at /etc/puppet/modules/stdlib/lib/puppet/functions/deprecation.rb:25:in deprecation')
Warning: ModuleLoader: module 'ceph' has unresolved dependencies - it will only see those that are resolved. Use 'puppet module list --tree' to see information about modules
(file & line not available)
Warning: This method is deprecated, please use the stdlib validate_legacy function, with Stdlib::Compat::Hash. There is further documentation for validate_legacy function in the README. at ["/var/lib/heat-config/heat-config-puppet/944cf9fa-229d-4b1f-8d75-49ea31a4de30.pp", 32]:["/etc/puppet/modules/tripleo/manifests/profile/base/ceph/client.pp", 29]
(at /etc/puppet/modules/stdlib/lib/puppet/functions/deprecation.rb:25:indeprecation')
Warning: ModuleLoader: module 'timezone' has unresolved dependencies - it will only see those that are resolved. Use 'puppet module list --tree' to see information about modules
(file & line not available)
Warning: This method is deprecated, please use the stdlib validate_legacy function, with Stdlib::Compat::Bool. There is further documentation for validate_legacy function in the README. at ["/var/lib/heat-config/heat-config-puppet/944cf9fa-229d-4b1f-8d75-49ea31a4de30.pp", 34]:
(at /etc/puppet/modules/stdlib/lib/puppet/functions/deprecation.rb:25:in deprecation')
Warning: This method is deprecated, please use the stdlib validate_legacy function, with Stdlib::Compat::Absolute_Path. There is further documentation for validate_legacy function in the README. at ["/var/lib/heat-config/heat-config-puppet/944cf9fa-229d-4b1f-8d75-49ea31a4de30.pp", 35]:["/etc/puppet/modules/tripleo/manifests/profile/base/time/ntp.pp", 29]
(at /etc/puppet/modules/stdlib/lib/puppet/functions/deprecation.rb:25:indeprecation')
Warning: This method is deprecated, please use the stdlib validate_legacy function, with Stdlib::Compat::String. There is further documentation for validate_legacy function in the README. at ["/var/lib/heat-config/heat-config-puppet/944cf9fa-229d-4b1f-8d75-49ea31a4de30.pp", 35]:["/etc/puppet/modules/tripleo/manifests/profile/base/time/ntp.pp", 29]
(at /etc/puppet/modules/stdlib/lib/puppet/functions/deprecation.rb:25:in deprecation')
Warning: This method is deprecated, please use the stdlib validate_legacy function, with Stdlib::Compat::Array. There is further documentation for validate_legacy function in the README. at ["/var/lib/heat-config/heat-config-puppet/944cf9fa-229d-4b1f-8d75-49ea31a4de30.pp", 35]:["/etc/puppet/modules/tripleo/manifests/profile/base/time/ntp.pp", 29]
(at /etc/puppet/modules/stdlib/lib/puppet/functions/deprecation.rb:25:indeprecation')
Warning: This method is deprecated, please use the stdlib validate_legacy function, with Pattern[]. There is further documentation for validate_legacy function in the README. at ["/var/lib/heat-config/heat-config-puppet/944cf9fa-229d-4b1f-8d75-49ea31a4de30.pp", 35]:["/etc/puppet/modules/tripleo/manifests/profile/base/time/ntp.pp", 29]
(at /etc/puppet/modules/stdlib/lib/puppet/functions/deprecation.rb:25:in deprecation')
Warning: This method is deprecated, please use the stdlib validate_legacy function, with Stdlib::Compat::Numeric. There is further documentation for validate_legacy function in the README. at ["/var/lib/heat-config/heat-config-puppet/944cf9fa-229d-4b1f-8d75-49ea31a4de30.pp", 35]:["/etc/puppet/modules/tripleo/manifests/profile/base/time/ntp.pp", 29]
(at /etc/puppet/modules/stdlib/lib/puppet/functions/deprecation.rb:25:indeprecation')
Warning: ModuleLoader: module 'openstacklib' has unresolved dependencies - it will only see those that are resolved. Use 'puppet module list --tree' to see information about modules
(file & line not available)
Warning: Scope(Class[Neutron]): neutron::rabbit_host, neutron::rabbit_hosts, neutron::rabbit_password, neutron::rabbit_port, neutron::rabbit_user and neutron::rabbit_virtual_host are deprecated. Please use neutron::default_transport_url instead.
Warning: Scope(Class[Ceilometer]): ceilometer::rabbit_host, ceilometer::rabbit_hosts, ceilometer::rabbit_password, ceilometer::rabbit_port, ceilometer::rabbit_userid and ceilometer::rabbit_virtual_host are deprecated. Please use ceilometer::default_transport_url instead.
Warning: This method is deprecated, please use match expressions with Stdlib::Compat::String instead. They are described at https://docs.puppet.com/puppet/latest/reference/lang_data_type.html#match-expressions. at :
(at /etc/puppet/modules/stdlib/lib/puppet/functions/deprecation.rb:25:in deprecation')
Warning: This method is deprecated, please use match expressions with Stdlib::Compat::Bool instead. They are described at https://docs.puppet.com/puppet/latest/reference/lang_data_type.html#match-expressions. at :
(at /etc/puppet/modules/stdlib/lib/puppet/functions/deprecation.rb:25:indeprecation')
Warning: Scope(Oslo::Messaging::Rabbit[neutron_config]): The oslo_messaging rabbit_host, rabbit_hosts, rabbit_port, rabbit_userid, rabbit_password, rabbit_virtual_host parameters have been deprecated by the [DEFAULT]\transport_url. Please use oslo::messaging::default::transport_url instead.
Warning: Scope(Oslo::Messaging::Rabbit[ceilometer_config]): The oslo_messaging rabbit_host, rabbit_hosts, rabbit_port, rabbit_userid, rabbit_password, rabbit_virtual_host parameters have been deprecated by the [DEFAULT]\transport_url. Please use oslo::messaging::default::transport_url instead.
Error: Command exceeded timeout
Error: /Stage[main]/Ceph::Osds/Ceph::Osd[/srv/data]/Exec[ceph-osd-activate-/srv/data]/returns: change from notrun to 0 failed: Command exceeded timeout
So the problem is most likely that puppet is trying to run ceph-disk activate on the OSD and hanging. Two ways to diagnose what's going wrong:
- Check the ceph logs, which I think are in
/var/logs/ceph/. - Check
journalctlforceph-disk activateerror messages. - Run
ceph-disk activate /srv/dataon the node as the root user and see what happens.
Welcome! Check out the Getting Started with Red Hat page for quick tours and guides for common tasks.
