Deploy (update) stack while one of osd has failed

Latest response

Community

Have faced quite interesting problem from operational perspective and looking forward to some advises how to go through.

So, have a OSP13 stack with integrated ceph deployed with using ceph-ansible. The stuck is up and running but one OSD is down due to physical disk failure. The new disk is on its way to me.

I have to add new functionality to the stack, so need to run deploy command. The deployment stops as ceph-asnible reports issue with the one of OSDs, I can't go through mistral step 2.

Any advises how run deploy on running stack and exclude the whole ceph-ansible flow or just exclude the single OSD from being checked ?

Piotr

PH
Log in to join the conversation

Responses

Daniel Macpherson's picture Red Hat Expert 1065 points
2 October 2018 11:47 AM

Hi Piotr,

What you want to do is pass node-specific disk configuration for the OSD with the failed disk. We've got a bug open at the moment to track documentation efforts on this item, plus the following comment in the bug shows how to accomplish this:

https://bugzilla.redhat.com/show_bug.cgi?id=1613933#c4

NR Active Contributor 245 points
8 October 2018 8:41 AM

Hi Piotr, we tried to deploy this. https://access.redhat.com/errata/RHEA-2018:2086 using this steps https://access.redhat.com/documentation/en-us/red_hat_openstack_platform/13/html/keeping_red_hat_openstack_platform_updated/updating_your_container_image_source#Configuring-Registry_Details-Satellite

During the upgrade we got these wartings

2018-10-04 16:25:35,387 INFO: Warning: ModuleLoader: module 'openstacklib' has unresolved dependencies - it will only see those that are resolved. Use 'puppet module list --tree' to see information about modules
2018-10-04 16:25:35,388 INFO:    (file & line not available)
2018-10-04 16:25:35,785 INFO: Notice: hiera(): Cannot load backend module_data: cannot load such file -- hiera/backend/module_data_backend
2018-10-04 16:25:35,849 INFO: Warning: This method is deprecated, please use the stdlib validate_legacy function,
2018-10-04 16:25:35,849 INFO:                     with Stdlib::Compat::Bool. There is further documentation for validate_legacy function in the README. at ["/etc/puppet/modules/ntp/manifests/init.pp", 54]:["/etc/puppet/modules/tripleo/manifests/profile/base/time/ntp.pp", 29]
2018-10-04 16:25:35,849 INFO:    (at /etc/puppet/modules/stdlib/lib/puppet/functions/deprecation.rb:28:in `deprecation')
2018-10-04 16:25:35,854 INFO: Warning: This method is deprecated, please use the stdlib validate_legacy function,
2018-10-04 16:25:35,855 INFO:                     with Stdlib::Compat::Absolute_Path. There is further documentation for validate_legacy function in the README. at ["/etc/puppet/modules/ntp/manifests/init.pp", 55]:["/etc/puppet/modules/tripleo/manifests/profile/base/time/ntp.pp", 29]
2018-10-04 16:25:35,855 INFO:    (at /etc/puppet/modules/stdlib/lib/puppet/functions/deprecation.rb:28:in `deprecation')
2018-10-04 16:25:35,875 INFO: Warning: This method is deprecated, please use the stdlib validate_legacy function,
2018-10-04 16:25:35,875 INFO:                     with Stdlib::Compat::String. There is further documentation for validate_legacy function in the README. at ["/etc/puppet/modules/ntp/manifests/init.pp", 56]:["/etc/puppet/modules/tripleo/manifests/profile/base/time/ntp.pp", 29]
2018-10-04 16:25:35,876 INFO:    (at /etc/puppet/modules/stdlib/lib/puppet/functions/deprecation.rb:28:in `deprecation')
2018-10-04 16:25:35,925 INFO: Warning: This method is deprecated, please use the stdlib validate_legacy function,
2018-10-04 16:25:35,925 INFO:                     with Stdlib::Compat::Array. There is further documentation for validate_legacy function in the README. at ["/etc/puppet/modules/ntp/manifests/init.pp", 66]:["/etc/puppet/modules/tripleo/manifests/profile/base/time/ntp.pp", 29]
2018-10-04 16:25:35,926 INFO:    (at /etc/puppet/modules/stdlib/lib/puppet/functions/deprecation.rb:28:in `deprecation')
2018-10-04 16:25:35,929 INFO: Warning: This method is deprecated, please use the stdlib validate_legacy function,
2018-10-04 16:25:35,929 INFO:                     with Pattern[]. There is further documentation for validate_legacy function in the README. at ["/etc/puppet/modules/ntp/manifests/init.pp", 68]:["/etc/puppet/modules/tripleo/manifests/profile/base/time/ntp.pp", 29]
2018-10-04 16:25:35,929 INFO:    (at /etc/puppet/modules/stdlib/lib/puppet/functions/deprecation.rb:28:in `deprecation')
2018-10-04 16:25:35,936 INFO: Warning: This method is deprecated, please use the stdlib validate_legacy function,
2018-10-04 16:25:35,936 INFO:                     with Stdlib::Compat::Numeric. There is further documentation for validate_legacy function in the README. at ["/etc/puppet/modules/ntp/manifests/init.pp", 76]:["/etc/puppet/modules/tripleo/manifests/profile/base/time/ntp.pp", 29]
2018-10-04 16:25:35,936 INFO:    (at /etc/puppet/modules/stdlib/lib/puppet/functions/deprecation.rb:28:in `deprecation')
2018-10-04 16:25:36,265 INFO: Warning: This method is deprecated, please use match expressions with Stdlib::Compat::Ipv6 instead. They are described at https://docs.puppet.com/puppet/latest/reference/lang_data_type.html#match-expressions. at ["/etc/puppet/modules/rabbitmq/manifests/install/rabbitmqadmin.pp", 37]:["/etc/puppet/modules/rabbitmq/manifests/init.pp", 316]
2018-10-04 16:25:36,265 INFO:    (at /etc/puppet/modules/stdlib/lib/puppet/functions/deprecation.rb:28:in `deprecation')
2018-10-04 16:25:36,445 INFO: Notice: Scope(Class[Tripleo::Firewall::Post]): At this stage, all network traffic is blocked.
2018-10-04 16:25:36,653 INFO: Warning: This method is deprecated, please use the stdlib validate_legacy function,
2018-10-04 16:25:36,654 INFO:                     with Stdlib::Compat::Hash. There is further documentation for validate_legacy function in the README. at ["/etc/puppet/modules/tripleo/manifests/profile/base/database/mysql.pp", 103]:["/etc/puppet/manifests/puppet-stack-config.pp", 97]
2018-10-04 16:25:36,654 INFO:    (at /etc/puppet/modules/stdlib/lib/puppet/functions/deprecation.rb:28:in `deprecation')
2018-10-04 16:25:36,696 INFO: Warning: ModuleLoader: module 'mysql' has unresolved dependencies - it will only see those that are resolved. Use 'puppet module list --tree' to see information about modules
2018-10-04 16:25:36,697 INFO:    (file & line not available)
2018-10-04 16:25:36,937 INFO: Warning: ModuleLoader: module 'keystone' has unresolved dependencies - it will only see those that are resolved. Use 'puppet module list --tree' to see information about modules
2018-10-04 16:25:36,937 INFO:    (file & line not available)
2018-10-04 16:25:37,506 INFO: Warning: ModuleLoader: module 'glance' has unresolved dependencies - it will only see those that are resolved. Use 'puppet module list --tree' to see information about modules
2018-10-04 16:25:37,506 INFO:    (file & line not available)
2018-10-04 16:25:37,655 INFO: Warning: ModuleLoader: module 'nova' has unresolved dependencies - it will only see those that are resolved. Use 'puppet module list --tree' to see information about modules
2018-10-04 16:25:37,655 INFO:    (file & line not available)
2018-10-04 16:25:37,773 INFO: Warning: Unknown variable: '::nova::db::mysql_api::setup_cell0'. at /etc/puppet/modules/nova/manifests/db/mysql.pp:53:28
2018-10-04 16:25:37,806 INFO: Warning: ModuleLoader: module 'neutron' has unresolved dependencies - it will only see those that are resolved. Use 'puppet module list --tree' to see information about modules
2018-10-04 16:25:37,806 INFO:    (file & line not available)
2018-10-04 16:25:38,433 INFO: Warning: ModuleLoader: module 'heat' has unresolved dependencies - it will only see those that are resolved. Use 'puppet module list --tree' to see information about modules
2018-10-04 16:25:38,433 INFO:    (file & line not available)
2018-10-04 16:25:38,544 INFO: Warning: ModuleLoader: module 'ironic' has unresolved dependencies - it will only see those that are resolved. Use 'puppet module list --tree' to see information about modules
2018-10-04 16:25:38,544 INFO:    (file & line not available)
2018-10-04 16:25:38,632 INFO: Warning: ModuleLoader: module 'swift' has unresolved dependencies - it will only see those that are resolved. Use 'puppet module list --tree' to see information about modules
2018-10-04 16:25:38,632 INFO:    (file & line not available)
2018-10-04 16:25:38,919 INFO: Warning: Scope(Class[Keystone]): keystone::rabbit_host, keystone::rabbit_hosts, keystone::rabbit_password, keystone::rabbit_port, keystone::rabbit_userid and keystone::rabbit_virtual_host are deprecated. Please use keystone::default_transport_url instead.
2018-10-04 16:25:40,419 INFO: Warning: Scope(Class[Glance::Notify::Rabbitmq]): glance::notify::rabbitmq::rabbit_host, glance::notify::rabbitmq::rabbit_hosts, glance::notify::rabbitmq::rabbit_password, glance::notify::rabbitmq::rabbit_port, glance::notify::rabbitmq::rabbit_userid and glance::notify::rabbitmq::rabbit_virtual_host are deprecated. Please use glance::notify::rabbitmq::default_transport_url instead.
2018-10-04 16:25:40,634 INFO: Warning: Scope(Class[Nova::Db]): placement_database_connection has no effect as of pike, and may be removed in a future release
2018-10-04 16:25:40,635 INFO: Warning: Scope(Class[Nova::Db]): placement_slave_connection has no effect as of pike, and may be removed in a future release
2018-10-04 16:25:40,791 INFO: Warning: ModuleLoader: module 'cinder' has unresolved dependencies - it will only see those that are resolved. Use 'puppet module list --tree' to see information about modules
2018-10-04 16:25:40,792 INFO:    (file & line not available)
2018-10-04 16:25:41,209 INFO: Warning: Unknown variable: 'until_complete_real'. at /etc/puppet/modules/nova/manifests/cron/archive_deleted_rows.pp:77:82
2018-10-04 16:25:41,251 INFO: Warning: This method is deprecated, please use match expressions with Stdlib::Compat::Array instead. They are described at https://docs.puppet.com/puppet/latest/reference/lang_data_type.html#match-expressions. at ["/etc/puppet/modules/nova/manifests/scheduler/filter.pp", 140]:["/etc/puppet/manifests/puppet-stack-config.pp", 396]
2018-10-04 16:25:41,251 INFO:    (at /etc/puppet/modules/stdlib/lib/puppet/functions/deprecation.rb:28:in `deprecation')
2018-10-04 16:25:41,462 INFO: Warning: Scope(Class[Neutron]): neutron::rabbit_host, neutron::rabbit_hosts, neutron::rabbit_password, neutron::rabbit_port, neutron::rabbit_user, neutron::rabbit_virtual_host and neutron::rpc_backend are deprecated. Please use neutron::default_transport_url instead.
2018-10-04 16:25:42,463 INFO: Warning: Unknown variable: 'methods_real'. at /etc/puppet/modules/swift/manifests/proxy/tempurl.pp:100:56
2018-10-04 16:25:42,464 INFO: Warning: Unknown variable: 'incoming_remove_headers_real'. at /etc/puppet/modules/swift/manifests/proxy/tempurl.pp:101:56
2018-10-04 16:25:42,464 INFO: Warning: Unknown variable: 'incoming_allow_headers_real'. at /etc/puppet/modules/swift/manifests/proxy/tempurl.pp:102:56
2018-10-04 16:25:42,464 INFO: Warning: Unknown variable: 'outgoing_remove_headers_real'. at /etc/puppet/modules/swift/manifests/proxy/tempurl.pp:103:56
2018-10-04 16:25:42,464 INFO: Warning: Unknown variable: 'outgoing_allow_headers_real'. at /etc/puppet/modules/swift/manifests/proxy/tempurl.pp:104:56
2018-10-04 16:25:42,537 INFO: Warning: Scope(Class[Swift::Storage::All]): The default port for the object storage server has changed from 6000 to 6200 and will be changed in a later release
2018-10-04 16:25:42,537 INFO: Warning: Scope(Class[Swift::Storage::All]): The default port for the container storage server has changed from 6001 to 6201 and will be changed in a later release
2018-10-04 16:25:42,537 INFO: Warning: Scope(Class[Swift::Storage::All]): The default port for the account storage server has changed from 6002 to 6202 and will be changed in a later release
2018-10-04 16:25:43,032 INFO: Warning: This method is deprecated, please use the stdlib validate_legacy function,
2018-10-04 16:25:43,033 INFO:                     with Stdlib::Compat::Integer. There is further documentation for validate_legacy function in the README. at ["/etc/puppet/modules/heat/manifests/wsgi/apache_api_cfn.pp", 125]:["/etc/puppet/manifests/puppet-stack-config.pp", 517]
2018-10-04 16:25:43,033 INFO:    (at /etc/puppet/modules/stdlib/lib/puppet/functions/deprecation.rb:28:in `deprecation')
2018-10-04 16:25:43,427 INFO: Warning: Unknown variable: '::ironic::conductor::swift_account'. at /etc/puppet/modules/ironic/manifests/glance.pp:117:30
2018-10-04 16:25:43,428 INFO: Warning: Unknown variable: '::ironic::conductor::swift_temp_url_key'. at /etc/puppet/modules/ironic/manifests/glance.pp:118:35
2018-10-04 16:25:43,428 INFO: Warning: Unknown variable: '::ironic::conductor::swift_temp_url_duration'. at /etc/puppet/modules/ironic/manifests/glance.pp:119:40
2018-10-04 16:25:43,450 INFO: Warning: Unknown variable: '::ironic::api::neutron_url'. at /etc/puppet/modules/ironic/manifests/neutron.pp:58:29
2018-10-04 16:25:44,430 INFO: Warning: ModuleLoader: module 'mistral' has unresolved dependencies - it will only see those that are resolved. Use 'puppet module list --tree' to see information about modules
2018-10-04 16:25:44,431 INFO:    (file & line not available)
2018-10-04 16:25:44,505 INFO: Warning: Unknown variable: '::mistral::database_idle_timeout'. at /etc/puppet/modules/mistral/manifests/db.pp:57:40
2018-10-04 16:25:44,506 INFO: Warning: Unknown variable: '::mistral::database_min_pool_size'. at /etc/puppet/modules/mistral/manifests/db.pp:58:40
2018-10-04 16:25:44,507 INFO: Warning: Unknown variable: '::mistral::database_max_pool_size'. at /etc/puppet/modules/mistral/manifests/db.pp:59:40
2018-10-04 16:25:44,507 INFO: Warning: Unknown variable: '::mistral::database_max_retries'. at /etc/puppet/modules/mistral/manifests/db.pp:60:40
2018-10-04 16:25:44,508 INFO: Warning: Unknown variable: '::mistral::database_retry_interval'. at /etc/puppet/modules/mistral/manifests/db.pp:61:40
2018-10-04 16:25:44,508 INFO: Warning: Unknown variable: '::mistral::database_max_overflow'. at /etc/puppet/modules/mistral/manifests/db.pp:62:40
2018-10-04 16:25:44,562 INFO: Warning: Scope(Class[Mistral]): mistral::rabbit_host, mistral::rabbit_hosts, mistral::rabbit_password, mistral::rabbit_port, mistral::rabbit_userid, mistral::rabbit_virtual_host and mistral::rpc_backend are deprecated. Please use mistral::default_transport_url instead.
2018-10-04 16:25:44,920 INFO: Warning: ModuleLoader: module 'zaqar' has unresolved dependencies - it will only see those that are resolved. Use 'puppet module list --tree' to see information about modules
2018-10-04 16:25:44,920 INFO:    (file & line not available)
2018-10-04 16:25:46,057 INFO: Warning: ModuleLoader: module 'oslo' has unresolved dependencies - it will only see those that are resolved. Use 'puppet module list --tree' to see information about modules
2018-10-04 16:25:46,057 INFO:    (file & line not available)
2018-10-04 16:25:46,139 INFO: Warning: Scope(Oslo::Messaging::Rabbit[keystone_config]): The oslo_messaging rabbit_host, rabbit_hosts, rabbit_port, rabbit_userid, rabbit_password, rabbit_virtual_host parameters have been deprecated by the [DEFAULT]\transport_url. Please use oslo::messaging::default::transport_url instead.
2018-10-04 16:25:47,166 INFO: Warning: Scope(Oslo::Messaging::Rabbit[glance_api_config]): The oslo_messaging rabbit_host, rabbit_hosts, rabbit_port, rabbit_userid, rabbit_password, rabbit_virtual_host parameters have been deprecated by the [DEFAULT]\transport_url. Please use oslo::messaging::default::transport_url instead.
2018-10-04 16:25:47,176 INFO: Warning: Scope(Oslo::Messaging::Rabbit[glance_registry_config]): The oslo_messaging rabbit_host, rabbit_hosts, rabbit_port, rabbit_userid, rabbit_password, rabbit_virtual_host parameters have been deprecated by the [DEFAULT]\transport_url. Please use oslo::messaging::default::transport_url instead.
2018-10-04 16:25:47,305 INFO: Warning: Scope(Oslo::Messaging::Rabbit[neutron_config]): The oslo_messaging rabbit_host, rabbit_hosts, rabbit_port, rabbit_userid, rabbit_password, rabbit_virtual_host parameters have been deprecated by the [DEFAULT]\transport_url. Please use oslo::messaging::default::transport_url instead.
2018-10-04 16:25:47,357 INFO: Warning: Scope(Neutron::Plugins::Ml2::Type_driver[local]): local type_driver is useful only for single-box, because it provides no connectivity between hosts
2018-10-04 16:25:48,007 INFO: Warning: Scope(Oslo::Messaging::Rabbit[mistral_config]): The oslo_messaging rabbit_host, rabbit_hosts, rabbit_port, rabbit_userid, rabbit_password, rabbit_virtual_host parameters have been deprecated by the [DEFAULT]\transport_url. Please use oslo::messaging::default::transport_url instead.

overcloud deployment failed afterwards.

LB Community Member 27 points
26 October 2018 6:09 PM

Hi there,

I'm interested to add the CEPH MDS to a deployed OSP.

Is there a document that describes the procedure of adding functionalities to the overcloud?

Thank you! Laszlo

Close

Welcome! Check out the Getting Started with Red Hat page for quick tours and guides for common tasks.