Failed OSD prevents ovecloud scale out.

Issue

• Stack deployed with Director

- 1 Director Node.
- 3 Controller Nodes.
- 3 Ceph Nodes.
- 4 OVS Compute Nodes.
- 6 OVS-DPDK Compute Nodes.

• While attempting to scale out one new compute node deployment failed during "overcloud.AllNodesDeploySteps.CephStorageDeployment_Step3.0".

2019-07-17 16:34:29Z [overcloud-AllNodesDeploySteps-2te6y4avlhlf-ControllerDeployment_Step3-qzjybsezyaia.2]: SIGNAL_IN_PROGRESS  Signal: deployment 7a899f81-abd7-4d67-8957-267981172848 succeeded
2019-07-17 16:34:29Z [overcloud-AllNodesDeploySteps-2te6y4avlhlf-ControllerDeployment_Step3-qzjybsezyaia.2]: UPDATE_COMPLETE  state changed
2019-07-17 16:34:29Z [overcloud-AllNodesDeploySteps-2te6y4avlhlf-ComputeDeployment_Step3-ukdnc6onmddz.0]: SIGNAL_IN_PROGRESS  Signal: deployment 3f17d75b-2073-45dc-82de-80d5c66e6ef7 succeeded
2019-07-17 16:34:29Z [overcloud-AllNodesDeploySteps-2te6y4avlhlf-ControllerDeployment_Step3-qzjybsezyaia.1]: SIGNAL_IN_PROGRESS  Signal: deployment 6d6bfbec-06f1-40fe-81e5-acc01536464d succeeded
2019-07-17 16:34:29Z [overcloud-AllNodesDeploySteps-2te6y4avlhlf-ComputeDeployment_Step3-ukdnc6onmddz.0]: UPDATE_COMPLETE  state changed
2019-07-17 16:34:30Z [overcloud-AllNodesDeploySteps-2te6y4avlhlf-ControllerDeployment_Step3-qzjybsezyaia.1]: UPDATE_COMPLETE  state changed
2019-07-17 16:34:37Z [overcloud-AllNodesDeploySteps-2te6y4avlhlf-ComputeDeployment_Step3-ukdnc6onmddz.1]: SIGNAL_IN_PROGRESS  Signal: deployment 1f1f7200-5a53-4547-9fcc-cddec9d9dbcf succeeded
2019-07-17 16:34:37Z [overcloud-AllNodesDeploySteps-2te6y4avlhlf-ComputeDeployment_Step3-ukdnc6onmddz.1]: UPDATE_COMPLETE  state changed
2019-07-17 16:34:37Z [overcloud-AllNodesDeploySteps-2te6y4avlhlf-ComputeDeployment_Step3-ukdnc6onmddz]: UPDATE_COMPLETE  Stack UPDATE completed successfully
2019-07-17 16:34:37Z [overcloud-AllNodesDeploySteps-2te6y4avlhlf.ComputeDeployment_Step3]: UPDATE_COMPLETE  state changed
2019-07-17 16:35:00Z [overcloud-AllNodesDeploySteps-2te6y4avlhlf-CephStorageDeployment_Step3-sxbsuetjweag.2]: SIGNAL_IN_PROGRESS  Signal: deployment 56fe0f00-c371-446e-aefe-4de907c69dc9 succeeded
2019-07-17 16:35:00Z [overcloud-AllNodesDeploySteps-2te6y4avlhlf-CephStorageDeployment_Step3-sxbsuetjweag.2]: UPDATE_COMPLETE  state changed
2019-07-17 16:35:15Z [overcloud-AllNodesDeploySteps-2te6y4avlhlf-CephStorageDeployment_Step3-sxbsuetjweag.1]: SIGNAL_IN_PROGRESS  Signal: deployment f90fdea8-d0a5-423e-aaa9-9dc8057ccbd0 succeeded
2019-07-17 16:35:16Z [overcloud-AllNodesDeploySteps-2te6y4avlhlf-CephStorageDeployment_Step3-sxbsuetjweag.1]: UPDATE_COMPLETE  state changed
2019-07-17 16:35:21Z [overcloud-AllNodesDeploySteps-2te6y4avlhlf-CephStorageDeployment_Step3-sxbsuetjweag.0]: SIGNAL_IN_PROGRESS  Signal: deployment 05bbeecb-243e-4346-b72e-5a3af67e7451 failed (6)
2019-07-17 16:35:21Z [overcloud-AllNodesDeploySteps-2te6y4avlhlf-CephStorageDeployment_Step3-sxbsuetjweag.0]: UPDATE_FAILED  Error: resources[0]: Deployment to server failed: deploy_status_code : Deployment exited with non-zer
o status code: 6
2019-07-17 16:35:21Z [overcloud-AllNodesDeploySteps-2te6y4avlhlf-CephStorageDeployment_Step3-sxbsuetjweag]: UPDATE_FAILED  Error: resources[0]: Deployment to server failed: deploy_status_code : Deployment exited with non-zero
status code: 6
2019-07-17 16:35:22Z [overcloud-AllNodesDeploySteps-2te6y4avlhlf.CephStorageDeployment_Step3]: UPDATE_FAILED  resources.CephStorageDeployment_Step3: Error: resources[0]: Deployment to server failed: deploy_status_code : Deploy
ment exited with non-zero status code: 6
2019-07-17 16:35:22Z [overcloud-AllNodesDeploySteps-2te6y4avlhlf.ControllerDeployment_Step3]: UPDATE_FAILED  UPDATE aborted
2019-07-17 16:35:22Z [overcloud-AllNodesDeploySteps-2te6y4avlhlf]: UPDATE_FAILED  resources.CephStorageDeployment_Step3: Error: resources[0]: Deployment to server failed: deploy_status_code : Deployment exited with non-zero st
atus code: 6
2019-07-17 16:35:22Z [overcloud-AllNodesDeploySteps-2te6y4avlhlf-ControllerDeployment_Step3-qzjybsezyaia.0]: UPDATE_FAILED  UPDATE aborted
2019-07-17 16:35:22Z [overcloud-AllNodesDeploySteps-2te6y4avlhlf-ControllerDeployment_Step3-qzjybsezyaia]: UPDATE_FAILED  Operation cancelled
2019-07-17 16:35:23Z [AllNodesDeploySteps]: UPDATE_FAILED  resources.AllNodesDeploySteps: resources.CephStorageDeployment_Step3: Error: resources[0]: Deployment to server failed: deploy_status_code : Deployment exited with non
-zero status code: 6
2019-07-17 16:35:23Z [overcloud]: UPDATE_FAILED  resources.AllNodesDeploySteps: resources.CephStorageDeployment_Step3: Error: resources[0]: Deployment to server failed: deploy_status_code : Deployment exited with non-zero stat
us code: 6

 Stack overcloud UPDATE_FAILED

Heat Stack update failed.
END return value: 1