overcloud deploy exit for Permission denied

Latest response

Environment
Red Hat OpenStack Platform 7.2

Issue
1)Heat stack-list stack_status is CREATE_COMPLETE
2)Nova/Neutron/Glance endpoints are not created
3)Overcloud deploys has only keystone endpoint
4)Overcloud endpoints not created on deploy
5)Overcloud deploy fails
6)Undercloud ping keystone publicURL success
7)Controller-0 mysql service start failes, but it can start manually

I deployed three controller nodes and three compute nodes, and I tryed four times always failed in the "Permission denied".
I think it is different with [1].

[1] (https://access.redhat.com/solutions/2444561)

[root@undercloud stack]# time openstack overcloud deploy \
> --templates templates/openstack-tripleo-heat-templates/ \
> -e templates/openstack-tripleo-heat-templates/environments/network-isolation.yaml \
> -e templates/network-environment.yaml \
> --control-flavor control --compute-flavor compute \
> --control-scale 3 \
> --compute-scale 3 \
> --ceph-storage-scale 0 \
> --block-storage-scale 0 \
> --swift-storage-scale 0 \
> --ntp-server 192.168.191.253 \
> --neutron-network-type vlan \
> --neutron-bridge-mappings datacentre:br-ex \
> --neutron-network-vlan-ranges datacentre:200:1200 \
> --neutron-disable-tunneling \
> --timeout 90
Deploying templates in the directory /home/stack/templates/openstack-tripleo-heat-templates
/root/.ssh/known_hosts updated.
Original contents retained as /root/.ssh/known_hosts.old
PKI initialization in init-keystone is deprecated and will be removed.
Warning: Permanently added '192.168.191.57' (ECDSA) to the list of known hosts.
Permission denied (publickey,gssapi-keyex,gssapi-with-mic).
ERROR: openstack Command '['ssh', '-oStrictHostKeyChecking=no', '-t', '-l', 'heat-admin', u'192.168.191.57', 'sudo', 'keystone-manage', 'pki_setup', '--keystone-user', "$(getent passwd | grep '^keystone' | cut -d: -f1)", '--keystone-group', "$(getent group | grep '^keystone' | cut -d: -f1)"]' returned non-zero exit status 255

real    34m29.371s
user    0m2.375s
sys 0m0.287s

[root@undercloud stack]# heat stack-list
+--------------------------------------+------------+-----------------+----------------------+
| id                                   | stack_name | stack_status    | creation_time        |
+--------------------------------------+------------+-----------------+----------------------+
| 663a6941-f497-4542-92e7-2ffa625daff6 | overcloud  | CREATE_COMPLETE | 2016-12-12T12:35:33Z |
+--------------------------------------+------------+-----------------+----------------------+

[root@undercloud stack]# openstack catalog list
+----------+----------+-------------------------------------------------+
| Name     | Type     | Endpoints                                       |
+----------+----------+-------------------------------------------------+
| keystone | identity | regionOne                                       |
|          |          |   publicURL: http://192.168.192.151:5000/v2.0   |
|          |          |   internalURL: http://192.168.192.151:5000/v2.0 |
|          |          |   adminURL: http://192.168.192.151:5000/v2.0    |
|          |          |                                                 |
+----------+----------+-------------------------------------------------+
[root@undercloud stack]# ping 192.168.192.151
PING 192.168.192.151 (192.168.192.151) 56(84) bytes of data.
64 bytes from 192.168.192.151: icmp_seq=1 ttl=64 time=0.174 ms
64 bytes from 192.168.192.151: icmp_seq=2 ttl=64 time=0.166 ms
64 bytes from 192.168.192.151: icmp_seq=3 ttl=64 time=0.160 ms
64 bytes from 192.168.192.151: icmp_seq=4 ttl=64 time=0.156 ms

[root@undercloud stack]# nova list
ERROR (EndpointNotFound): publicURL endpoint for compute service not found
[root@undercloud stack]# glance image-list
public endpoint for image service not found
[root@undercloud stack]# neutron net-list
publicURL endpoint for network service not found

The undercloud node network:

[root@undercloud stack]# neutron net-list
+--------------------------------------+--------------+-------------------------------------------------------+
| id                                   | name         | subnets                                               |
+--------------------------------------+--------------+-------------------------------------------------------+
| 29921fa5-9c05-44d8-8f3d-c8c302c3cb63 | external     | 2898def8-2030-496b-80f1-9a1c11ece3a4 192.168.192.0/24 |
| 02f781ea-b38c-4bc6-91d2-388bcd0c8181 | storage_mgmt | 7763b44d-67a7-4a54-8e59-ff53a630556e 192.168.200.0/24 |
| 33f4d8a7-2995-454e-a7ca-0d9a04d746e5 | storage      | 322a964e-8dbe-4e65-a25b-0ca98f6b8c72 172.100.0.0/16   |
| 931564f1-2d41-4493-a898-95274a4b6c44 | tenant       | 8e5216a0-7cab-4410-8e47-cede6c104ecc 192.168.193.0/24 |
| 96dc121c-a22c-4b29-bc8c-70f2d68f9b66 | internal_api | 9ae7355b-e029-450d-a79d-662e60af5cb0 192.168.190.0/24 |
| f46a025c-6698-40c1-be08-64bc1944a2bd | ctlplane     | 920e01c1-ae01-4fd3-b093-892b3804b536 192.168.191.0/24 |
+--------------------------------------+--------------+-------------------------------------------------------+

The controller-0 os-collect-config is the ERRORs:

Dec 13 10:31:13 overcloud-controller-0.localdomain os-collect-config[4698]: haproxy/haproxy.cfg]/returns: executed successfully\u001b[0m\n\u001b[mNotice: /Stage[main]/Haproxy::Config/Concat[/etc/haproxy/haproxy.cfg]/Exec[concat_/etc/haproxy/haproxy.cfg]: Triggered 'refresh' from 41 events\u001b[0m\n\u001b[mNotice: /Stage[main]/Haproxy::Config/Concat[/etc/haproxy/haproxy.cfg]/File[/etc/haproxy/haproxy.cfg]/content: content changed '{md5}1f337186b0e1ba5ee82760cb437fb810' to '{md5}94f27aa67c234585323bf6b6746093bf'\u001b[0m\n\u001b[mNotice: /File[/etc/haproxy/haproxy.cfg]/seluser: seluser changed 'unconfined_u' to 'system_u'\u001b[0m\n\u001b[mNotice: Finished catalog run in 41.79 seconds\u001b[0m\n", "deploy_stderr": "Device \"br_ex\" does not exist.\nDevice \"ovs_system\" does not exist.\n\u001b[1;31mError: Could not prefetch mysql_user provider 'mysql': Execution of '/usr/bin/mysql -NBe SELECT CONCAT(User, '@',Host) AS User FROM mysql.user' returned 1: ERROR 2002 (HY000): Can't connect to local MySQL server through socket '/var/lib/mysql/mysql.sock' (2)\u001b[0m\n\u001b[1;31mError: Could not prefetch mysql_database provider 'mysql': Execution of '/usr/bin/mysql -NBe show databases' returned 1: ERROR 2002 (HY000): Can't connect to local MySQL server through socket '/var/lib/mysql/mysql.sock' (2)\u001b[0m\n", "deploy_status_code": 0}
Dec 13 10:31:13 overcloud-controller-0.localdomain os-collect-config[4698]: Error: Could not prefetch mysql_user provider 'mysql': Execution of '/usr/bin/mysql -NBe SELECT CONCAT(User, '@',Host) AS User FROM mysql.user' returned 1: ERROR 2002 (HY000): Can't connect to local MySQL server through socket '/var/lib/mysql/mysql.sock' (2)
Dec 13 10:31:13 overcloud-controller-0.localdomain os-collect-config[4698]: Error: Could not prefetch mysql_database provider 'mysql': Execution of '/usr/bin/mysql -NBe show databases' returned 1: ERROR 2002 (HY000): Can't connect to local MySQL server through socket '/var/lib/mysql/mysql.sock' (2)
Dec 13 10:33:07 overcloud-controller-0.localdomain os-collect-config[4698]: lic_vip-then-haproxy]/returns: executed successfully\u001b[0m\n\u001b[mNotice: /Stage[main]/Main/Pacemaker::Constraint::Base[redis_vip-then-haproxy]/Exec[Creating order constraint redis_vip-then-haproxy]/returns: executed successfully\u001b[0m\n\u001b[mNotice: /Stage[main]/Main/Pacemaker::Constraint::Base[storage_vip-then-haproxy]/Exec[Creating order constraint storage_vip-then-haproxy]/returns: executed successfully\u001b[0m\n\u001b[mNotice: /Stage[main]/Main/Pacemaker::Constraint::Colocation[public_vip-with-haproxy]/Pcmk_constraint[colo-ip-192.168.192.151-haproxy-clone]/ensure: created\u001b[0m\n\u001b[mNotice: Finished catalog run in 52.40 seconds\u001b[0m\n", "deploy_stderr": "Device \"br_ex\" does not exist.\nDevice \"ovs_system\" does not exist.\n\u001b[1;31mError: Could not prefetch mysql_user provider 'mysql': Execution of '/usr/bin/mysql -NBe SELECT CONCAT(User, '@',Host) AS User FROM mysql.user' returned 1: ERROR 2002 (HY000): Can't connect to local MySQL server through socket '/var/lib/mysql/mysql.sock' (2)\u001b[0m\n\u001b[1;31mError: Could not prefetch mysql_database provider 'mysql': Execution of '/usr/bin/mysql -NBe show databases' returned 1: ERROR 2002 (HY000): Can't connect to local MySQL server through socket '/var/lib/mysql/mysql.sock' (2)\u001b[0m\n", "deploy_status_code": 0}
Dec 13 10:33:08 overcloud-controller-0.localdomain os-collect-config[4698]: Error: Could not prefetch mysql_user provider 'mysql': Execution of '/usr/bin/mysql -NBe SELECT CONCAT(User, '@',Host) AS User FROM mysql.user' returned 1: ERROR 2002 (HY000): Can't connect to local MySQL server through socket '/var/lib/mysql/mysql.sock' (2)
Dec 13 10:33:08 overcloud-controller-0.localdomain os-collect-config[4698]: Error: Could not prefetch mysql_database provider 'mysql': Execution of '/usr/bin/mysql -NBe show databases' returned 1: ERROR 2002 (HY000): Can't connect to local MySQL server through socket '/var/lib/mysql/mysql.sock' (2)

Responses