During upgrade RHOSP from 16.2 to 17.1, when "openstack undercloud upgrade -y" execution, below failure occured. "FATAL | Run undercloud-upgrade-ephemeral-heat.py | undercloud | error"

Issue

• When running 'openstack undercloud upgrade -y' we get the following error:

2024-06-10 17:15:27.204008 | 5cb9018b-5c05-62e3-3488-000000000711 |      FATAL | Run undercloud-upgrade-ephemeral-heat.py | undercloud | error={"changed": true, "cmd": "/var/lib/tripleo-config/scripts/undercloud-upgrade-ephemeral-heat.py", "delta": "0:00:09.596571", "end": "2024-06-10 17:15:27.182327", "msg": "non-zero return code", "rc": 1, "start": "2024-06-10 17:15:17.585756", "stderr": "INFO:undercloud:Exporting network from stack overcloud to /home/stack/overcloud-deploy/overcloud/tripleo-overcloud-network-data.yaml\nAnsible execution failed. playbook: /usr/share/ansible/tripleo-playbooks/cli-overcloud-network-extract.yaml, Run Status: failed, Return Code: 2\nException occured while running the command\nTraceback (most recent call last):\n  File \"/usr/lib/python3.6/site-packages/tripleoclient/command.py\", line 32, in run\n    super(Command, self).run(parsed_args)\n  File \"/usr/lib/python3.6/site-packages/osc_lib/command/command.py\", line 39, in run\n    return super(Command, self).run(parsed_args)\n  File \"/usr/lib/python3.6/site-packages/cliff/command.py\", line 186, in run\n    return_code = self.take_action(parsed_args) or 0\n  File \"/usr/lib/python3.6/site-packages/tripleoclient/v2/overcloud_network.py\", line 77, in take_action\n    extra_vars=extra_vars,\n  File \"/usr/lib/python3.6/site-packages/tripleoclient/utils.py\", line 775, in run_ansible_playbook\n    raise RuntimeError(err_msg)\nRuntimeError: Ansible execution failed. playbook: /usr/share/ansible/tripleo-playbooks/cli-overcloud-network-extract.yaml, Run Status: failed, Return Code: 2\nAnsible execution failed. playbook: /usr/share/ansible/tripleo-playbooks/cli-overcloud-network-extract.yaml, Run Status: failed, Return Code: 2\nTraceback (most recent call last):\n  File \"/var/lib/tripleo-config/scripts/undercloud-upgrade-ephemeral-heat.py\", line 443, in <module>\n    main()\n  File \"/var/lib/tripleo-config/scripts/undercloud-upgrade-ephemeral-heat.py\", line 381, in main\n    export_networks(stack, stack_dir, args.cloud)\n  File \"/var/lib/tripleo-config/scripts/undercloud-upgrade-ephemeral-heat.py\", line 278, in export_networks\n    '--yes'], env={'OS_CLOUD': cloud})\n  File \"/usr/lib64/python3.6/subprocess.py\", line 311, in check_call\n    raise CalledProcessError(retcode, cmd)\nsubprocess.CalledProcessError: Command '['openstack', 'overcloud', 'network', 'extract', '--stack', 'overcloud', '--output', '/home/stack/overcloud-deploy/overcloud/tripleo-overcloud-network-data.yaml', '--yes']' returned non-zero exit status 1.", "stderr_lines": ["INFO:undercloud:Exporting network from stack overcloud to /home/stack/overcloud-deploy/overcloud/tripleo-overcloud-network-data.yaml", "Ansible execution failed. playbook: /usr/share/ansible/tripleo-playbooks/cli-overcloud-network-extract.yaml, Run Status: failed, Return Code: 2", "Exception occured while running the command", "Traceback (most recent call last):", "  File \"/usr/lib/python3.6/site-packages/tripleoclient/command.py\", line 32, in run", "    super(Command, self).run(parsed_args)", "  File \"/usr/lib/python3.6/site-packages/osc_lib/command/command.py\", line 39, in run", "    return super(Command, self).run(parsed_args)", "  File \"/usr/lib/python3.6/site-packages/cliff/command.py\", line 186, in run", "    return_code = self.take_action(parsed_args) or 0", "  File \"/usr/lib/python3.6/site-packages/tripleoclient/v2/overcloud_network.py\", line 77, in take_action", "    extra_vars=extra_vars,", "  File \"/usr/lib/python3.6/site-packages/tripleoclient/utils.py\", line 775, in run_ansible_playbook", "    raise RuntimeError(err_msg)", "RuntimeError: Ansible execution failed. playbook: /usr/share/ansible/tripleo-playbooks/cli-overcloud-network-extract.yaml, Run Status: failed, Return Code: 2", "Ansible execution failed. playbook: /usr/share/ansible/tripleo-playbooks/cli-overcloud-network-extract.yaml, Run Status: failed, Return Code: 2", "Traceback (most recent call last):", "  File \"/var/lib/tripleo-config/scripts/undercloud-upgrade-ephemeral-heat.py\", line 443, in <module>", "    main()", "  File \"/var/lib/tripleo-config/scripts/undercloud-upgrade-ephemeral-heat.py\", line 381, in main", "    export_networks(stack, stack_dir, args.cloud)", "  File \"/var/lib/tripleo-config/scripts/undercloud-upgrade-ephemeral-heat.py\", line 278, in export_networks", "    '--yes'], env={'OS_CLOUD': cloud})", "  File \"/usr/lib64/python3.6/subprocess.py\", line 311, in check_call", "    raise CalledProcessError(retcode, cmd)", "subprocess.CalledProcessError: Command '['openstack', 'overcloud', 'network', 'extract', '--stack', 'overcloud', '--output', '/home/stack/overcloud-deploy/overcloud/tripleo-overcloud-network-data.yaml', '--yes']' returned non-zero exit status 1."], "stdout": "\r\nPLAY [Overcloud Network Extract Networks] **************************************\n2024-06-10 17:15:22.182574 | 5cb9018b-5c05-fc8a-9ff5-000000000008 |    SKIPPED | fail | localhost\n2024-06-10 17:15:22.183430 | 5cb9018b-5c05-fc8a-9ff5-000000000008 |     TIMING | fail | localhost | 0:00:00.070665 | 0.02s\n2024-06-10 17:15:22.210867 | 5cb9018b-5c05-fc8a-9ff5-000000000009 |    SKIPPED | fail | localhost\n2024-06-10 17:15:22.211677 | 5cb9018b-5c05-fc8a-9ff5-000000000009 |     TIMING | fail | localhost | 0:00:00.098915 | 0.02s\n2024-06-10 17:15:22.216023 | 5cb9018b-5c05-fc8a-9ff5-00000000000a |       TASK | Check if output file exists\n2024-06-10 17:15:22.443613 | 5cb9018b-5c05-fc8a-9ff5-00000000000a |         OK | Check if output file exists | localhost\n2024-06-10 17:15:22.444645 | 5cb9018b-5c05-fc8a-9ff5-00000000000a |     TIMING | Check if output file exists | localhost | 0:00:00.331882 | 0.23s\n2024-06-10 17:15:22.472406 | 5cb9018b-5c05-fc8a-9ff5-00000000000b |    SKIPPED | fail | localhost\n2024-06-10 17:15:22.473211 | 5cb9018b-5c05-fc8a-9ff5-00000000000b |     TIMING | fail | localhost | 0:00:00.360449 | 0.02s\n2024-06-10 17:15:22.478220 | 5cb9018b-5c05-fc8a-9ff5-00000000000d |       TASK | Get network data from overcloud stack\n2024-06-10 17:15:26.619861 | 5cb9018b-5c05-fc8a-9ff5-00000000000d |      FATAL | Get network data from overcloud stack | localhost | error={\"changed\": false, \"error\": \"'tripleo_net_idx'\", \"msg\": \"Error getting network data from overcloud stack overcloud: 'tripleo_net_idx'\", \"network_data\": [], \"success\": false}\n2024-06-1