"openstack command not working with stackrc" || Delhi CXP.

Issue

• "openstack command working with stackrc" to get detailed error run any oepnstack command followed by --debug

• We found the following in /var/log/containers/httpd/keystone_wsgi_error.log:

[Wed Apr 17 00:30:00.500567 2024] [wsgi:error] [pid 25] [remote 10.166.22.139:41374]     dbapi_connection = rec.get_connection()
[Wed Apr 17 00:30:00.500571 2024] [wsgi:error] [pid 25] [remote 10.166.22.139:41374]   File "/usr/lib64/python3.6/site-packages/sqlalchemy/pool/base.py", line 584, in get_connection
[Wed Apr 17 00:30:00.500575 2024] [wsgi:error] [pid 25] [remote 10.166.22.139:41374]     self.__connect()
[Wed Apr 17 00:30:00.500579 2024] [wsgi:error] [pid 25] [remote 10.166.22.139:41374]   File "/usr/lib64/python3.6/site-packages/sqlalchemy/pool/base.py", line 630, in __connect
[Wed Apr 17 00:30:00.500581 2024] [wsgi:error] [pid 25] [remote 10.166.22.139:41374]     connection = pool._invoke_creator(self)
[Wed Apr 17 00:30:00.500585 2024] [wsgi:error] [pid 25] [remote 10.166.22.139:41374]   File "/usr/lib64/python3.6/site-packages/sqlalchemy/engine/strategies.py", line 114, in connect
[Wed Apr 17 00:30:00.500587 2024] [wsgi:error] [pid 25] [remote 10.166.22.139:41374]     return dialect.connect(*cargs, **cparams)
[Wed Apr 17 00:30:00.500590 2024] [wsgi:error] [pid 25] [remote 10.166.22.139:41374]   File "/usr/lib64/python3.6/site-packages/sqlalchemy/engine/default.py", line 453, in connect
[Wed Apr 17 00:30:00.500593 2024] [wsgi:error] [pid 25] [remote 10.166.22.139:41374]     return self.dbapi.connect(*cargs, **cparams)
[Wed Apr 17 00:30:00.500596 2024] [wsgi:error] [pid 25] [remote 10.166.22.139:41374]   File "/usr/lib/python3.6/site-packages/pymysql/__init__.py", line 94, in Connect
[Wed Apr 17 00:30:00.500598 2024] [wsgi:error] [pid 25] [remote 10.166.22.139:41374]     return Connection(*args, **kwargs)
[Wed Apr 17 00:30:00.500602 2024] [wsgi:error] [pid 25] [remote 10.166.22.139:41374]   File "/usr/lib/python3.6/site-packages/pymysql/connections.py", line 327, in __init__
[Wed Apr 17 00:30:00.500604 2024] [wsgi:error] [pid 25] [remote 10.166.22.139:41374]     self.connect()
[Wed Apr 17 00:30:00.500607 2024] [wsgi:error] [pid 25] [remote 10.166.22.139:41374]   File "/usr/lib/python3.6/site-packages/pymysql/connections.py", line 587, in connect
[Wed Apr 17 00:30:00.500609 2024] [wsgi:error] [pid 25] [remote 10.166.22.139:41374]     self._get_server_information()
[Wed Apr 17 00:30:00.500613 2024] [wsgi:error] [pid 25] [remote 10.166.22.139:41374]   File "/usr/lib/python3.6/site-packages/pymysql/connections.py", line 969, in _get_server_information
[Wed Apr 17 00:30:00.500615 2024] [wsgi:error] [pid 25] [remote 10.166.22.139:41374]     packet = self._read_packet()
[Wed Apr 17 00:30:00.500618 2024] [wsgi:error] [pid 25] [remote 10.166.22.139:41374]   File "/usr/lib/python3.6/site-packages/pymysql/connections.py", line 660, in _read_packet
[Wed Apr 17 00:30:00.500620 2024] [wsgi:error] [pid 25] [remote 10.166.22.139:41374]     % (packet_number, self._next_seq_id))
[Wed Apr 17 00:30:00.500630 2024] [wsgi:error] [pid 25] [remote 10.166.22.139:41374] oslo_db.exception.DBConnectionError: (pymysql.err.InternalError) Packet sequence number wrong - got 1 expected 0
[Wed Apr 17 00:30:00.500633 2024] [wsgi:error] [pid 25] [remote 10.166.22.139:41374] (Background on this error at: http://sqlalche.me/e/2j85)

• We get the following when attempting to list the VMs:

(undercloud) (PROD)stack@director:~$ openstack server list --debug
START with options: server list --debug
options: Namespace(access_token='***', access_token_endpoint='', access_token_type='', aodh_endpoint='', application_credential_id='', application_credential_name='', application_credential_secret='***', auth_methods='', auth_type='password', auth_url='https://10.10.10.141:13000', cacert='/etc/pki/ca-trust/source/anchors/cm-local-ca.pem', cert='', client_id='', client_secret='***', cloud='', code='', debug=True, default_domain='default', default_domain_id='', default_domain_name='', deferred_help=False, discovery_endpoint='', domain_id='', domain_name='', endpoint='', identity_provider='', identity_provider_url='', insecure=None, inspector_api_version='1', inspector_url=None, interface='public', key='', log_file=None, openid_scope='', os_alarming_api_version='2', os_baremetal_api_version='1.58', os_beta_command=False, os_compute_api_version='', os_container_infra_api_version='1', os_data_processing_api_version='1.1', os_data_processing_url='', os_database_api_version='1', os_dns_api_version='2', os_event_api_version='2', os_identity_api_version='3', os_image_api_version='', os_key_manager_api_version='1', os_loadbalancer_api_version='2.0', os_metrics_api_version='1', os_network_api_version='', os_object_api_version='', os_orchestration_api_version='1', os_placement_api_version='1.0', os_project_id=None, os_project_name=None, os_queues_api_version='2', os_tripleoclient_api_version='1', os_volume_api_version='', os_workflow_api_version='2', passcode='', password='***', profile='', project_domain_id='', project_domain_name='Default', project_id='', project_name='admin', protocol='', redirect_uri='', region_name='', remote_project_domain_id='', remote_project_domain_name='', remote_project_id='', remote_project_name='', roles='', service_provider='', service_provider_endpoint='', service_provider_entity_id='', system_scope='', timing=False, token='***', trust_id='', user='', user_domain_id='', user_domain_name='Default', user_id='', username='admin', verbose_level=3, verify=None)
Auth plugin password selected
auth_config_hook(): {'api_timeout': None, 'verify': True, 'cacert': '/etc/pki/ca-trust/source/anchors/cm-local-ca.pem', 'cert': None, 'key': None, 'baremetal_status_code_retries': '5', 'baremetal_introspection_status_code_retries': '5', 'image_status_code_retries': '5', 'disable_vendor_agent': {}, 'interface': 'public', 'floating_ip_source': 'neutron', 'image_api_use_tasks': False, 'image_format': 'qcow2', 'message': '', 'network_api_version': '2', 'object_store_api_version': '1', 'secgroup_source': 'neutron', 'status': 'active', 'auth': {'user_domain_name': 'Default', 'project_domain_name': 'Default', 'project_name': 'admin'}, 'verbose_level': 3, 'deferred_help': False, 'debug': True, 'default_domain': 'default', 'timing': False, 'inspector_api_version': '1', 'auth_url': 'https://10.10.10.141:13000', 'username': 'admin', 'password': '***', 'beta_command': False, 'identity_api_version': '3', 'queues_api_version': '2', 'database_api_version': '1', 'tripleoclient_api_version': '1', 'data_processing_api_version': '1.1', 'loadbalancer_api_version': '2.0', 'workflow_api_version': '2', 'container_infra_api_version': '1', 'baremetal_api_version': '1.58', 'orchestration_api_version': '1', 'dns_api_version': '2', 'key_manager_api_version': '1', 'event_api_version': '2', 'placement_api_version': '1.0', 'metrics_api_version': '1', 'alarming_api_version': '2', 'region_name': '', 'auth_type': 'password', 'networks': []}
defaults: {'api_timeout': None, 'verify': True, 'cacert': None, 'cert': None, 'key': None, 'auth_type': 'password', 'baremetal_status_code_retries': 5, 'baremetal_introspection_status_code_retries': 5, 'image_status_code_retries': 5, 'disable_vendor_agent': {}, 'interface': None, 'floating_ip_source': 'neutron', 'image_api_use_tasks': False, 'image_format': 'qcow2', 'message': '', 'network_api_version': '2', 'object_store_api_version': '1', 'secgroup_source': 'neutron', 'status': 'active'}
cloud cfg: {'api_timeout': None, 'verify': True, 'cacert': '/etc/pki/ca-trust/source/anchors/cm-local-ca.pem', 'cert': None, 'key': None, 'baremetal_status_code_retries': '5', 'baremetal_introspection_status_code_retries': '5', 'image_status_code_retries': '5', 'disable_vendor_agent': {}, 'interface': 'public', 'floating_ip_source': 'neutron', 'image_api_use_tasks': False, 'image_format': 'qcow2', 'message': '', 'network_api_version': '2', 'object_store_api_version': '1', 'secgroup_source': 'neutron', 'status': 'active', 'auth': {'user_domain_name': 'Default', 'project_domain_name': 'Default', 'project_name': 'admin'}, 'verbose_level': 3, 'deferred_help': False, 'debug': True, 'default_domain': 'default', 'timing': False, 'inspector_api_version': '1', 'auth_url': 'https://10.10.10.141:13000', 'username': 'admin', 'password': '***', 'beta_command': False, 'identity_api_version': '3', 'queues_api_version': '2', 'database_api_version': '1', 'tripleoclient_api_version': '1', 'data_processing_api_version': '1.1', 'loadbalancer_api_version': '2.0', 'workflow_api_version': '2', 'container_infra_api_version': '1', 'baremetal_api_version': '1.58', 'orchestration_api_version': '1', 'dns_api_version': '2', 'key_manager_api_version': '1', 'event_api_version': '2', 'placement_api_version': '1.0', 'metrics_api_version': '1', 'alarming_api_version': '2', 'region_name': '', 'auth_type': 'password', 'networks': []}
compute API version 2.1, cmd group openstack.compute.v2
identity API version 3, cmd group openstack.identity.v3
image API version 2, cmd group openstack.image.v2
network API version 2, cmd group openstack.network.v2
object_store API version 1, cmd group openstack.object_store.v1
volume API version 3, cmd group openstack.volume.v3
messaging API version 2, cmd group openstack.messaging.v2
database API version 1, cmd group openstack.database.v1
tripleoclient API version 1, cmd group openstack.tripleoclient.v1
data_processing API version 1.1, cmd group openstack.data_processing.v1
load_balancer API version 2.0, cmd group openstack.load_balancer.v2
neutronclient API version 2, cmd group openstack.neutronclient.v2
workflow_engine API version 2, cmd group openstack.workflow_engine.v2
container_infra API version 1, cmd group openstack.container_infra.v1
baremetal API version 1.58, cmd group openstack.baremetal.v1
baremetal_introspection API version 1, cmd group openstack.baremetal_introspection.v1
orchestration API version 1, cmd group openstack.orchestration.v1
dns API version 2, cmd group openstack.dns.v2
key_manager API version 1, cmd group openstack.key_manager.v1
event API version 2, cmd group openstack.event.v2
placement API version 1.0, cmd group openstack.placement.v1
metric API version 1, cmd group openstack.metric.v1
alarming API version 2, cmd group openstack.alarming.v2
Auth plugin password selected
auth_config_hook(): {'api_timeout': None, 'verify': True, 'cacert': '/etc/pki/ca-trust/source/anchors/cm-local-ca.pem', 'cert': None, 'key': None, 'baremetal_status_code_retries': '5', 'baremetal_introspection_status_code_retries': '5', 'image_status_code_retries': '5', 'disable_vendor_agent': {}, 'interface': 'public', 'floating_ip_source': 'neutron', 'image_api_use_tasks': False, 'image_format': 'qcow2', 'message': '', 'network_api_version': '2', 'object_store_api_version': '1', 'secgroup_source': 'neutron', 'status': 'active', 'auth': {'user_domain_name': 'Default', 'project_domain_name': 'Default', 'project_name': 'admin'}, 'verbose_level': 3, 'deferred_help': False, 'debug': True, 'default_domain': 'default', 'timing': False, 'inspector_api_version': '1', 'auth_url': 'https://10.10.10.141:13000', 'username': 'admin', 'password': '***', 'beta_command': False, 'identity_api_version': '3', 'queues_api_version': '2', 'database_api_version': '1', 'tripleoclient_api_version': '1', 'data_processing_api_version': '1.1', 'loadbalancer_api_version': '2.0', 'workflow_api_version': '2', 'container_infra_api_version': '1', 'baremetal_api_version': '1.58', 'orchestration_api_version': '1', 'dns_api_version': '2', 'key_manager_api_version': '1', 'event_api_version': '2', 'placement_api_version': '1.0', 'metrics_api_version': '1', 'alarming_api_version': '2', 'region_name': '', 'auth_type': 'password', 'networks': []}
command: server list -> openstackclient.compute.v2.server.ListServer (auth=True)
Auth plugin password selected
auth_config_hook(): {'api_timeout': None, 'verify': True, 'cacert': '/etc/pki/ca-trust/source/anchors/cm-local-ca.pem', 'cert': None, 'key': None, 'baremetal_status_code_retries': '5', 'baremetal_introspection_status_code_retries': '5', 'image_status_code_retries': '5', 'disable_vendor_agent': {}, 'interface': 'public', 'floating_ip_source': 'neutron', 'image_api_use_tasks': False, 'image_format': 'qcow2', 'message': '', 'network_api_version': '2', 'object_store_api_version': '1', 'secgroup_source': 'neutron', 'status': 'active', 'auth': {'user_domain_name': 'Default', 'project_domain_name': 'Default', 'project_name': 'admin'}, 'additional_user_agent': [('osc-lib', '1.14.1')], 'verbose_level': 3, 'deferred_help': False, 'debug': True, 'default_domain': 'default', 'timing': False, 'inspector_api_version': '1', 'auth_url': 'https://10.10.10.141:13000', 'username': 'admin', 'password': '***', 'beta_command': False, 'identity_api_version': '3', 'queues_api_version': '2', 'database_api_version': '1', 'tripleoclient_api_version': '1', 'data_processing_api_version': '1.1', 'loadbalancer_api_version': '2.0', 'workflow_api_version': '2', 'container_infra_api_version': '1', 'baremetal_api_version': '1.58', 'orchestration_api_version': '1', 'dns_api_version': '2', 'key_manager_api_version': '1', 'event_api_version': '2', 'placement_api_version': '1.0', 'metrics_api_version': '1', 'alarming_api_version': '2', 'region_name': '', 'auth_type': 'password', 'networks': []}
Using auth plugin: password
Using parameters {'auth_url': 'https://10.10.10.141:13000', 'project_name': 'admin', 'project_domain_name': 'Default', 'username': 'admin', 'user_domain_name': 'Default', 'password': '***'}
Get auth_ref
REQ: curl -g -i --cacert "/etc/pki/ca-trust/source/anchors/cm-local-ca.pem" -X GET https://10.10.10.141:13000 -H "Accept: application/json" -H "User-Agent: openstacksdk/0.36.5 keystoneauth1/3.17.4 python-requests/2.20.0 CPython/3.6.8"
Starting new HTTPS connection (1): 10.10.10.141:13000
https://10.10.10.141:13000 "GET / HTTP/1.1" 300 270
RESP: [300] Content-Length: 270 Content-Type: application/json Date: Tue, 16 Apr 2024 18:42:25 GMT Location: https://10.10.10.141:13000/v3/ Server: Apache Vary: X-Auth-Token x-openstack-request-id: req-fbf17b82-a9cd-4358-a401-f9ce4627adb4
RESP BODY: {"versions": {"values": [{"id": "v3.13", "status": "stable", "updated": "2019-07-19T00:00:00Z", "links": [{"rel": "self", "href": "https://10.10.10.141:13000/v3/"}], "media-types": [{"base": "application/json", "type": "application/vnd.openstack.identity-v3+json"}]}]}}
GET call to https://10.10.10.141:13000/ used request id req-fbf17b82-a9cd-4358-a401-f9ce4627adb4
Making authentication request to https://10.10.10.141:13000/v3/auth/tokens
https://10.10.10.141:13000 "POST /v3/auth/tokens HTTP/1.1" 500 531
Request returned failure status: 500
Internal Server Error (HTTP 500)
Traceback (most recent call last):
  File "/usr/lib/python3.6/site-packages/cliff/app.py", line 394, in run_subcommand
    self.prepare_to_run_command(cmd)
  File "/usr/lib/python3.6/site-packages/openstackclient/shell.py", line 166, in prepare_to_run_command
    return super(OpenStackShell, self).prepare_to_run_command(cmd)
  File "/usr/lib/python3.6/site-packages/osc_lib/shell.py", line 493, in prepare_to_run_command
    self.client_manager.auth_ref
  File "/usr/lib/python3.6/site-packages/osc_lib/clientmanager.py", line 202, in auth_ref
    self._auth_ref = self.auth.get_auth_ref(self.session)
  File "/usr/lib/python3.6/site-packages/keystoneauth1/identity/generic/base.py", line 208, in get_auth_ref
    return self._plugin.get_auth_ref(session, **kwargs)
  File "/usr/lib/python3.6/site-packages/keystoneauth1/identity/v3/base.py", line 188, in get_auth_ref
    authenticated=False, log=False, **rkwargs)
  File "/usr/lib/python3.6/site-packages/keystoneauth1/session.py", line 1124, in post
    return self.request(url, 'POST', **kwargs)
  File "/usr/lib/python3.6/site-packages/keystoneauth1/session.py", line 961, in request
    raise exceptions.from_response(resp, method, url)
keystoneauth1.exceptions.http.InternalServerError: Internal Server Error (HTTP 500)
clean_up ListServer: Internal Server Error (HTTP 500)
Traceback (most recent call last):
  File "/usr/lib/python3.6/site-packages/osc_lib/shell.py", line 136, in run
    ret_val = super(OpenStackShell, self).run(argv)
  File "/usr/lib/python3.6/site-packages/cliff/app.py", line 281, in run
    result = self.run_subcommand(remainder)
  File "/usr/lib/python3.6/site-packages/osc_lib/shell.py", line 176, in run_subcommand
    ret_value = super(OpenStackShell, self).run_subcommand(argv)
  File "/usr/lib/python3.6/site-packages/cliff/app.py", line 394, in run_subcommand
    self.prepare_to_run_command(cmd)
  File "/usr/lib/python3.6/site-packages/openstackclient/shell.py", line 166, in prepare_to_run_command
    return super(OpenStackShell, self).prepare_to_run_command(cmd)
  File "/usr/lib/python3.6/site-packages/osc_lib/shell.py", line 493, in prepare_to_run_command
    self.client_manager.auth_ref
  File "/usr/lib/python3.6/site-packages/osc_lib/clientmanager.py", line 202, in auth_ref
    self._auth_ref = self.auth.get_auth_ref(self.session)
  File "/usr/lib/python3.6/site-packages/keystoneauth1/identity/generic/base.py", line 208, in get_auth_ref
    return self._plugin.get_auth_ref(session, **kwargs)
  File "/usr/lib/python3.6/site-packages/keystoneauth1/identity/v3/base.py", line 188, in get_auth_ref
    authenticated=False, log=False, **rkwargs)
  File "/usr/lib/python3.6/site-packages/keystoneauth1/session.py", line 1124, in post
    return self.request(url, 'POST', **kwargs)
  File "/usr/lib/python3.6/site-packages/keystoneauth1/session.py", line 961, in request
    raise exceptions.from_response(resp, method, url)
keystoneauth1.exceptions.http.InternalServerError: Internal Server Error (HTTP 500)

END return value: 1
(undercloud) (PROD)stack@director:~$ openstack stack list --debug
START with options: stack list --debug
options: Namespace(access_token='***', access_token_endpoint='', access_token_type='', aodh_endpoint='', application_credential_id='', application_credential_name='', application_credential_secret='***', auth_methods='', auth_type='password', auth_url='https://10.10.10.141:13000', cacert='/etc/pki/ca-trust/source/anchors/cm-local-ca.pem', cert='', client_id='', client_secret='***', cloud='', code='', debug=True, default_domain='default', default_domain_id='', default_domain_name='', deferred_help=False, discovery_endpoint='', domain_id='', domain_name='', endpoint='', identity_provider='', identity_provider_url='', insecure=None, inspector_api_version='1', inspector_url=None, interface='public', key='', log_file=None, openid_scope='', os_alarming_api_version='2', os_baremetal_api_version='1.58', os_beta_command=False, os_compute_api_version='', os_container_infra_api_version='1', os_data_processing_api_version='1.1', os_data_processing_url='', os_database_api_version='1', os_dns_api_version='2', os_event_api_version='2', os_identity_api_version='3', os_image_api_version='', os_key_manager_api_version='1', os_loadbalancer_api_version='2.0', os_metrics_api_version='1', os_network_api_version='', os_object_api_version='', os_orchestration_api_version='1', os_placement_api_version='1.0', os_project_id=None, os_project_name=None, os_queues_api_version='2', os_tripleoclient_api_version='1', os_volume_api_version='', os_workflow_api_version='2', passcode='', password='***', profile='', project_domain_id='', project_domain_name='Default', project_id='', project_name='admin', protocol='', redirect_uri='', region_name='', remote_project_domain_id='', remote_project_domain_name='', remote_project_id='', remote_project_name='', roles='', service_provider='', service_provider_endpoint='', service_provider_entity_id='', system_scope='', timing=False, token='***', trust_id='', user='', user_domain_id='', user_domain_name='Default', user_id='', username='admin', verbose_level=3, verify=None)
Auth plugin password selected
auth_config_hook(): {'api_timeout': None, 'verify': True, 'cacert': '/etc/pki/ca-trust/source/anchors/cm-local-ca.pem', 'cert': None, 'key': None, 'baremetal_status_code_retries': '5', 'baremetal_introspection_status_code_retries': '5', 'image_status_code_retries': '5', 'disable_vendor_agent': {}, 'interface': 'public', 'floating_ip_source': 'neutron', 'image_api_use_tasks': False, 'image_format': 'qcow2', 'message': '', 'network_api_version': '2', 'object_store_api_version': '1', 'secgroup_source': 'neutron', 'status': 'active', 'auth': {'user_domain_name': 'Default', 'project_domain_name': 'Default', 'project_name': 'admin'}, 'verbose_level': 3, 'deferred_help': False, 'debug': True, 'default_domain': 'default', 'timing': False, 'inspector_api_version': '1', 'auth_url': 'https://10.10.10.141:13000', 'username': 'admin', 'password': '***', 'beta_command': False, 'identity_api_version': '3', 'queues_api_version': '2', 'database_api_version': '1', 'tripleoclient_api_version': '1', 'data_processing_api_version': '1.1', 'loadbalancer_api_version': '2.0', 'workflow_api_version': '2', 'container_infra_api_version': '1', 'baremetal_api_version': '1.58', 'orchestration_api_version': '1', 'dns_api_version': '2', 'key_manager_api_version': '1', 'event_api_version': '2', 'placement_api_version': '1.0', 'metrics_api_version': '1', 'alarming_api_version': '2', 'region_name': '', 'auth_type': 'password', 'networks': []}
defaults: {'api_timeout': None, 'verify': True, 'cacert': None, 'cert': None, 'key': None, 'auth_type': 'password', 'baremetal_status_code_retries': 5, 'baremetal_introspection_status_code_retries': 5, 'image_status_code_retries': 5, 'disable_vendor_agent': {}, 'interface': None, 'floating_ip_source': 'neutron', 'image_api_use_tasks': False, 'image_format': 'qcow2', 'message': '', 'network_api_version': '2', 'object_store_api_version': '1', 'secgroup_source': 'neutron', 'status': 'active'}
cloud cfg: {'api_timeout': None, 'verify': True, 'cacert': '/etc/pki/ca-trust/source/anchors/cm-local-ca.pem', 'cert': None, 'key': None, 'baremetal_status_code_retries': '5', 'baremetal_introspection_status_code_retries': '5', 'image_status_code_retries': '5', 'disable_vendor_agent': {}, 'interface': 'public', 'floating_ip_source': 'neutron', 'image_api_use_tasks': False, 'image_format': 'qcow2', 'message': '', 'network_api_version': '2', 'object_store_api_version': '1', 'secgroup_source': 'neutron', 'status': 'active', 'auth': {'user_domain_name': 'Default', 'project_domain_name': 'Default', 'project_name': 'admin'}, 'verbose_level': 3, 'deferred_help': False, 'debug': True, 'default_domain': 'default', 'timing': False, 'inspector_api_version': '1', 'auth_url': 'https://10.10.10.141:13000', 'username': 'admin', 'password': '***', 'beta_command': False, 'identity_api_version': '3', 'queues_api_version': '2', 'database_api_version': '1', 'tripleoclient_api_version': '1', 'data_processing_api_version': '1.1', 'loadbalancer_api_version': '2.0', 'workflow_api_version': '2', 'container_infra_api_version': '1', 'baremetal_api_version': '1.58', 'orchestration_api_version': '1', 'dns_api_version': '2', 'key_manager_api_version': '1', 'event_api_version': '2', 'placement_api_version': '1.0', 'metrics_api_version': '1', 'alarming_api_version': '2', 'region_name': '', 'auth_type': 'password', 'networks': []}
compute API version 2.1, cmd group openstack.compute.v2
identity API version 3, cmd group openstack.identity.v3
image API version 2, cmd group openstack.image.v2
network API version 2, cmd group openstack.network.v2
object_store API version 1, cmd group openstack.object_store.v1
volume API version 3, cmd group openstack.volume.v3
messaging API version 2, cmd group openstack.messaging.v2
database API version 1, cmd group openstack.database.v1
tripleoclient API version 1, cmd group openstack.tripleoclient.v1
data_processing API version 1.1, cmd group openstack.data_processing.v1
load_balancer API version 2.0, cmd group openstack.load_balancer.v2
neutronclient API version 2, cmd group openstack.neutronclient.v2
workflow_engine API version 2, cmd group openstack.workflow_engine.v2
container_infra API version 1, cmd group openstack.container_infra.v1
baremetal API version 1.58, cmd group openstack.baremetal.v1
baremetal_introspection API version 1, cmd group openstack.baremetal_introspection.v1
orchestration API version 1, cmd group openstack.orchestration.v1
dns API version 2, cmd group openstack.dns.v2
key_manager API version 1, cmd group openstack.key_manager.v1
event API version 2, cmd group openstack.event.v2
placement API version 1.0, cmd group openstack.placement.v1
metric API version 1, cmd group openstack.metric.v1
alarming API version 2, cmd group openstack.alarming.v2
Auth plugin password selected
auth_config_hook(): {'api_timeout': None, 'verify': True, 'cacert': '/etc/pki/ca-trust/source/anchors/cm-local-ca.pem', 'cert': None, 'key': None, 'baremetal_status_code_retries': '5', 'baremetal_introspection_status_code_retries': '5', 'image_status_code_retries': '5', 'disable_vendor_agent': {}, 'interface': 'public', 'floating_ip_source': 'neutron', 'image_api_use_tasks': False, 'image_format': 'qcow2', 'message': '', 'network_api_version': '2', 'object_store_api_version': '1', 'secgroup_source': 'neutron', 'status': 'active', 'auth': {'user_domain_name': 'Default', 'project_domain_name': 'Default', 'project_name': 'admin'}, 'verbose_level': 3, 'deferred_help': False, 'debug': True, 'default_domain': 'default', 'timing': False, 'inspector_api_version': '1', 'auth_url': 'https://10.10.10.141:13000', 'username': 'admin', 'password': '***', 'beta_command': False, 'identity_api_version': '3', 'queues_api_version': '2', 'database_api_version': '1', 'tripleoclient_api_version': '1', 'data_processing_api_version': '1.1', 'loadbalancer_api_version': '2.0', 'workflow_api_version': '2', 'container_infra_api_version': '1', 'baremetal_api_version': '1.58', 'orchestration_api_version': '1', 'dns_api_version': '2', 'key_manager_api_version': '1', 'event_api_version': '2', 'placement_api_version': '1.0', 'metrics_api_version': '1', 'alarming_api_version': '2', 'region_name': '', 'auth_type': 'password', 'networks': []}
command: stack list -> heatclient.osc.v1.stack.ListStack (auth=True)
Auth plugin password selected
auth_config_hook(): {'api_timeout': None, 'verify': True, 'cacert': '/etc/pki/ca-trust/source/anchors/cm-local-ca.pem', 'cert': None, 'key': None, 'baremetal_status_code_retries': '5', 'baremetal_introspection_status_code_retries': '5', 'image_status_code_retries': '5', 'disable_vendor_agent': {}, 'interface': 'public', 'floating_ip_source': 'neutron', 'image_api_use_tasks': False, 'image_format': 'qcow2', 'message': '', 'network_api_version': '2', 'object_store_api_version': '1', 'secgroup_source': 'neutron', 'status': 'active', 'auth': {'user_domain_name': 'Default', 'project_domain_name': 'Default', 'project_name': 'admin'}, 'additional_user_agent': [('osc-lib', '1.14.1')], 'verbose_level': 3, 'deferred_help': False, 'debug': True, 'default_domain': 'default', 'timing': False, 'inspector_api_version': '1', 'auth_url': 'https://10.10.10.141:13000', 'username': 'admin', 'password': '***', 'beta_command': False, 'identity_api_version': '3', 'queues_api_version': '2', 'database_api_version': '1', 'tripleoclient_api_version': '1', 'data_processing_api_version': '1.1', 'loadbalancer_api_version': '2.0', 'workflow_api_version': '2', 'container_infra_api_version': '1', 'baremetal_api_version': '1.58', 'orchestration_api_version': '1', 'dns_api_version': '2', 'key_manager_api_version': '1', 'event_api_version': '2', 'placement_api_version': '1.0', 'metrics_api_version': '1', 'alarming_api_version': '2', 'region_name': '', 'auth_type': 'password', 'networks': []}
Using auth plugin: password
Using parameters {'auth_url': 'https://10.10.10.141:13000', 'project_name': 'admin', 'project_domain_name': 'Default', 'username': 'admin', 'user_domain_name': 'Default', 'password': '***'}
Get auth_ref
REQ: curl -g -i --cacert "/etc/pki/ca-trust/source/anchors/cm-local-ca.pem" -X GET https://10.10.10.141:13000 -H "Accept: application/json" -H "User-Agent: openstacksdk/0.36.5 keystoneauth1/3.17.4 python-requests/2.20.0 CPython/3.6.8"
Starting new HTTPS connection (1): 10.10.10.141:13000
https://10.10.10.141:13000 "GET / HTTP/1.1" 300 270
RESP: [300] Content-Length: 270 Content-Type: application/json Date: Tue, 16 Apr 2024 18:51:28 GMT Location: https://10.10.10.141:13000/v3/ Server: Apache Vary: X-Auth-Token x-openstack-request-id: req-de39c331-4967-48dc-bf09-dfafa6923048
RESP BODY: {"versions": {"values": [{"id": "v3.13", "status": "stable", "updated": "2019-07-19T00:00:00Z", "links": [{"rel": "self", "href": "https://10.10.10.141:13000/v3/"}], "media-types": [{"base": "application/json", "type": "application/vnd.openstack.identity-v3+json"}]}]}}
GET call to https://10.10.10.141:13000/ used request id req-de39c331-4967-48dc-bf09-dfafa6923048
Making authentication request to https://10.10.10.141:13000/v3/auth/tokens
https://10.10.10.141:13000 "POST /v3/auth/tokens HTTP/1.1" 500 531
Request returned failure status: 500
Internal Server Error (HTTP 500)
Traceback (most recent call last):
  File "/usr/lib/python3.6/site-packages/cliff/app.py", line 394, in run_subcommand
    self.prepare_to_run_command(cmd)
  File "/usr/lib/python3.6/site-packages/openstackclient/shell.py", line 166, in prepare_to_run_command
    return super(OpenStackShell, self).prepare_to_run_command(cmd)
  File "/usr/lib/python3.6/site-packages/osc_lib/shell.py", line 493, in prepare_to_run_command
    self.client_manager.auth_ref
  File "/usr/lib/python3.6/site-packages/osc_lib/clientmanager.py", line 202, in auth_ref
    self._auth_ref = self.auth.get_auth_ref(self.session)
  File "/usr/lib/python3.6/site-packages/keystoneauth1/identity/generic/base.py", line 208, in get_auth_ref
    return self._plugin.get_auth_ref(session, **kwargs)
  File "/usr/lib/python3.6/site-packages/keystoneauth1/identity/v3/base.py", line 188, in get_auth_ref
    authenticated=False, log=False, **rkwargs)
  File "/usr/lib/python3.6/site-packages/keystoneauth1/session.py", line 1124, in post
    return self.request(url, 'POST', **kwargs)
  File "/usr/lib/python3.6/site-packages/keystoneauth1/session.py", line 961, in request
    raise exceptions.from_response(resp, method, url)
keystoneauth1.exceptions.http.InternalServerError: Internal Server Error (HTTP 500)
clean_up ListStack: Internal Server Error (HTTP 500)
Traceback (most recent call last):
  File "/usr/lib/python3.6/site-packages/osc_lib/shell.py", line 136, in run
    ret_val = super(OpenStackShell, self).run(argv)
  File "/usr/lib/python3.6/site-packages/cliff/app.py", line 281, in run
    result = self.run_subcommand(remainder)
  File "/usr/lib/python3.6/site-packages/osc_lib/shell.py", line 176, in run_subcommand
    ret_value = super(OpenStackShell, self).run_subcommand(argv)
  File "/usr/lib/python3.6/site-packages/cliff/app.py", line 394, in run_subcommand
    self.prepare_to_run_command(cmd)
  File "/usr/lib/python3.6/site-packages/openstackclient/shell.py", line 166, in prepare_to_run_command
    return super(OpenStackShell, self).prepare_to_run_command(cmd)
  File "/usr/lib/python3.6/site-packages/osc_lib/shell.py", line 493, in prepare_to_run_command
    self.client_manager.auth_ref
  File "/usr/lib/python3.6/site-packages/osc_lib/clientmanager.py", line 202, in auth_ref
    self._auth_ref = self.auth.get_auth_ref(self.session)
  File "/usr/lib/python3.6/site-packages/keystoneauth1/identity/generic/base.py", line 208, in get_auth_ref
    return self._plugin.get_auth_ref(session, **kwargs)
  File "/usr/lib/python3.6/site-packages/keystoneauth1/identity/v3/base.py", line 188, in get_auth_ref
    authenticated=False, log=False, **rkwargs)
  File "/usr/lib/python3.6/site-packages/keystoneauth1/session.py", line 1124, in post
    return self.request(url, 'POST', **kwargs)
  File "/usr/lib/python3.6/site-packages/keystoneauth1/session.py", line 961, in request
    raise exceptions.from_response(resp, method, url)
keystoneauth1.exceptions.http.InternalServerError: Internal Server Error (HTTP 500)

END return value: 1