After upgrade from OSP10 to OSP13, the time to list ports has increased

Issue

• After the upgrade from OSP10 to OSP13, the time to list ports has drastically increased, currently a list of ports takes around ~14 minutes to complete:

[root@undercloud osp_files(keystone_admin_v3)]# openstack port list --timing
 +------------------------------------------------------------------------+-------------+
| URL                                                                    |     Seconds |
+------------------------------------------------------------------------+-------------+
| GET http://internal-vip.localdomain:35357/v3                           |    0.011667 |
| POST http://internal-vip.localdomain:35357/v3/auth/tokens              |    0.385285 |
| POST http://internal-vip.localdomain:35357/v3/auth/tokens              |    0.378411 |
| GET http://internal-vip.localdomain:9696/                              |    0.005149 |
| GET http://internal-vip.localdomain:9696/v2.0/ports                    | 1380.582514 |
| Total                                                                  | 1381.363026 |
+------------------------------------------------------------------------+-------------+

• The OSP10 deployment took around ~8 minutes to complete this task.

• This OSP13 deployment has 10K ports, 300 networks.

• We would like to now if this is something expected, having such an increase in listing times afer the upgrade and is it possible to reduce these times ?

• It affects start times of the dhcp agents in production , so it affects the downtime of the control-plane after a restart of the neutron dhcp agent.