VMs not working on compute

Issue

• The VM on the computes seems to be in active state and we can login into them too but it the VMs are not responsive.

• We see the following error messages in /var/log/messages:

Jun 13 04:35:40 overcloud-compute-0 ovs-vswitchd[2669]: ovs|18827|dpdk|ERR|EAL: memzone_reserve_aligned_thread_unsafe(): No more room in config
Jun 13 04:35:40 overcloud-compute-0 ovs-vswitchd[2669]: ovs|18828|dpdk|ERR|VHOST_CONFIG: Failed to create IOTLB cache pool (iotlb_2669_41_19)
Jun 13 04:35:40 overcloud-compute-0 ovs-vswitchd[2669]: ovs|18839|dpdk|ERR|EAL: memzone_reserve_aligned_thread_unsafe(): No more room in config
Jun 13 04:35:40 overcloud-compute-0 ovs-vswitchd[2669]: ovs|18840|dpdk|ERR|VHOST_CONFIG: Failed to create IOTLB cache pool (iotlb_2669_41_20)
Jun 13 04:35:40 overcloud-compute-0 ovs-vswitchd[2669]: ovs|18849|dpdk|ERR|EAL: memzone_reserve_aligned_thread_unsafe(): No more room in config
Jun 13 04:35:40 overcloud-compute-0 ovs-vswitchd[2669]: ovs|18850|dpdk|ERR|VHOST_CONFIG: Failed to create IOTLB cache pool (iotlb_2669_41_21)
Jun 13 04:35:40 overcloud-compute-0 ovs-vswitchd[2669]: ovs|18861|dpdk|ERR|EAL: memzone_reserve_aligned_thread_unsafe(): No more room in config
Jun 13 04:35:40 overcloud-compute-0 ovs-vswitchd[2669]: ovs|18862|dpdk|ERR|VHOST_CONFIG: Failed to create IOTLB cache pool (iotlb_2669_41_22)
Jun 13 04:35:40 overcloud-compute-0 ovs-vswitchd[2669]: ovs|18871|dpdk|ERR|EAL: memzone_reserve_aligned_thread_unsafe(): No more room in config
Jun 13 04:35:40 overcloud-compute-0 ovs-vswitchd[2669]: ovs|18872|dpdk|ERR|VHOST_CONFIG: Failed to create IOTLB cache pool (iotlb_2669_41_23)
Jun 13 04:35:40 overcloud-compute-0 ovs-vswitchd[2669]: ovs|18883|dpdk|ERR|EAL: memzone_reserve_aligned_thread_unsafe(): No more room in config
Jun 13 04:35:40 overcloud-compute-0 ovs-vswitchd[2669]: ovs|18884|dpdk|ERR|VHOST_CONFIG: Failed to create IOTLB cache pool (iotlb_2669_41_24)
Jun 13 04:35:40 overcloud-compute-0 ovs-vswitchd[2669]: ovs|18893|dpdk|ERR|EAL: memzone_reserve_aligned_thread_unsafe(): No more room in config
Jun 13 04:35:40 overcloud-compute-0 ovs-vswitchd[2669]: ovs|18894|dpdk|ERR|VHOST_CONFIG: Failed to create IOTLB cache pool (iotlb_2669_41_25)

• We reserved 16GB per numa cell for the dpdk-mem-socket:

# ovs-vsctl list Open_vSwitch:other_config
ovs-vsctl_list_Open_vSwitch:other_config        : {dpdk-extra=" -n 4", dpdk-init="true", dpdk-lcore-mask="550000000000055", dpdk-socket-mem="16384,16384", pmd-cpu-mask=faa0000000000faa, pmd-rxq-assign=roundrobin}