Red Hat Training

A Red Hat training course is available for Red Hat OpenStack Platform

11.7. Tuning the Undercloud

The advice in this section aims to help increase the performance of your Undercloud. Implement the recommendations as necessary.
  • The OpenStack Authentication service (keystone) uses a token-based system for access to other OpenStack services. After a certain period, the database accumulates many unused tokens. It is recommended you create a cronjob to flush the token table in the database. For example, to flush the token table at 4 a.m. each day: 
    0 04 * * * /bin/keystone-manage token_flush
  • Heat stores a copy of all template files in its database's raw_template table each time you run openstack overcloud deploy. The raw_template table retains all past templates and grows in size. To remove unused templates in the raw_templates table, create a daily cronjob that clears unused templates that exist in the database for longer than a day: 
    0 04 * * * /bin/heat-manage purge_deleted -g days 1
  • The openstack-heat-engine and openstack-heat-api services might consume too many resources at times. If so, set max_resources_per_stack=-1 in /etc/heat/heat.conf and restart the heat services: 
    $ sudo systemctl restart openstack-heat-engine openstack-heat-api
  • Sometimes the director might not have enough resources to perform concurrent node provisioning. The default is 10 nodes at the same time. To reduce the number of concurrent nodes, set the max_concurrent_builds parameter in /etc/nova/nova.conf to a value less than 10 and restart the nova services: 
    $ sudo systemctl restart openstack-nova-api openstack-nova-scheduler
  • Edit the /etc/my.cnf.d/server.cnf file. Some recommended values to tune include:
    max_connections
    Number of simultaneous connections to the database. The recommended value is 4096.
    innodb_additional_mem_pool_size
    The size in bytes of a memory pool the database uses to store data dictionary information and other internal data structures. The default is usually 8M and an ideal value is 20M for the Undercloud.
    innodb_buffer_pool_size
    The size in bytes of the buffer pool, the memory area where the database caches table and index data. The default is usually 128M and an ideal value is 1000M for the Undercloud.
    innodb_flush_log_at_trx_commit
    Controls the balance between strict ACID compliance for commit operations, and higher performance that is possible when commit-related I/O operations are rearranged and done in batches. Set to 1.
    innodb_lock_wait_timeout
    The length of time in seconds a database transaction waits for a row lock before giving up. Set to 50.
    innodb_max_purge_lag
    This variable controls how to delay INSERT, UPDATE, and DELETE operations when purge operations are lagging. Set to 10000.
    innodb_thread_concurrency
    The limit of concurrent operating system threads. Ideally, provide at least two threads for each CPU and disk resource. For example, if using a quad-core CPU and a single disk, use 10 threads.
  • Ensure that heat has enough workers to perform an Overcloud creation. Usually, this depends on how many CPUs the Undercloud has. To manually set the number of workers, edit the /etc/heat/heat.conf file, set the num_engine_workers parameter to the number of workers you need (ideally 4), and restart the heat engine: 
    $ sudo systemctl restart openstack-heat-engine