Scenario tests simulate a typical end user action workflow to test the integration points between services. The testing framework conducts the configuration, tests the integration between services, and is then removed automatically. Tag the tests with the services that they relate to, to make it clear which client libraries the test uses.

A scenario is based on a use case, for example:

API tests validate the OpenStack API. Tests use the OpenStack Integration Test Suite implementation of the OpenStack API. You can use both valid and invalid JSON to ensure that error responses are valid. You can run tests independently and you do not have to rely on the previous test state.

Edit the undercloud.conf file, located in the home directory of the stack user. Ensure that the enable_tempest parameter is set to true:

enable_tempest = true

If your undercloud is already installed, you can edit the undercloud.conf file and then run the openstack undercloud install command to include the extra configuration in the undercloud:

$ openstack undercloud upgrade

You are now ready to install the tempest packages and plugins, described in Section 3.3, “Installing the OpenStack Integration Test Suite Packages”.

To run the OpenStack Integration Test Suite, you must first install the necessary packages and create a configuration file that informs the Integration Test Suite where to find the various OpenStack services and other testing behaviour switches.

To install the OpenStack Integration Test Suite, the following networks must be available within your Red Hat OpenStack Platform environment:

These networks must be connected through a router.

Create the private network. Specify the following options according to your network deployment:

$ openstack network create <network_name> --share
$ openstack subnet create <subnet_name> --subnet-range <address/prefix> \
  --network <network_name>
$ openstack router create <router_name>
$ openstack router add subnet <router_name> <subnet_name>

Create the public network. Specify the following options according to your network deployment:

$ openstack network create <network_name> --external \
  --provider-network-type flat
$ openstack subnet create <subnet_name> --subnet-range <address/prefix> \
  --gateway <default_gateway> --no-dhcp --network <network_name>
$ openstack router set <router_name> --external-gateway <public_network_name>

You are now ready to install and configure the OpenStack Integration Test Suite within the tempest virtual machine. For more information, see Section 3.3, “Installing the OpenStack Integration Test Suite Packages”.

To inspect the region name of your undercloud, run the following commands:

$ source stackrc
$ openstack region list

To inspect the region name of your overcloud, run the following commands:

$ source overcloudrc
$ openstack region list

The discover-tempest-config command generates the tempest.conf file automatically. However, you must ensure that the tempest.conf file corresponds to the configuration of your environment.

$ openstack extension list [--network] [--compute] [--volume] [--identity]

[service_available]
heat = True

[heat_plugin]
username = demo
password = ***
project_name = demo
admin_username = admin
admin_password = ****
admin_project_name = admin
auth_url = http://10.0.0.110:5000//v3
auth_version = 3
user_domain_id = default
project_domain_id = default
user_domain_name = Default
project_domain_name = Default
region = regionOne
fixed_network_name = demo_project_network
network_for_ssh = public
floating_network_name = nova
instance_type = m1.nano
minimal_instance_type = m1.micro
image_ref = 7faed41e-a56c-4971-bf48-24e4e23e69a5
minimal_image_ref = 7faed41e-a56c-4971-bf48-24e4e23e69a5

$ openstack role add --user demo --project demo member

Verify your current tempest configuration:

# tempest verify-config -o <output>

output is the output file where Tempest writes your updated configuration. This is different from your original configuration file.

The default location for log files is the logs directory within your tempest workspace.

To change this directory, in tempest.conf, under the [DEFAULT] section, set log_dir to the desired directory:

[DEFAULT]
log_dir = <directory>

If you have your own logging configuration file, in tempest.conf, under the [DEFAULT] section, set log_config_append to your file:

[DEFAULT]
log_config_append = <file>

If you set the log_config_append attribute, Tempest ignores all other logging configuration in tempest.conf, including the log_dir attribute.

The OpenStack Integration Test Suite provides stable interfaces to test the API microversions. This section contains information about implementing microversion tests using these interfaces.

First, you must configure options in the tempest.conf configuration file to specify the target microversions. Configure these options to ensure that the supported microversions correspond to the microversions used in the OpenStack cloud. You can specify a range of target microversions to run multiple microversion tests in a single Integration Test Suite operation.

For example, to limit the range of microversions for the compute service, in the [compute] section of your configuration file, assign values to the min_microversion and max_microversion parameters:

[compute]
min_microversion = 2.14
max_microversion = latest

Run the tempest-run command with either the --list-tests or -l options to get a list of available tempest tests:

# tempest run -l

Smoke testing is a type of preliminary testing which only covers the most important functionality. While they are not comprehensive, running smoke tests can save time if they do identify a problem.

# tempest run --smoke

A whitelist file is a file that contains regular expressions to select tests that you want to include. Regular expressions are separated by a newline.

Run the tempest run command with either the --whitelist-file or -w options to use a whitelist file:

# tempest run -w <whitelist_file>

A blacklist file is a file that contains regular expressions to select tests that you want to exclude. Regular expressions are separated by a newline.

Run the tempest run command with either the --blacklist-file or -b options to use a blacklist file:

# tempest run -b <blacklist_file>

Run the tests serially:

# tempest run --serial

Run the tests in parallel. Parallel testing is the default:

# tempest run --parallel

Use the --concurrency or -c option to specify the number of workers to use when running tests in parallel:

# tempest run --concurrency <workers>

By default, the Integration Test Suite uses one worker for each CPU available.

Run specific tests with the --regex regular expression option. The regular expression must be Python regular expression:

# tempest run --regex <regex>

For example, use the following example command to run all tests that have names beginning with tempest.scenario:

# tempest run --regex ^tempest.scenario

Complete the following steps to download and configure your tempest container:

The container generates or retrieves the tempest.conf file and runs tests. You can perform these operations from outside the container:

First, you must initialize the saved state. This creates the file saved_state.json, which prevents the cleanup from deleting objects that need to be kept. Typically you would run cleanup with --init-saved-state prior to a tempest run. If this is not the case, saved_state.json must be edited to remove objects that you want cleanup to delete.

# tempest cleanup --init-saved-state

Run the cleanup:

# tempest cleanup

The tempest cleanup command deletes tempest resources but does not delete projects or the tempest administrator account.

It is recommended to perform a dry run before executing the real cleanup. A dry run lists the files that would be deleted by a cleanup, but does not delete any files. The files are listed in the dry_run.json file. Check the dry_run.json file to ensure that the cleanup does not delete any files that you require for your environment.

# tempest cleanup --dry-run

Run the following command to delete all tempest resources, including projects, but not the tempest administrator account:

# tempest cleanup --delete-tempest-conf-objects