Chapter 5. Using ostestr to Run Tempest

Ostestr is an OpenStack wrapper for the testr test runner.

  1. You can view a list of the available tests: 

    $ ostestr -l
    Note

    Before running tests, you can use tempest cleanup --init-saved-state to capture the current state. This will make cleanup easier. For more information see Chapter 6, Cleaning Tempest Resources.

  2. You can run specific tests using ostestr.

    • To run just one test, use: 

      ostestr --regex '(test_regex)'

    • To run multiple tests: 

      ostestr --regex '(test1|test2|test3)'

5.1. Running Smoke Tests

Smoke testing is a type of preliminary testing which only covers the most important functionality. While they are not comprehensive, running smoke tests can save time if they do identify a problem.

To run the smoke tests: 

$ ostestr --regex '(.*smoke)'

5.2. Running Specific Tests Using Whitelist Files

A whitelist file contains specific tests to include.

  1. Create a whitelist.txt file in your tempest workspace: 

    $ touch whitelist.txt

  2. Append all the tests you want to run to the whitelist.txt file: 

    $ cat whitelist.txt
keystone_tempest_plugin.*
networking_bgpvpn_tempest.tests*

  3. Run those tests using ostestr: 

    $ ostestr -w <path to whitelist tests file>

5.3. Skipping Tests Using Blacklist Files

A blacklist file contains a list of tests to exclude.

  1. Create a skip_test.txt file in your tempest workspace: 

    $ touch skip_test.txt

  2. Append all the tests to skip to the skip_test.txt file. 

    $ cat skip_test.txt
keystone_tempest_plugin.*
networking_bgpvpn_tempest.tests*

  3. Run tests using ostestr: 

    $ ostestr -b <path to skip test file>

  4. You can run specific tests by skipping the listed tests: 

    ostestr --regex '(tempest.api.object_storage) -b <path to skip test file>

5.4. Running Tests in Parallel Concurrently, or Serially

  • Run the tests in series: 

    $ ostestr --serial

  • Run the tests in parallel (this is the default): 

    $ ostestr --parallel

    Specify the number of workers to use when running tests in parallel: 

    $ ostestr --concurrency _<workers>_

    Alternatively: 

    $ ostestr -c _<workers>_

    By default, this is set to the number of CPUs.