Chapter 6. Certification Tests

Certification tests must be run on the OpenStack deployment-under-test based on the type of OpenStack application undergoing certification. For more information on certification targets, refer Red Hat OpenStack Certification Policy Guide.

6.1. Running Certification Tests for Products Implementing OpenStack APIs

If the OpenStack application undergoing certification falls under this category (see Red Hat OpenStack Certification Policy Guide for more details), complete the following steps on the test server to run certification tests on the OpenStack deployment-under-test/test client.

Note

This category includes OpenStack plugins/drivers which implement OpenStack APIs for Networking, Block Storage, Data Processing and File Share services.

6.2. Running OpenStack API/Tempest Tests and Supportability Tests

  1. In your test server launch Red Hat certification web UI in a browser using the http://machine-ip link
  2. Type Red Hat account credentials enabled for certification in the Username and Password fields. Click Login .
  3. On the Red Hat Certification Home Page, click the Server settings tab.

    Figure 6.1. Server Setting Tab

    Server Setting Tab
  4. In the Register a System field, type the hostname or IP address of the Overcloud node where the plugin-under-test/driver-under-test is installed and then click Add.

    For more information on registering a system, refer to Registering a System using Redhat-Certification.

  5. Click the existing product entry from Red Hat Certification Home Page and then the relevant certification entry from the Certifications Page.

    Figure 6.2. Available Certification Test

    Available Certification Test

    For RHOSP 14, follow the steps in section Running Multi-Host Test in RHOSP 14

    For RHOSP 13 continue with the following the steps:

  6. In the Testing Page, click Add System.
  7. In the Select Host Page, select the host/hostname of the Overcloud node where the plugin-under-test/driver-under-test is installed and then click Test.

    A certification test plan is created on the Testing Page for plugin-under-test/driver-under-test.

    After the certification test plan run is complete and the test plan is ready, the status column displays a “Finished test run” status and a Continue Testing button.

    Figure 6.3. Continue Testing

    Continue Testing
  8. Click Continue Testing.
  9. In the Testing page, select openstack/supportable, openstack/director, and openstack/tempest_config checkboxes and then select the relevant Feature test checkbox from the openstack/cinder_*, openstack/manila_*, openstack/neutron_* and openstack/sahara options based on the type of OpenStack plugin/driver undergoing certification.
  10. Click Run Selected.

    Figure 6.4. Run Tempest API and Supportable Tests

    Run Tempest API and Supportable Tests

    Certification tests are run on the Overcloud node where the plugin-under-test/driver-under-test is configured. The supportable test run is followed by the relevant API/Tempest test run.

Result

Certification tests are run on the plugin-under-test/driver-under-test. The status of the certification test run is displayed on the Testing Page under the relevant hostname.

Figure 6.5. Test Run Status

Test Run Status

After the test run completes, the test logs from the openstack/supportable, openstack/director, and the API/ Tempest tests are stored in .xml.gz format. . Refer to Section 7.1, “Viewing and Submitting the Test Logs using web UI” to submit the test logs.

6.3. Running Multi-Host Test in RHOSP 14

  1. In the Testing Page, click Select Test Systems.

    Select Test Systems
  2. In the Select Host Page, the Partner decides which host will run the test from the selected roles. From the drop-down list, choose the role for a host as controller or compute.

    Roles Selection
  3. Click Test

    Note

    The test generated for Controller node depends on the features that the Partner plugin supports. Tests for Compute node includes supportable, sosreport, self_check, and director.

  4. Click Start Test Run. The mandatory tests will get executed, however, you can check or uncheck the optional tests. The test run executes at the same time for Controller and Compute nodes.

    Start Test Run
  5. Click on the generated test results, this will take you to the Progress tab. Click on the Submit Result. The Red Hat certification team verifies the test results.

    Note

    Currently, multi-host testing does not support Sandboxing and CLI. Testing can be performed only through WebUI.

6.4. tempest_config Test

This test checks the generation of tempest_config based on the lab environment details. Being an interactive test it asks for the following details:

Important

If the system under test is subscribed to OpenStack product repositories the tempest will be installed. If the product repositories are not installed you will get a failure notification that the tempest cannot be installed.

  • In the keystone auth url field, type the Keystone authentication URL that allows access to service endpoints.

    Figure 6.6. Keystone Authentication URL

    Keystone Authentication URL
  • In the OpenStack admin username field, type the OpenStack username of a user with the admin role.
  • In the OpenStack admin password field, type the corresponding OpenStack password of the user with the admin role.
  • Modify the Edit tempest.conf field if required and then click Submit.

    Figure 6.7. Edit Tempest.conf

    Edit Tempest.conf

    The tempest_config test automatically generates a tempest.conf file during run-time. If you need to alter this configuration, place a full tempest.conf into the /etc/redhat-certification-openstack directory that will override the automatic configuration. The configuration override can assist in addressing tempest issues but it does not change the certification testing requirements to be satisfied by tempest.

Note

While it is not explicitly required that tempest automatically corrects configuration of the testing environment for your product it is suggested that you open a bug with the upstream tempest project, against the RHOSP Tempest component, or the Red Hat Certification Component. The RHOS Tempest component link can be used to file the downstream bugs on bugzilla. Incase of the tempest issue, in the Component field select openstack-tempest from the drop-down list. If there is a bug in the component you can select the related component like, openstack-neutron, openstack-cinder or openstack-manila. This process will help you to avoid the repeated correction of configuration.

6.5. Running Certification Tests for Products Consuming OpenStack APIs

If the OpenStack application undergoing certification falls under this category see Red Hat OpenStack Certification Policy Guide, and complete the steps on the xref:prepare-the-test-server test server to run certification tests on the OpenStack deployment-under-test.

Note

This category includes products which generally facilitate an OpenStack deployment or complement the Cloud Infrastructure with additional functionalities such as configuration, scaling and management. OpenStack management applications, monitoring applications and OpenStack-enabled applications such as virtual network functions fall in this category.

6.6. Running Trusted Container Test

Partners need to perform following steps on Red Hat certification web UI to run the trusted container test:

  1. Select the trusted container test.

    Figure 6.8. Trusted Container test in web UI

    Select Trusted Container Test
  2. Click on the Run Selected button.
  3. As this session is interactive, during execution it will prompt to provide the following information:

    1. Reason for non Red Hat containers found on System-under-test
    2. Select the checkboxes for the containers for which test needs to be performed.

6.7. Running OpenStack Director Test and Supportability Tests

  1. On Red Hat Certification Home Page, click the Server settings tab.

    Figure 6.9. Server Settings

    Server Settings
  2. In the Register a System field, type the hostname or IP address of the Overcloud node where the application-under-test is installed and then click Add.
  3. Click the existing product entry from Red Hat Certification Home Page and then click the relevant certification entry from the Certifications Page.

    Figure 6.10. Available Certification Test

    Available Certification Test

    The Progress Page opens and displays the certification tests available in the certification test suite and the progress of the previous runs (if any).

  4. Click the Testing link to open the Testing Page.
  5. In the Testing Page, click System.
  6. In the Select Host Page, select the host/hostname of the Overcloud node where the application-under-test is configured and then click Test .

    The Testing Page opens and a certification test plan is created for the application-under-test.

    After the certification test plan run is complete and the test plan is ready, the status column displays a "Finished test run" status and a Continue Testing button.

    Figure 6.11. Continue Testing

    Continue Testing
  7. Click Continue Testing.
  8. Select interactive next to the openstack/supportable checkbox and then click Run Selected.

    Figure 6.12. Run Supportable Tests

    Run Supportable Tests

    Result

Certification tests are run on the application-under-test. The status of the certification test run is displayed on the Testing Page under the relevant hostname.

Figure 6.13. Test Run Status

Test Run Status

After the test run completes, the test logs from the openstack/supportable tests are stored in the same log file as for the openstack/director test on the test server. See section Section 7.1, “Viewing and Submitting the Test Logs using web UI” in the guide to submit the test logs.

6.8. Running Certification Tests using Red Hat Certification CLI

To run the certification tests using Red Hat Certification CLI (rhcert-cli), execute the following commands on the System Under Test (SUT):

#rhcert-cli clean
#rhcert-cli plan
#rhcert-cli run --test cinder_volumes
Note

Tests to be run depends on the type of driver/plugin, and features implemented by the driver/plugin.

#rhcert-cli run --test supportable --test tempest_config --test cinder_volumes --test cinder_consistency_groups

The above tests are the mandatory tests for a plugin/driver for cinder that has volumes and consistency_group features implemented.

All component-based tests in openstack are tagged, which means the following commands will run all the tests for cinder, manila or neutron, respectively:

#rhcert-cli run --tag cinder
#rhcert-cli run --tag manila
#rhcert-cli run --tag neutron

After the tests run, the test logs/results are automatically collected in a single .xml.gz file. To save the test results/logs, run the following command on the image-under-test.

# rhcert-cli save --server [hostname/IP address of LTS]
Note

In the above command LTS stands for Linux Test Server.

It is also possible to directly submit the test results/logs for validation without saving them on the image-under-test.