Chapter 8. Running Tests and Submitting Logs for Review

8.1. Selecting and Running Tests

There are two types of tests:

  • Automated tests run when selected without user intervention.
  • Interactive tests are labeled as such and require additional user input for completion.

Use the following steps to choose which tests to run:

For certifications that are associated with an entry on the catalog

  1. Click the Certifications tab, then click the name of the Red Hat product on the line that corresponds with your system or component.

  2. The Certification section of the certification entry should be highlighted when the entry loads.

    If it is not,

    1. Click Certification on the left of the page.
    2. Click the Testing tab.
    3. Click the Continue Testing button and skip ahead to Step 3, below, to select tests.

For sandbox certifications

  1. Click the Sandboxes tab, then the name of the Red Hat product on the line that corresponds with your system or component.
  2. Click the Continue Testing button on the Testing tab.

  3. Choose the test(s) you wish to run by selecting the check box next to the test(s) and begin the run by clicking the Run Selected button.

    Note
    • You can run tests in any order and in any combination.
    • If the test is interactive, you will be prompted for additional information (insert or remove a USB3 device, for example) during the test.
    • The tests will run and display their progress on screen.
    • After the run finishes, it will appear in the list of runs and the Continue Testing button will reappear.
    • You can then run additional tests or view the logs from the previous run(s) and submit results.

8.2. View the Test Logs and Submit the Logs for Review

You can see the test runs on the Testing tab of the certification on your LTS. Click on the entries under Run, such as 2017-06-30 12:59:04 to see what tests were performed and whether they passed or failed. Clicking a result will give more detailed information about that run of the test.

To submit results from a run where the certification is associated with the catalog:

  1. Click on the run you wish to submit.

    Optionally, you can use the drop-down boxes under the Save Assignment column to choose which test plan item that test result will satisfy.

  2. Click the Submit Results button at the bottom of the page to send the results from the displayed run to the Red Hat Certification Catalog.

To submit results from a sandbox run:

  1. Click on the run you wish to submit.
  2. In the Action drop-down box, select Download and download the results file.
  3. Go to Red Hat Certification and open the certification.
  4. Go to the Testing tab and select Upload Results File.

To submit results using CLI:

To submit the test logs using Red Hat Certification CLI, run the # rhcert-cli submit command on the SUT.

Type your Red Hat account credentials previously enabled for certification in the Red Hat Catalog Username and Password. The Certification ID is generated when you successfully create a certification request. Type the ID of the certification request in the Certification ID dialog box.

The # rhcert-cli submit command works only if the image has a network that can connect to the Red Hat services. The command submits the latest timestamped test logs on your host/image to Red Hat certification services for review. The test log file is reviewed by Red Hat certification operations team. The certification results are displayed on Red Hat Certification web user interface.

If SUT does not have internet access, save the test logs on the SUT using the # rhcert-cli save --server [hostname/IP address of LTS] command. The rhcert-cli save command can also be implemented on the LTS.

8.3. Red Hat Review of Test Results

After you submit your results, the review team will analyze their contents and award credit for each passing test that is part of the test plan.

As they verify each passing test, the team sets each test plan item to Confirmed on the certification site’s test plan, which you can see under the Results tab on the catalog. This allows you to see at a glance which tests are outstanding and which have been verified as passing.

If any problems are found, the review team will update the certification request with a question, which will automatically be emailed to the person who submitted the cert.

You can see all the discussion, and respond to or ask any questions, on the Dialog tab of the certification.

8.4. Completing Certifications

A certification is complete once all the items on the official test plan have been reviewed and found to have passing results. At this point the certification can be closed and published, or closed and left unpublished.

Supplemental certifications always remain unpublished, and system or component certs can also be closed and left unpublished if the vendor does not want to publicly advertise the certification status or the existence of the system/component (most certifications are closed and published).

The system information and the discussions between the tester and review team will not be visible to the general public in a published cert. All that customers can see when viewing published certs is basic information about the system.