Chapter 8. Types of Certification Tests

The certification includes self-check, supportable, director_undercloud and bare metal tests. The following sections explains about the test and their sub-test along with the success criteria that is to be an expected outcome.

8.1. self_check Test

The Red Hat Certification Self Check test also known as rhcert/self_check confirms that all the software packages required in the certification process are installed and that they have not been altered. This ensures that the test environment is ready for the certification process and that all the certification software packages are supportable.

Note

The certification packages must not be modified for certification testing or for any other purpose.

Success Criteria:

The test environment includes all the packages required in the certification process and the packages have not been modified.

8.2. Supportable Test

The supportable test, also known as openstack/supportable, ensure that the test environment is compliant with Red Hat’s support policy. This test is required for all OpenStack software certifications. The test confirms that the test node (an OpenStack deployment-under-test) consists only of components supported by Red Hat (Red Hat OpenStack Platform, Red Hat Enterprise Linux).

An OpenStack deployment-under-test refers to the node where the plugin/application-under-test is installed.

Supportability tests must be run on both the control node and the compute node.

Compute Node Considerations:

  • Update the kernel test section to clarify that the compute node needs to use the GA kernel, or it will exit review. Review will need to account for how did the RHEL cert go.
  • Driver Update Programs (DUPs) are acceptable on the compute node but will cause the test to exit review. Review needs to confirm the DUP that aligns with the one used in the corresponding RHEL certification.

The openstack/supportable tests include the following subtests:

8.2.1. Kernel

Verifies that the kernel utilized by Red Hat Enterprise Linux and included in the deployment-under-test is from Red Hat, is appropriate and supported for the version of Red Hat Enterprise Linux included in the OpenStack deployment-under-test, and has not been modified.

The kernel version may be the original General Availability (GA) version or any subsequent kernel errata released by Red Hat for the Red Hat Enterprise Linux minor release.

Note

For more information on Red Hat Enterprise Linux Life Cycle and Kernel Versions, refer to the Red Hat Enterprise Linux Life Cycle and Red Hat Enterprise Linux Release Dates.

The kernel subtest also ensures that the kernel is not tainted when running in the environment. For more information about kernel tainting, refer to Why is the kernel "tainted" and how are the taint values deciphered?.

Success Criteria:

  • The running kernel is a Red Hat kernel, and is released by Red Hat for use with the RHEL version, and
  • The running kernel is not tainted

8.2.2. Kernel Modules

Confirms that the loaded kernel modules are from Red Hat, either from the running kernel’s package or from a Red Hat Driver Update (see Where can I download Driver Update Program (DUP) disks).

The kernel module subtest also ensures the kernel modules do not identify as Technology Preview when running in the environment (see What does a "Technology Preview" feature mean).

Success Criteria:

The kernel modules are from and supported by Red Hat.

8.2.3. Hardware Health

The Hardware Health subtest checks the system’s health by testing if the hardware is supported, meets the requirements, and has any known hardware vulnerabilities. The subtest does the following:

  • Checks that the Red Hat Enterprise Linux (RHEL) kernel does not identify hardware as unsupported. When the kernel identifies unsupported hardware, it will display an unsupported hardware message in the system logs and/or trigger an unsupported kernel taint. This subtest prevents customers from possible production risks which may arise from running Red Hat products on unsupported configurations and environments.

    In hypervisor, partitioning, cloud instances, and other virtual machine situations, the kernel may trigger an unsupported hardware message or taint based on the hardware data presented to RHEL by the virtual machine (VM).

  • Checks that the system under test (SUT) meets the minimum hardware requirements[1] as follows:

    • RHEL 8: Minimum system RAM should be 1.5GB, per CPU logical core count[2]
    • RHEL 7: Minimum system RAM should be 1GB, per CPU logical core count[3]
  • Checks if the kernel has reported any known hardware vulnerabilities, if those vulnerabilities have mitigations and if those mitigations have resolved the vulnerability. Many mitigations are automatic to ensure that customers do not need to take active steps to resolve vulnerabilities. In some cases this is not possible; where most of these remaining cases require changes to the configuration of the system BIOS/firmware which may not be modifiable by customers in all situations.
  • Confirms the system does not have any offline CPUs.
  • Confirms if Simultaneous Multithreading (SMT) is available, enabled, and active in the system.

Failing any of these tests will result in a WARN from the test suite and should be verified by the partner to have correct and intended behavior.

Success Criteria:

  • The kernel does not have the UNSUPPORTEDHARDWARE taint bit set.
  • The kernel does not report an unsupported hardware system message.
  • The kernel should not report any vulnerabilities with mitigations as vulnerable.
  • The kernel does not report the logic core to installed memory ratio as out of range.
  • The kernel does not report CPUs in an offline state.

8.2.4. Installed RPMs

Confirms that RPM packages installed on the system are from Red Hat and are not modified. This prevents potential significant risks to customer environments and ensures that customers make use of a supported environment.

Note

Non-Red Hat packages may not be installed in the OpenStack deployment-under-test.

Success Criteria:

  • The installed Red Hat provided RPM packages are from Red Hat product(s) available in the offering, and
  • The installed Red Hat RPM packages are not modified.

8.2.5. System Report

Red Hat uses a tool called sos which collects configuration and diagnostics information from a RHEL system to assist customers in troubleshooting a RHEL system and following the recommended practices.

The System Report subtest ensures that the sos tool functions as expected on the image or system and captures a basic sosreport. For more information about sosreport, refer to https://access.redhat.com/solutions/3592.

Success Criteria:

A basic sosreport can be captured on the OpenStack deployment-under-test.

8.2.6. SELinux

Confirms that SELinux is running in enforcing mode on the OpenStack deployment-under test.

Note

Security-Enhanced Linux (SELinux) adds Mandatory Access Control (MAC) to the Linux kernel, and is enabled by default in Red Hat Enterprise Linux.

SELinux policy is administratively-defined, enforced system-wide, and is not set at user discretion, reducing vulnerability to privilege escalation attacks helping limit the damage made by configuration mistakes. If a process becomes compromised, the attacker only has access to the normal functions of that process, and the files that the process has been configured to.

For more information on SELinux in RHEL, refer to the SELinux Users and Administrators Guide.

Success Criteria:

SELinux is configured and running in enforcing mode on the OpenStack deployment-under-test.

8.3. Director_undercloud

The Director_undercloud test also known as openstack/director ensures that the deployment-under-test is originally installed using Red Hat OpenStack Platform Director. This test is required for all OpenStack software certifications.

Red Hat OpenStack Platform Director is the supported toolset for installing and managing a Red Hat OpenStack Platform environment in production. It helps in easy installation of a lean and robust OpenStack cloud. It is specifically targeted for enterprise cloud environments where updates, upgrades, and infrastructure control are critical for underlying OpenStack operations.

For more information on installing Red Hat OpenStack Platform Director, refer the Director Installation and Usage Guide.

Success Criteria:

The deployment-under-test is originally installed using Red Hat OpenStack Platform Director.

8.4. Bare Metal Test

The following sub-tests comprise the bare metal test.

8.4.1. Bare Metal InstackStackrc Validation

Validates the instackenv.json and stackrc files.

Success Criteria:

  • Checks if the instackenv.json and stackrc files exist in the specified location and validate the content of instackenv.json file, and
  • Requires validation check if the file is a valid json file and the specified BMC IPs are reachable.

8.4.2. Bare Metal Driver Validation

Compares the drivers configured on the SUT with the drivers supported by Red Hat. If a driver mismatch occurs the subtest generates a Review State and exits.The drivers supported by Red Hat are part of the test suite

Success Criteria:

  • The specified driver should match with the driver in instackenv.json file, and
  • If the drivers do not match the test will exit with a Review State. In this scenario, the Red Hat certification team will manually check the instackenv.json file and the specified driver to validate if the drivers are supported drivers.

8.4.3. Bare Metal Undercloud Validation

Checks if tests are running from the undercloud node. If the tests are not running from this node, the test fails and you need to rerun the test.

Success Criteria:

Testing undercloud artifacts to check if the test ran from the undercloud node.

Note

The undercloud node is the valid node.

8.4.4. Bare Metal Enrolling Test

Checks if bare metal driver is successfully able to enroll the hardware node using the BMC IP. The enrollment process involves driver to communicate properly with BMC IP. The BMC changes the Power State and Provisioning State of the enrolled nodes to off and available.

The test also checks if the stack overcloud exists and if the nodes are already added. It deletes the stack and nodes if they exist, and then tries to enroll nodes based on the instackenv.json file. The test is failed if any of the stages fail.

Success Criteria:

Enrolled nodes are expected to be in Power and Provisioning state.

8.4.5. Bare Metal Inspecting Test

Once the operator sets the required driver_info fields, IronicInspectingTest allows Bare Metal service to discover required node properties.

Success Criteria:

Node properties should be correctly populated so that BMC can gather hardware details based on the instructions provided by the driver.

8.4.6. Bare Metal Deploying Test

Once the inspection is completed successfully,the Bare Metal Deploying Test will try to nova boot two virtual machines (VMs') by assigning a custom flavor to the nodes. This helps to check if BMCs can provide VMs' with the required boot images, and then try to boot up the instances.

Success Criteria:

Start of VMs' with Active status attached to them.

8.4.7. Bare Metal Redeploying Test

Tries to redeploy nova instances.

Success Criteria:

All the stages covered previously should pass in the redeploy too. The test will try to enroll and inspect the hardware instances, and will deploy the instances based on the enroll and inspect stages.

Revised on 2021-04-23 06:42:19 UTC



[2] For more information about hardware support available in RHEL 7 but removed from RHEL 8, see chapter Hardware Enablement in the Considerations in Adopting Red Hat Enterprise Linux 8 document.
[3] For more information about hardware support available in RHEL 6 but removed from RHEL 7, see chapter Changes to Packages, Functionality, and Support in the RHEL 7 Migration Planning Guide document.