Chapter 8. Types of Certification Tests
The certification includes self-check, supportable, director_undercloud and bare metal tests. The following sections explains about the test and their sub-test along with the success criteria that is to be an expected outcome.
8.1. self_check Test
The Red Hat Certification Self Check test also known as rhcert/self_check confirms that all the software packages required in the certification process are installed and that they have not been altered. This ensures that the test environment is ready for the certification process and that all the certification software packages are supportable.
The certification packages must not be modified for certification testing or for any other purpose.
Success Criteria:
The test environment includes all the packages required in the certification process and the packages have not been modified.
8.2. Supportable Test
The supportable test, also known as openstack/supportable, ensure that the test environment is compliant with Red Hat’s support policy. This test is required for all OpenStack software certifications. The test confirms that the test node (an OpenStack deployment-under-test) consists only of components supported by Red Hat (Red Hat OpenStack Platform, Red Hat Enterprise Linux).
An OpenStack deployment-under-test refers to the node where the plugin/application-under-test is installed.
Supportability tests must be run on both the control node and the compute node.
Compute Node Considerations:
- Update the kernel test section to clarify that the compute node needs to use the GA kernel, or it will exit review. Review will need to account for how did the RHEL cert go.
- Driver Update Programs(DUPs) are acceptable on the compute node but will cause the test to exit review. Review needs to confirm the DUP that aligns with the one used in the corresponding RHEL certification.
The openstack/supportable tests include the following subtests:
8.2.1. Kernel
Verifies that the kernel utilized by Red Hat Enterprise Linux and included in the deployment-under-test is from Red Hat, is appropriate and supported for the version of Red Hat Enterprise Linux included in the OpenStack deployment-under-test, and has not been modified.
The kernel version may be the original General Availability (GA) version or any subsequent kernel errata released by Red Hat for the Red Hat Enterprise Linux minor release.
For more information on Red Hat Enterprise Linux Life Cycle and Kernel Versions, refer to the Red Hat Enterprise Linux Life Cycle and Red Hat Enterprise Linux Release Dates.
The kernel subtest also ensures that the kernel is not tainted when running in the environment. For more information about kernel tainting, refer to Why is the kernel "tainted" and how are the taint values deciphered?.
Success Criteria:
- The running kernel is a Red Hat kernel, and is released by Red Hat for use with the RHEL version, and
- The running kernel is not tainted
8.2.2. Kernel Modules
Confirms that the loaded kernel modules are from Red Hat, either from the running kernel’s package or from a Red Hat Driver Update (see Where can I download Driver Update Program (DUP) disks).
The kernel module subtest also ensures the kernel modules do not identify as Technology Preview when running in the environment (see What does a "Technology Preview" feature mean).
Success Criteria:
The kernel modules are from and supported by Red Hat.
8.2.3. Unsupported Hardware
Confirms that the Red Hat kernel has not identified unsupported hardware. This prevents customer production risks which arise from running Red Hat products on unsupported configurations and environments. The kernel is aware of hardware that is not considered supportable for a variety of reasons. When the kernel identifies such hardware it will either provide an unsupported hardware message in the system logs or trigger a kernel taint.
Success Criteria:
Tests are run on certified and supported hardware. For a complete list of hardware certified for RHEL 6 or RHEL 7, refer to the Red Hat Ecosystem Catalog.
8.2.4. Installed RPMs
Confirms that RPM packages installed on the system are from Red Hat and are not modified. This prevents potential significant risks to customer environments and ensures that customers make use of a supported environment.
Non-Red Hat packages may not be installed in the OpenStack deployment-under-test.
Success Criteria:
- The installed Red Hat provided RPM packages are from Red Hat product(s) available in the offering, and
- The installed Red Hat RPM packages are not modified.
8.2.5. System Report
Red Hat uses a tool called sos which collects configuration and diagnostics information from a RHEL system to assist customers in troubleshooting a RHEL system and following the recommended practices.
The System Report subtest ensures that the sos tool functions as expected on the image or system and captures a basic sosreport. For more information about sosreport, refer to https://access.redhat.com/solutions/3592.
Success Criteria:
A basic sosreport can be captured on the OpenStack deployment-under-test.
8.2.6. SELinux
Confirms that SELinux is running in enforcing mode on the OpenStack deployment-under test.
Security-Enhanced Linux (SELinux) adds Mandatory Access Control (MAC) to the Linux kernel, and is enabled by default in Red Hat Enterprise Linux.
SELinux policy is administratively-defined, enforced system-wide, and is not set at user discretion, reducing vulnerability to privilege escalation attacks helping limit the damage made by configuration mistakes. If a process becomes compromised, the attacker only has access to the normal functions of that process, and the files that the process has been configured to.
For more information on SELinux in RHEL, refer to the SELinux Users and Administrators Guide.
Success Criteria:
SELinux is configured and running in enforcing mode on the OpenStack deployment-under-test.
8.3. Director_undercloud
The Director_undercloud test also known as openstack/director ensures that the deployment-under-test is originally installed using Red Hat OpenStack Platform Director. This test is required for all OpenStack software certifications.
Red Hat OpenStack Platform Director is the supported toolset for installing and managing a Red Hat OpenStack Platform environment in production. It helps in easy installation of a lean and robust OpenStack cloud. It is specifically targeted for enterprise cloud environments where updates, upgrades, and infrastructure control are critical for underlying OpenStack operations.
For more information on installing Red Hat OpenStack Platform Director, refer the Director Installation and Usage Guide.
Success Criteria:
The deployment-under-test is originally installed using Red Hat OpenStack Platform Director.
8.4. Bare Metal Test
The following sub-tests comprise the bare metal test.
8.4.1. Bare Metal InstackStackrc Validation
Validates the instackenv.json and stackrc files.
Success Criteria:
-
Checks if the
instackenv.jsonandstackrcfiles exist in the specified location and validate the content ofinstackenv.jsonfile, and - Requires validation check if the file is a valid json file and the specified BMC IPs are reachable.
8.4.2. Bare Metal Driver Validation
Compares the drivers configured on the SUT with the drivers supported by Red Hat. If a driver mismatch occurs the subtest generates a Review State and exits.The drivers supported by Red Hat are part of the test suite
Success Criteria:
-
The specified driver should match with the driver in
instackenv.jsonfile, and -
If the drivers do not match the test will exit with a Review State. In this scenario, the Red Hat certification team will manually check the
instackenv.jsonfile and the specified driver to validate if the drivers are supported drivers.
8.4.3. Bare Metal Undercloud Validation
Checks if tests are running from the undercloud node. If the tests are not running from this node, the test fails and you need to rerun the test.
Success Criteria:
Testing undercloud artifacts to check if the test ran from the undercloud node.
The undercloud node is the valid node.
8.4.4. Bare Metal Enrolling Test
Checks if bare metal driver is successfully able to enroll the hardware node using the BMC IP. The enrollment process involves driver to communicate properly with BMC IP. The BMC changes the Power State and Provisioning State of the enrolled nodes to off and available.
The test also checks if the stack overcloud exists and if the nodes are already added. It deletes the stack and nodes if they exist, and then tries to enroll nodes based on the instackenv.json file. The test is failed if any of the stages fail.
Success Criteria:
Enrolled nodes are expected to be in Power and Provisioning state.
8.4.5. Bare Metal Inspecting Test
Once the operator sets the required driver_info fields, IronicInspectingTest allows Bare Metal service to discover required node properties.
Success Criteria:
Node properties should be correctly populated so that BMC can gather hardware details based on the instructions provided by the driver.
8.4.6. Bare Metal Deploying Test
Once the inspection is completed successfully,the Bare Metal Deploying Test will try to nova boot two virtual machines (VMs') by assigning a custom flavor to the nodes. This helps to check if BMCs can provide VMs' with the required boot images, and then try to boot up the instances.
Success Criteria:
Start of VMs' with Active status attached to them.
8.4.7. Bare Metal Redeploying Test
Tries to redeploy nova instances.
Success Criteria:
All the stages covered previously should pass in the redeploy too. The test will try to enroll and inspect the hardware instances, and will deploy the instances based on the enroll and inspect stages.
Revised on 2018-05-28 08:38:44 EDT
Where did the comment section go?
Red Hat's documentation publication system recently went through an upgrade to enable speedier, more mobile-friendly content. We decided to re-evaluate our commenting platform to ensure that it meets your expectations and serves as an optimal feedback mechanism. During this redesign, we invite your input on providing feedback on Red Hat documentation via the discussion platform.