Chapter 5. Testing requirements
The testing requirements include execution of the
redhat-certification-software test suite, including all of the tests described in the following sections. The execution of the test suite happens in a single sequential session orchestrated by the
redhat-certification application. The test execution is to be conducted while the software to be certified is running or the results will be rejected.
This test run provides a certification test result in a single file, which is provided to Red Hat. The file includes results for the following tests:
- rhcert/self check,
- rpm validation tests,
- sosreport, and
- supportability tests.
5.1. Red Hat certification self check (rhcert/selfcheck)
The Red Hat Certification self check test also known as
rhcert/selfcheck confirms that all the software packages required in the certification process are installed and that they have not been altered. This ensures that the test environment is ready for the certification process and that all the installed certification software packages are supportable.
The test environment includes all the packages required in the certification process and the packages have not been modified.
5.2. RPM test
The RPM test is also known as software/rpm. The test validates that a packaged rpm software under test adheres to Red Hat’s best practices for RPM packaging. If the software product under test is packaged as an RPM the subsequent subtests will be executed.
It is not a requirement for Red Hat Enterprise Linux software certification to package as an RPM, nor that the packaged RPM subtests should undergo PASS results.
The software/rpm test includes the following subtests:
5.2.1. RPM provenance
The RPM provenance subtest verifies if RPM packaged software under test and its RPM packaged dependencies have identifiable provenance in accordance with Red Hat’s best practices for RPM packaging.
- Non Red Hat packages are identified as the software product under test or dependencies of the software product under test.
- Files are tracked within the packages
5.2.2. RPM version handling
The RPM version handling subtest verifies if the RPM packaged software product under test and it’s RPM packaged dependencies are versioned in accordance with Red Hat’s best practices for RPM packaging.
- Packages and changes to packages are versioned
5.2.3. RPM dependency tracking
The RPM dependency tracking subtest verifies if the RPM packaged software product under test and its dependencies are tracked with validity in accordance with Red Hat’s best practices for RPM dependency tracking.
- Success Criteria*
- All dependencies are validly tracked.
The Supportability tests, also known as
software/supportable, ensures that the Red Hat Enterprise Linux (RHEL) remains supportable by Red Hat with the software to be certified installed and running.
The software/supportable tests include the following subtests:
5.3.1. Log versions
The Log versions subtest verifies the version of Red Hat Enterprise Linux (RHEL) installed on the image.
The Kernel subtest confirms the kernel that the image is running is from Red Hat, is appropriate and supports the RHEL version that is undergoing certification, and has not been modified. The kernel version may be the original General Availability (GA) version or any subsequent kernel errata released for the RHEL major + minor release.
The subtest also ensures that the kernel is not tainted when running in the environment.
- The running kernel is a Red Hat kernel.
- The running kernel is released by Red Hat for use with RHEL 8.
- The running kernel is not tainted.
5.3.3. Kernel modules
The Kernel modules subtest confirms the loaded kernel modules are from Red Hat, either from the running kernel’s package or a Red Hat Driver Update. The kernel module subtest also ensures the kernel modules do not identify as a Technology Preview and does not identify as having triggered a slow path in the kernel.
- The kernel modules are from Red Hat and supported.
5.3.4. Third-party kernel modules
The use of third-party kernel modules has the potential to introduce risks to the Red Hat kernel that may not be fully ascertained during certification. As a result, when third party kernel modules are required certification aims to establish a mutual understanding of and the ability to convey ownership and responsibility of support as to facilitate resolution of customer issues. Red Hat reserves the right to deny a certification whenever third-party kernel modules are required. Third-party kernel modules, if required, are subject to additional verification included but not limited to the following:
Partner must ensure that they perform the following tasks:
- Agree that you understand and will act according to the policies defined in Red Hat’s production scope of coverage
- Agree that you understand and will act according to the policies defined in Red Hat’s third party support policy
- Provide Red Hat the documentation of kernel modules written for joint customers
- Provide Red Hat the contact information of your application support team and kernel engineering support team
- Declare your ownership and support to the module
Declare that module will not interfere with the Red Hat Enterprise Linux kernel or
- Declare that module is not a hardware driver
Third-party kernel module must:
Show the module name, size, and dependencies in the
Show the module name, filename, license, and description in the
modinfocommand output, aligned with the partner documentation
Show that the module is signed and supported by you in the
Be loaded after the final
Be delivered and packaged in an RPM or other format that is signed by the partner and provides a mechanism to validate both the in-memory and on-disk kernel module
If delivered and packaged in an RPM, then ensure that the RPM:
- Meets the standard RHEL 8 RPM certification requirements
Shows a description that clearly defines the ownership and support of the included modules is the vendor’s responsibility in
Shows the supported Red Hat kernel range for the kernel modules in
rpm -q --requiresoutput
- Show the module name, size, and dependencies in the
5.3.5. Hardware Health
The Hardware Health subtest checks the system’s health by testing if the hardware is supported, meets the requirements, and has any known hardware vulnerabilities. The subtest does the following:
Checks that the Red Hat Enterprise Linux (RHEL) kernel does not identify hardware as unsupported. When the kernel identifies unsupported hardware, it will display an unsupported hardware message in the system logs and/or trigger an unsupported kernel taint. This subtest prevents customers from possible production risks which may arise from running Red Hat products on unsupported configurations and environments.
In hypervisor, partitioning, cloud instances, and other virtual machine situations, the kernel may trigger an unsupported hardware message or taint based on the hardware data presented to RHEL by the virtual machine (VM).
Checks that the system under test (SUT) meets the minimum hardware requirements as follows:
- Checks if the kernel has reported any known hardware vulnerabilities, if those vulnerabilities have mitigations and if those mitigations have resolved the vulnerability. Many mitigations are automatic to ensure that customers do not need to take active steps to resolve vulnerabilities. In some cases this is not possible; where most of these remaining cases require changes to the configuration of the system BIOS/firmware which may not be modifiable by customers in all situations.
- Confirms the system does not have any offline CPUs.
- Confirms if Simultaneous Multithreading (SMT) is available, enabled, and active in the system.
Failing any of these tests will result in a WARN from the test suite and should be verified by the partner to have correct and intended behavior.
- The kernel does not have the UNSUPPORTEDHARDWARE taint bit set.
- The kernel does not report an unsupported hardware system message.
- The kernel should not report any vulnerabilities with mitigations as vulnerable.
- The kernel does not report the logic core to installed memory ratio as out of range.
- The kernel does not report CPUs in an offline state.
The Hypervisor/Partitioning subtest confirms that the host architecture of the test environment is supported by Red Hat Enterprise Linux (RHEL) 8.
- The PASS scenarios of hypervisor/partitioning on Baremetal for RHEL 8 is x86_64, ppc64le, s390x and aarch64.
- The PASS scenarios of hypervisor/partitioning is x86_64 on, RHEL KVM, VMware, RHEV, QEMU, and HyperV.
5.3.7. Filesystem layout
The Filesystem Layout confirms that the type and minimum size of an image follow the guidelines for each RHEL release. This ensures that the image has a reasonable amount of space required to operate effectively, run applications, and install upgrades for customer use.
- RHEL 8: The root file system for RHEL 8.x is 10GB or greater, and the boot file system is 1GB or greater on an xfs or ext formatted partition.
5.3.8. Installed Red Hat rpms
The Installed Red Hat rpms confirms that RPM packages installed on the system from Red Hat are from the Red Hat Enterprise Linux (RHEL) 8 baseOS and application stream (AppStream) modules, and are not modified.
Non-Red Hat packages may be installed if they are necessary to enable the application, where they are documented, and if they DO NOT modify or conflict with Red Hat packages/software. Red Hat performs a detailed review after the test results are submitted to Red Hat to confirm success or failure when non Red Hat packages are installed.
- The installed Red Hat rpm packages are from RHEL 8.
- The installed Red Hat rpm packages are not modified.
- The installed non-Red Hat rpm packages are the application or are necessary to enable the application and are documented for joint customers as such.
- The installed non-Red Hat rpm packages do not conflict with Red Hat provided packages/software available in Red Hat products included in the offering.
5.3.9. Software repositories
Software repositories confirms that relevant Red Hat repositories are configured and GPG keys are imported on the environment under test to avoid potential significant risks from customers being unable to upgrade their Red Hat Enterprise Linux (RHEL) 8 content.
Red Hat provides core software packages/content in Red Hat official software repositories (included with attached subscriptions) which are signed with GPG keys to ensure authenticity of the distributed files. Software provided as part of these repositories is fully supported and reliable for customer production environments.
Non-Red Hat repositories may be configured if they are necessary to enable the application but they must be properly described and approved.
- RHEL 8 repositories - BaseOS and AppStream must be enabled
- GPG keys for RHEL 8 repositories are already imported in the environment
- The valid repositories are Red Hat Update Infrastructure, Red Hat Satellite, and Red Hat Content Delivery Network
- Configured non-Red Hat repositories are described and required for the proper operation or installation of the application to be certified
- Configured non-Red Hat repositories are required for the proper operation of the public cloud when testing is conducted on a Red Hat certified public cloud
5.3.10. Trusted containers
The trusted containers test verifies that any containers installed on the environment under test are known and provided by Red Hat Enterprise Linux (RHEL) 8. It is expected that customers maintain their ability to utilize containers on RHEL 8 in the presence of certified applications. You will be required to clarify the presence of any non RHEL 8 containers.
- The RHEL 8 container tool set is installed and operational
- Any containers present in the environment are supplied as part of a RHEL 8 subscription
The default RHEL 8 container registry
This program only certifies non-containerized applications for RHEL 8.
Red Hat Insights gives customers the ability to predict and prevent problems before they occur through ongoing, in-depth analysis of their infrastructure. It is expected that customers maintainer their ability to utilize Red Hat Insights in the presence of certified applications. The Insights subtest verifies the insights-client rpm is installed and operational.
- The insights-client package is installed and operational.
5.3.12. RPM freshness
RPM freshness confirms that all important and critical security errata released against Red Hat packages are installed. Red Hat encourages partners to update their test environments whenever an errata is released. This test prompts partners to clarify, displays a status (REVIEW) status, and will require detailed review at Red Hat to confirm success or failure if important and critical security errata packages are not installed.
- All important and critical security errata released for installed Red Hat packages are current.
5.3.13. SELinux enforcing
Security-Enhanced Linux (SELinux) enforcing subtest confirms that SELinux is enabled and running in enforcing mode on the image. SELinux adds Mandatory Access Control (MAC) to the Linux kernel, and is enabled by default in Red Hat Enterprise Linux (RHEL). SELinux policy is administratively-defined and enforced system-wide. SELinux reduces vulnerability to privilege escalation attacks and limits the damage made during configuration. If a process becomes compromised, the attacker only has access to the normal functions of that process and the files that process has been configured to have access to.
- SELinux is configured and running in enforcing mode on the image.
5.3.14. Software modules
Red Hat Enterprise Linux (RHEL) 8 modularity feature is a collection of package available on the system. The software modules test validates modules available on RHEL 8 system.
The test fails if there are non-Red Hat software modules.
5.4. System report (sosreport)
The sosreport test, also known as
software/sosreport, captures a basic sosreport.
Red Hat uses a tool called sos to collect the configuration and diagnostic information from a RHEL system, and to assist customers in troubleshooting their system and following recommended practices. The system report subtest ensures that the sos tool functions as expected on the environment under test and captures a basic sosreport.
- A basic sosreport can be captured on the environment under test.