An Overview of Red Hat’s Secure Development Lifecycle (SDL) practices

Updated -

How does Red Hat align our SDL practices with widely used industry practices?

Red Hat Product Security works with each of our software engineering teams to promote secure development practices and features which enable us to build high-quality software to meet our customers’ business needs. To do so, we created the Red Hat Secure Software Management Lifecycle (SSML), a Software Development Lifecycle approach that directly aligns with the NIST Secure Software Development Framework (NIST SSDF SP-800-218) as well as OWASP guidance and various ISO standards. That framework articulates/aggregates commonly accepted and desired actions to effectively manage software by establishing a core set of software development practices. Each practice is then further defined by a tiered system of standards demonstrating the degree to which the software is meeting these practices.

The goal of these SDL practices and standards is to reduce the number of vulnerabilities in released software, mitigate the potential impact of undetected or unaddressed vulnerabilities, and address the root causes of vulnerabilities to prevent future recurrences.

Preparing the organization

Successful implementation of SDL practices requires all participants to be aware of and understand the expectations of those practices. Education and training are key elements; but so also are the plans and processes which are designed to support those practices. The Red Hat Product Security team has invested in developing:

  • A mature incident response plan (IRP) - How we respond to vulnerabilities and coordinate resolution among the many teams which play a role in vulnerability management
  • SDL training - Success with institutionalizing practices starts with education and establishing a common understanding
  • Governance and process - These define how we align industry research and market requirements to help facilitate our goals to protect our infrastructure and our code
  • An integrated tooling and data strategy - Open development models present a unique challenge in ensuring that all components are identified, tracked (origin and provenance), registered and approved. Red Hat is actively developing a component registry, our open security flaw database, and the integration of test and scan tools, providing engineering teams data and automation to help meet these high security standards.

Each of these topics scale across our entire portfolio, supporting our engineering teams as they continually bake in higher security standards into our development cycle and mature their secure development processes.

Producing well-secured software

When we talk about the recipe for well-secured software (software with minimal security vulnerabilities in its releases), specifically the code itself, the two critical ingredients are hardening and addressing vulnerabilities. Security hardening, such as two factor authentication or logging, provides customers with controls to tighten the security of production environments. Our goal is to have code that is free of vulnerabilities. This requires vulnerabilities to be identified and prioritized through risk analysis. Red Hat code is subject to a combination of vulnerability and malware scanners within our internal pipeline. These scans are done prior to the independent vulnerability analysis we’ve always done when new vulnerabilities are reported. Security architects from Red Hat Product Security then work with each software engineering team to review features, implement hardening, and respond appropriately to address vulnerabilities and weaknesses. In addition, as part of our SDL process, we conduct threat models, fuzzing, SAST and DAST testing, penetration testing and other heuristic tools. These tests, models, and reviews provide engineering teams with additional data on risks and potential considerations which decrease risk and improve security functionality for an enterprise.

Protecting the software

Not only do we want to produce hardened software, but we also employ practices that reduce the likelihood of unauthorized tampering and maintain the integrity of the software. We go through extensive efforts to make sure the software we produce is protected while it is built, to prevent supply chain attacks from occurring. We employ a stringent set of security practices around the storage and manipulation of our source code, including a process for signing product releases to provide consumers with verification of legitimacy. Finally, all releases are archived for use when we backport future fixes to remove discovered vulnerabilities.

Responding to vulnerabilities

Response to vulnerabilities is often seen as a reactive process. A scan is run, vulnerabilities are identified, and fixes are developed. We believe that responding to vulnerabilities is now, more than ever, a proactive process as well. Red Hat has engaged in and pushed industry improvements to this process for over 20 years. Simply put, we do things differently. We do this better.

  • We monitor weaknesses and flaws before they become known vulnerabilities.
  • Regardless of whether software includes an affected component, our analysis includes a review of environmental aspects of how the component is built, the libraries used, and any default configurations.
  • All vulnerabilities receive a CVE.
  • We provide a severity rating which prioritizes the risk assessment for customers. This is not based on the base flaw metrics only, but based on the additional environmental aspects mentioned above.
  • We publish our vulnerabilities openly using industry standards and formats.

How do we validate our efforts?

The term validation not only represents the practices and process to continually review all SDL practices, but should also define the collective degree or standard by which our software meets SDL practices. We’ve implemented this as a tiered methodology; ie. tier 3 standards are a degree higher than tier 2 standards. Red Hat validates against our SDL practices at multiple stages within the timeline of a traditional secure development lifecycle.

  1. Our team of security architects work closely with each engineering team to review security policies and features during design or planning of that software. NIST SSDF refers to this practice as an “architecture review”.
  2. The Red Hat Product Security team performs a validation of SDL evidence for our software wishing to attest to a standards tier.
  3. This article already references how we respond to vulnerabilities; however, our vulnerability data also serves as a validation check for our software engineering teams. By reporting vulnerability trends with our engineering teams, we help ensure a development focus on eliminating the backlog of noncritical vulnerabilities.

Comments