The Source of Vulnerabilities, How Red Hat finds out about vulnerabilities.

Red Hat Product Security track lots of data about every vulnerability affecting every Red Hat product. We make all this data available on our Measurement page and from time to time write various blog posts and reports about interesting metrics or trends.

One metric we've not written about since 2009 is the source of the vulnerabilities we fix. We want to answer the question of how did Red Hat Product Security first hear about each vulnerability?

Every vulnerability that affects a Red Hat product is given a master tracking bug in Red Hat bugzilla. This bug contains a whiteboard field with a comma separated list of metadata including the dates we found out about the issue, and the source. You can get a file containing all this information already gathered for every CVE. A few months ago we updated our 'daysofrisk' command line tool to parse the source information allowing anyone to quickly create reports like this one.

So let's take a look at some example views of recent data: every vulnerability fixed in every Red Hat product in the 12 months up to 30th August 2014 (a total of 1012 vulnerabilities).

Firstly a chart just giving the breakdown of how we first found out about each issue:

• CERT: Issues reported to us from a national cert like CERT/CC or CPNI, generally in advance of public disclosure
• Individual: Issues reported to Red Hat Product Security directly by a customer or researcher, generally in advance of public disclosure
• Red Hat: Issues found by Red Hat employees
• Relationship: Issues reported to us by upstream projects, generally in advance of public disclosure
• Peer vendors: Issues reported to us by other OS distributions, through relationships
  or a shared private forum
• Internet: For issues not disclosed in advance we monitor a number of mailing lists and security web pages of upstream projects
• CVE: If we've not found out about an issue any other way, we can catch it from the list of public assigned CVE names from Mitre

Next a breakdown of if we knew about the issue in advance. For the purposes of our reports we count knowing the same day of an issue as not knowing in advance, even though we might have had a few hours notice: There are few interesting observations from this data:

• Red Hat employees find a lot of vulnerabilities. We don't just sit back and wait for others to find flaws for us to fix, we actively look for issues ourselves and these are found by engineering, quality assurance, as well as our security teams. 17% of all the issues we fixed in the year were found by Red Hat employees. The issues we find are shared back in advance where possible to upstream and other peer vendors (generally via the 'distros' shared private forum).
• Relationships matter. When you are fixing vulnerabilities in third party software, having a relationship with the upstream makes a big difference. But
  it's really important to note here that this should never be a one-way street, if an upstream is willing to give Red Hat information about flaws in advance,
  then we need to be willing to add value to that notification by sanity checking the draft advisory, checking the patches, and feeding back the
  results from our quality testing. A recent good example of this is the OpenSSL CCS Injection flaw; our relationship with OpenSSL gave us advance
  notice of the issue and we found a mistake in the advisory as well as a mistake in the patch which otherwise would have caused OpenSSL to have to have
  done a secondary fix after release. Only two of the dozens of companies prenotified about those OpenSSL issues actually added value back to OpenSSL.
• Red Hat can influence the way this metric looks; without a dedicated security team a vendor could just watch what another vendor does and copy them,
  or rely on public feeds such as the list of assigned CVE names from Mitre. We can make the choice to invest to find more issues and build upstream relationships.