Red Hat Security: Determining your risk

Red Hat continues to be a leader in transparency regarding security problems that are discovered in our software and the steps we take to fix them. We publish data about vulnerabilities on our security metrics page and recently launched an API Service that allows easier (and searchable) access to the same data. This data is important to administrators for understanding what known security problems exist and determining what they should do about it.

Pitfalls of comparing version numbers

Comparing version numbers with Common Vulnerabilities and Exposures (CVE) advisories is a common mistake people, and 3rd party security scanners, make when trying to determine if systems are vulnerable. To truly understand the real risk a system is exposed to by a flaw one must know what the CVE affects, how the vulnerability is fixed, what you have installed, and where it came from in order to be able to properly compare things. The data available from our website can get you started but an understanding of your system is needed to complete the picture.

Suppose someone installs an Apache httpd RPM from a non-Red Hat source and they compare its version with data from one of Red Hat’s CVEs to see if it is affected. Suppose the version of the installed RPM is lower than the one that Red Hat released to fix the CVE. What does that tell you about the vulnerability? Absolutely nothing. The 3rd party httpd RPM may contain an older build of httpd that is unaffected by the CVE (perhaps the flaw was in code introduced at a later date), an older build that is affected by the CVE, a newer build (there’s no guarantee that the 3rd party used the same versioning system as either Apache or Red Hat) that is still affected by the CVE, or a newer version that is not affected by the CVE. Or it may contain an arbitrarily different patchlevel of httpd, containing some patches that Red Hat’s build does not have but excluding others that Red Hat does include. Simply put, there is no useful information that can be gathered from such a simple comparison.

What if a server only has Red Hat software installed, is there something we can say then? Perhaps, but it’s not as simple as most people assume.

First, just because the newest version of a package in a given channel is affected by a CVE – and therefore we release an update to fix it – does not mean that older versions of that package are necessarily affected. Going back and testing for the presence of the vulnerability in all previous versions is not something that is typically done unless there is a good reason. So if you are running an older version and a new package is released that fixes a CVE, that is not a guarantee that you are vulnerable to that CVE. In addition, some CVEs require a combination of packages before the system is vulnerable, eg. kernel-1234 and openssh-5678. Security scanners usually fail to identify such cases and can generate false-positive alerts if only one of the packages is installed or has a lower version.

Second, version comparison is completely untenable when you’re comparing packages from different channels or repositories. A simple example would be comparing a kernel from RHEL 6 with RHEL 7. The kernel you have installed on your RHEL 6 machines is kernel-2.6.32-264.el6. What if Red Hat releases kernel-3.10.0-513.el7 to fix a RHEL 7 CVE? A naive version comparison would lead you to believe that the RHEL 6 kernel is vulnerable, but that may or may not be true. The RHEL 6 kernel may have been patched to fix this vulnerability in 2.6.32-263. Or it may never have been affected in the first place.

In reality no one would compare RHEL 6 packages with RHEL 7 versions. But the same holds true inside a single RHEL version. Red Hat releases software in many different streams: “regular” RHEL and Extended Update Support to name the two most common. You can know for example that kernel-2.6.32-264.el6 is “newer” than kernel-2.6.32-263.el6, but what about kernel-2.6.32-263.4.el6_4 (a hypothetical EUS kernel)? Again a naive version comparison would lead you to believe that the 264.el6 kernel was “newer”, but again it’s impossible to say. The 263.4.el6_4 build may have more patches, fewer, or an arbitrarily different set of patches from the 264.el6 build (technically possible, but highly unlikely in practice).

And the same applies not just to kernels in the major Red Hat product lines, but to all packages in all different channels. Consider a bundled library as part of an add-on product. When presented with a CVE in a bundled library the application developer generally has two options: pull into the application the version of the library that contains the fix (which might have “moved on” and may contain many other changes including new features or API changes, which may require a large revalidation effort at least) or to backport the fix to the old version and branch the version numbers. As soon as you branch the version numbers you have the same problem you do when comparing RHEL with EUS versions. Version comparison is simply not useful when you start backporting fixes to old versions of software in different channels (which Red Hat does constantly for different products). You can only compare versions between software that was intended to end up on the same machine; inside a base channel and child channel combination for example.

Third, even if you account for the above there are still other considerations. CPU architecture can matter. If support for a given architecture was at a developer-preview or beta level when an update was made, then it was probably built from a different source RPM than the “regular” architectures and may have an arbitrarily different version number. Or you might have to consider the RPM’s Epoch. Epoch, while never included in the filename and sometimes not displayed by programs and default settings, is nevertheless the highest priority field (if it exists) in determining which RPM is the “newer” version. Are you sure that the data source and security scanner you are using is considering Epoch? What about the RPM Obsoletes tag?

It’s a complicated problem. Correctly detecting vulnerability against a particular CVE is difficult or perhaps impossible given merely RPM version information. Many 3rd party scanners are unaware of the complexity of the problem, leading to false positives or, worse yet, to false negatives.

Getting the information needed to make a decision

The three data resources that Red Hat provides to help explain vulnerabilities and determine if you are vulnerable are the Common Vulnerability Reporting Framework (CVRF) data, the Common Vulnerability and Exposures (CVE) feed, and the Open Vulnerability and Assessment Language (OVAL) data.

The CVRF data is a representation of the update content itself and is not designed for scanning purposes. CVRF data does help explain the impact of the erratum and how it affects your system.

Likewise the CVE feed helps explain what the vulnerability affects and why you should care. It represents the vulnerability itself but not necessarily how to detect it on your system.

OVAL data is machine-readable data that combines much of what is found in the CVRF and CVE advisories. It provides scanners with the additional information (beyond just version numbers) they need to be able to properly detect vulnerability to a CVE. OpenSCAP is one such OVAL-compliant scanner, and the only scanner that Red Hat currently supports. Currently Red Hat only publishes OVAL data for RHEL, but support for additional products is coming in the future.

And in conclusion…

There are several ways to answer the “am I vulnerable” question. For managing individual systems you can use yum-plugin-security or OpenSCAP. To simultaneously manage multiple systems there are additional subscription options available: Red Hat Insights or Satellite. Insights can detect and warn about a subset of “high priority” issues (and also non-CVE related things like performance-related configurations). Satellite is intended to be a full-service management platform for handling the configuration, provisioning, entitlement, and reporting (including which CVEs are applicable) of Red Hat systems and software.

The security metrics data we provide is very useful for determining “What is the problem”, “How important is it”, and “Should I care”, but to answer the question “Which of my systems are affected” you need something more; something that knows the details of your systems’ channel subscriptions and which version (if any) of the updated packages are applicable to them.