Tofinosecurity.com uses cookies for analytics and functionality purposes.
To change your cookie settings or find out more, click here.
If you continue browsing our website or close this banner, you accept these cookies.

The US government’s ICS-Computer Emergency Response Team (ICS-CERT) tracks and publishes Security Advisories for known security vulnerabilities found in industrial products. In the entire decade prior to the discovery of Stuxnet (July 2010), ICS-CERT published 5 security advisories involving 3 vendors.

Compare this to 2011, when there were:

215 publicly disclosed vulnerabilities

104 security advisories

39 vendors involved in those security advisories

By late 2012, the total publically disclosed vulnerabilities topped 569.

And remember that these vulnerabilities are typically disclosed to the world prior to ICS vendors having patches available.

Furthermore, 40% of disclosed vulnerabilities include working attack code. This means that individuals can download exploit tools and run them against a target with little understanding of control systems or the consequences of their actions. And download and attack they do - ICS-CERT reported over 20,000 reports of unauthorized internet access to control systems in the last half of 2012.

Welcome to the Industrial Security Patching Treadmill!

Since security researchers, hackers, and issues have essentially migrated from the IT environment, it’s not surprising that we look to that world for a solution. There, security vulnerabilities are addressed by applying software patches - a constant cycle involving multiple patches over the life of a product. In fact, the typical IT computer needs patching (with a full reboot) at least once per week.

You don’t need to be a SCADA/ICS expert to realize that shutdowns of that frequency just aren’t feasible for critical infrastructure control systems.

So How Many Patches Does a Control System Need?

One might argue that a control system requires fewer patches than an IT system…or that the software footprint is smaller, the code quality better… If so, then maybe the patching cycle could be synchronized with annual maintenance shutdowns. In this scenario, patching could be a workable solution to address software vulnerabilities.

To determine if this was an option, in 2008 I participated in the analysis of a U.S. refinery process control network (PCN). There were 85 computers on the refinery PCN, and a similar number of industrial controllers. Although we could only gather reliable data for 78 of the computers, we determined that they were running 272 distinct processes/applications.

A search of the National Vulnerability Database (NVD) found that 48 of these processes had one or more serious security vulnerabilities. Across the refinery PCN, there were 5,455 publically known vulnerabilities, an average of 70 per machine. An aggressive Windows OS patch program reduced this number by almost 50%, but that still left 2,284 published vulnerabilities remaining. Why? Because the applications involved did not have a means of automated patching.

Latent PLC Vulnerabilities Are Not All Disclosed at Once

And let’s not forget the latent vulnerabilities lurking in the control system. Academic research tells us that most commercial software contains 3 - 10 defects for every thousand lines of code (KLOC), and that 1% to 5% of these result in vulnerabilities. That works out to between 0.03 and 0.5 vulnerabilities per KLOC.

So what does that mean in real life?

Take Windows XP, for example…it contains about 40 million lines of code (40,000 KLOC). As of October 2012, about 1106 moderate or severe vulnerabilities have been listed in the NVD for Windows XP - that’s a Vulnerability/KLOC ratio of about 0.0276.

Whatever your experience using Windows XP, seems it’s on the low end of vulnerabilities and pretty good from a security point of view!

Looking back at the history of Windows XP vulnerabilities, we have to assume that SCADA/ICS vulnerabilities won’t all be disclosed at one time. We’ll likely see a relatively small number of disclosures in the first few years, as researchers begin to investigate the products in the industrial space. Then, after SCADA/ICS products have been exposed to widespread security scrutiny, a virtual avalanche of vulnerabilities may occur, resulting in the need to install control system patches on a weekly basis.

We Can’t Ignore SCADA/ICS Firmware

It’s also obvious that the firmware in PLC and DCS controllers will also have vulnerabilities and will require patching. Controllers typically contain between 1,000 KLOC and 5,000 KLOC of firmware. Based on the analysis used above, this means that each is likely to contain between 30 and 150 vulnerabilities. If the vulnerability disclosure curves are similar to those we’ve seen in the IT sector, we can expect a low number of patches in the immediate future, followed by an epidemic in a few years.

The above analysis clearly indicates that the frequency of patching needed to address future SCADA/ICS vulnerabilities in both controllers and computers is likely to exceed the tolerance of most SCADA/ICS operators for system shutdowns.

Tune in to next week’s blog and learn about the impact of patches, what happens when there are no patches, and why many SCADA/ICS customers simply don’t want to patch...

Do you think that patching is a workable solution for securing SCADA/ICS control systems? Do you have any patching success or horror stories to share? Let me know your thoughts.

Comments

While not an answer to everything (e.g. firmware patches), the use of application whitelisting on PC based SCADA, and PLC engineering tools can extend the periodicity between patches, allowing them to be scheduled with plant outages.

Absolutely - application whitelisting is a very promising compensating control for vulnerabilities on the server or desktop. I will discuss a few others over the next few weeks, including "network whitelisting" for non-PC control assets like PLCs.

The statistic comes from Sean McBride of Critical Intelligence Inc. Sean presented an excellent paper last year at S4 2012 called "Documenting the “Lost Decade:” An Analysis of Publicly-Disclosed ICS-Specific Vulnerabilities since 2001”, and shared his 2013 update with us in February.

Thank you for your answer. However I am greatly confused now (I hope I am getting something wrong!). In the "SCADA safety in numbers" report form Positive Technologies, the numbers are TOTALLY different and are much smaller (like less than 200 vulnerabilities in total by the end of 2012). However, their report was admitted as being accurate by the community (I am speaking about comments in mailing list scadasec@news.infracritical.com). I quickly looked up the differences and realized, that e.g. Honeywell vulnerabilities are not included in the report from Positive Technologies. Seems like finding a reliable source of information is not a trivial task (what is not surprising). But I am surprised that their report differs so much from McBride's.

Just to be clear, Sean McBride informed us that there are 569 SCADA/ICS published vulnerabilities as a cumulative total since 2001. Only 248 new vulnerabilities were actually announced in 2012. So that might be part of the difference.

Another comment on patching. Some industries, like pharmaceutical cannot patch their systems without re-certifying the entire plant, what is a very lengthy and costly procedure (up till 1 year and millions of whatever currency). So, what is their solution? In this case I support the general attitude of Tofino, that Digital Bond is inadequate in their demands and expectations.