When Microsoft, Adobe, and Apple learn of critical flaws in their products, …

Programmable logic controllers like these are used to operate machinery used in refineries and other critical infrastructure. Despite their sensitity, manufacturers often decline to fix software flaws that could allow the devices to be remotely hijacked.

The number of security holes that remain unpatched in software used to control refineries, factories, and other critical infrastructure is growing. It's becoming so common that security researchers have coined the term "forever days" to refer to the unfixed vulnerabilities.

The latest forever day vulnerability was disclosed in robotics software marketed by ABB, a maker of ICS (industrial control systems) for utilities and factories. According to an advisory (PDF) issued last week by the US Cyber Emergency Response Team, the flaw in ABB WebWare Server won't be fixed even though it provides the means to remotely execute malicious code on computers that run the application.

"Because these are legacy products nearing the end of their life cycle, ABB does not intend to patch these vulnerable components," the advisory stated. The notice went on to say that the development of a working exploit would require only a medium skill level on the part of the attacker.

Representatives of ABB didn't respond to requests to comment for this article.

Forever day is a play on "zero day," a phrase used to classify vulnerabilities that come under attack before the responsible manufacturer has issued a patch. Also called iDays, or "infinite days" by some researchers, forever days refer to bugs that never get fixed—even when they're acknowledged by the company that developed the software. In some cases, rather than issuing a patch that plugs the hole, the software maker simply adds advice to user manuals showing how to work around the threat.

"They're just not going to get patched," said Terry McCorkle, an independent security researcher who specializes in ICS devices used to control equipment on factory floors, dams, and in other industrial settings. "The big question is how many of their clients are actually set up to take those advisories and take action upon them?"

Engineers use ABB WebWare Server to control giant robotic arms used in factories that make cars and other goods. A buffer overflow flaw affecting several COM and ActiveX components gives attackers the means to shut down or take control of PCs that run the software by sending them specially manipulated data. Because the flaws reside in ActiveX controls, the attack code could be included and scripted in webpages residing on remote servers.

McCorkle, who along with security researcher Billy Rios discovered and reported the flaw to ABB, said it could be exploited by attackers who want to get their hands on blueprints and other proprietary data stored on an engineer's computer. He said the vulnerability might also be targeted by people who want to sabotage a factory's operations.

Enter GE, Siemens, Schneider

ABB is hardly the only ICS provider to foster forever day bugs in software that connects to critical infrastructure. In November, the US CERT issued an advisory (PDF) warning of cross-site scripting bugs in Proficy Historian. That's a set of software components General Electric sells to help engineers collect and archive product information used in SCADA, or supervisory control and data acquisition, systems.

Even though the bug was remotely exploitable and required only low to moderate skill on the part of the attacker, GE said it wouldn't fix it. The software was considered a "legacy component." Instead, GE issued instructions for uninstalling it.

GE public relations manager Eli Holman defended the decision not to issue an update fixing the bug.

"We advised our customers that the 'Administrative Website' option will be removed as an installation option in the next release of Proficy Historian and that an updated alternative will be re-introduced in a future version of the product," Holman wrote in an e-mail. "In the interim, we recommended that customers uninstall the Web administrator and instead use the fully-supported Historian Administrator thick client."

Another buggy PLC that won't be fixed anytime soon is the Siemens's SIMATIC controller used in plants in the water, wastewater, oil, gas, and chemical industries. According to an advisory issued in September, "Siemens currently has no plans to patch this vulnerability," which stemmed from an overflow that could allow attackers to execute arbitrary code on the targeted human-machine interface system.

Earlier this year, researchers discovered a series of additional vulnerabilities in the same product. Some of them, including default administrator passwords that are easy to recover, remain unfixed. Instead, "Siemens has changed the documentation to encourage users to change the password at first login."

Exception that proves the rule

Not all ICS manufacturers are criticized as being reluctant to patch their wares. After McCorkle alerted Invensys to vulnerabilities in its Wonderware Information Server, the company engineers engaged him in hours of meetings so they could better understand the threat.

"They actually had us go sit in on Webex sessions with them and they'd ask us, 'Is this a good way to fix this bug?' They spent a lot of time trying to fix the problems."

But researchers interviewed for this article said such anecdotes are the exception. More typical, they say, are experiences of Dillon Beresford, who last year uncovered a slew of vulnerabilities in the same Siemens PLCs targeted by the Stuxnet worm. More than eight months later, the former researcher for NSS Labs said many of the weaknesses affecting the S7-300 and S7-1200 remain unaddressed. He told Ars he ultimately curtailed research on the devices after growing tired of waiting for fixes.

"I did my part and I'm waiting for them to do their part in terms of patching the vulnerabilities," he said. Siemens representatives didn't respond to emails requesting comment for this article.

Beresford went on to condemn the practice of allowing known vulnerabilities to remain in software that's often used to manage plants and factories that some say could be targeted in terrorist or cyberwar attacks.

"It just doesn't seem very viable in terms of a defense in depth posture," he added. "It seems like an excuse to me, and a poor one at that, to claim these vulnerabilities can't be patched."

When I first saw "ICS" I thought "Android Ice Cream Sandwich", which left me confused as I really hope that no critical infrastructure is running on Android.

Right, I read it the same way.

A little OT, but remember the old NT 4.0 EULA that talked about Windows having Java, which shouldn't be used in critical infrastructure (like nuclear plants)? Well, since Android is Java-based, is there a EULA somewhere that warns people to not use it in critical situations?

ICS has meant "Industrial Control Systems" for DECADES before Google came along and created Ice Cream Sandwich and the rest of the Internet shortened it to ICS. Just because there are fanboys does not make ignorance OK, and does not make the article a troll. Join the rest of the responsible world that does not live solely in the land of consumer electronics.

ICS has meant "Industrial Control Systems" for DECADES before Google came along and created Ice Cream Sandwich and the rest of the Internet shortened it to ICS. Just because there are fanboys does not make ignorance OK, and does not make the article a troll. Join the rest of the responsible world that does not live solely in the land of consumer electronics.

Maybe but I've been involved in industrial controls for years and seldom see systems referred to by that designation.

Wouldn't these companies be using their own products?At least for Siemens and GE it seems likely they'd be using their own controllers in large scale within the corporation. Pretty amazing that they take security so lightly.

Not that these companies don't have to be the starting point, but what are the odds of patches actually being applied? My impression is that these are systems that are validated and tested to no end before they are installed, and then rarely updated lest they break something. After all, we don't see their customers being too outraged (or not publicly, at any rate).

So while, yes, I am a little uncomfortable with the idea that none of these holes are closed, I suspect the vendors aren't the only ones at fault.

Not that these companies don't have to be the starting point, but what are the odds of patches actually being applied? My impression is that these are systems that are validated and tested to no end before they are installed, and then rarely updated lest they break something. After all, we don't see their customers being too outraged (or not publicly, at any rate).

So while, yes, I am a little uncomfortable with the idea that none of these holes are closed, I suspect the vendors aren't the only ones at fault.

Sorry if I suffer a reading comprehension fail, but since the article details more than one customer contacting the vendor and not getting a good support-reply, who do you blame other then the vendor for that?

Larger companies tend to patch their PLCs, at least occasionally, but a lot of medium and small companies rarely if ever patch their PLCs. That being said, with as much as ABB charges for their hardware, it's ridiculous for them not to patch their bugs. And kudos to Wonderware (Invensys) for patching their software. Yes they charge $4K for per client and $10K for a dev license and they still can't make their zoom work right, but at least they take security seriously.

ICS has meant "Industrial Control Systems" for DECADES before Google came along and created Ice Cream Sandwich and the rest of the Internet shortened it to ICS. Just because there are fanboys does not make ignorance OK, and does not make the article a troll. Join the rest of the responsible world that does not live solely in the land of consumer electronics.

Not that these companies don't have to be the starting point, but what are the odds of patches actually being applied? My impression is that these are systems that are validated and tested to no end before they are installed, and then rarely updated lest they break something. After all, we don't see their customers being too outraged (or not publicly, at any rate).

So while, yes, I am a little uncomfortable with the idea that none of these holes are closed, I suspect the vendors aren't the only ones at fault.

Sorry if I suffer a reading comprehension fail, but since the article details more than one customer contacting the vendor and not getting a good support-reply, who do you blame other then the vendor for that?

The way I read it, they aren't regular customers but security researchers. I'm not sure GE would mind loosing their bussiness.

Sorry if I suffer a reading comprehension fail, but since the article details more than one customer contacting the vendor and not getting a good support-reply, who do you blame other then the vendor for that?

I'm not sure which customers you're referring to - I see a lot of security researchers contacting the vendor and not much happening (except questionable advisories, the implementation of which they also question), but none of the vendors customers. Of course, I'm reading it in slightly piecemeal fashion at work, so I might have missed a paragraph or two in there.

Not that these companies don't have to be the starting point, but what are the odds of patches actually being applied? My impression is that these are systems that are validated and tested to no end before they are installed, and then rarely updated lest they break something. After all, we don't see their customers being too outraged (or not publicly, at any rate).

So while, yes, I am a little uncomfortable with the idea that none of these holes are closed, I suspect the vendors aren't the only ones at fault.

Sorry if I suffer a reading comprehension fail, but since the article details more than one customer contacting the vendor and not getting a good support-reply, who do you blame other then the vendor for that?

The way I read it, they aren't regular customers but security researchers. I'm not sure GE would mind loosing their bussiness.

&

PhoenixEnigma wrote:

BINARYGOD wrote:

Sorry if I suffer a reading comprehension fail, but since the article details more than one customer contacting the vendor and not getting a good support-reply, who do you blame other then the vendor for that?

I'm not sure which customers you're referring to - I see a lot of security researchers contacting the vendor and not much happening (except questionable advisories, the implementation of which they also question), but none of the vendors customers. Of course, I'm reading it in slightly piecemeal fashion at work, so I might have missed a paragraph or two in there.

Oops, I misread and though the last section was talking about a customer, not the same sec researcher...

*goes back to patiently waiting for Legend of Grimlock to be released*

That theses companies take such horrendous time to patch if ever one of these systems is probably the cost and time needed to not just produce the update but also test it for all the hardware... But considering what their customers pay for the stuff they should do it on a daily basis almost...

It's rather terrifying however that these companies get away with it since it could create some major crashes and potentially kill people if a hacker with "evil" purposes gets in to the system...

ICS=industrial control systems, duhhhhhh only the last year (or less) it's meaning been more used as abbreviation for Ice Cream Sandwich

(unnecessary info ABB= Allmänna Svenska Elektriska Aktiebolaget (=Asea) Brown Boveri, couldn't understand why they shortened the name ... know this since I had some boring school work about ABB)

So how many people must die before these people will feel responsible? Thank goodness for class action lawsuits and the right to own guns. I for one will be more than willing to take out a murdering VP if a loved one of mine is injured.

So how many people must die before these people will feel responsible? Thank goodness for class action lawsuits and the right to own guns. I for one will be more than willing to take out a murdering VP if a loved one of mine is injured.

So how many people must die before these people will feel responsible? Thank goodness for class action lawsuits and the right to own guns. I for one will be more than willing to take out a murdering VP if a loved one of mine is injured.

Well, Douglas N. Mitchell, better hope no VP's of industrial systems companies get murdered in the next few years, what with your name attached to such a psychopathic sentiment. Vigilante killings of execs because their company failed to update a security hole which black hat hackers in turn used to "injure" someone? Seriously, Rambo?

All of this begs the question why is this stuff connected at all to the internet? Why not have an air gap between the management stations for these and the "regular" office pc's. I mean someone can still be fooled into inserting a flash drive into one of them, but why make it any easier?

Pretty good article, but the picture makes me laugh. The picture is there 984 controller series with 800 series I/O. This controller series has no Ethernet interfaces. The best this series could offer was a 9600 baud, 19200 on a good day, modbus serial interface. On some models there proprietary high speed interface Modbus Plus which is based on RS485 two wire. Yes if you can get access to the network you can do what ever you want, but the places where these devices run are locked down pretty tight. If you make it in then you already have a reason to be there. When these devices were designed in the mid 80's 10bT ethernet was just finding its legs and thinnet 10b2 had only been on the market since the early 80's.

Now, the issue arises when 3rd party devices that convert Modbus+ to ethernet are added to the system. It still requires specialized software to talk to the devices, but it opens up these controllers to a wider range of attack if not installed correctly.

I am amazed to learn that with all the hoopla over terrorism and cyberwarfar from the last few years that this remains a problem with our private industry.

The manufacturers of ICS PLCs have a product cycle of a few years. The equipment lasts 5-10 years (or longer), in a plant designed and built to last 20-50 years. When the replacement product comes out (without the old bugs, but, of course, with a brand new set of bugs), the older generation loses support. This is the problem. Equipment that's designed to last 5-10 years, in critical infrastructure, should have more than 2-3 years of support from the manufacturer. The problem has nothing to do with Government spending. It's PLC manufacturers wanting to not support their old equipment.

The solution is to have an "air gap" between SCADA systems and the Internet. This is because, no matter how good you make the security on these devices, there will be a hacker who'll find a security hole. The "air gap" solution to this particular problem has been known since the Internet came about, and it's been ignored for just as long.

How many articles NOT relating to Android has ICS been used in lately? Well, one, obviously.

It would be my guess that the industry will just wait to see if someone actually does mess with their ICS before taking it seriously. Why waste the time and money if no one actually exploits the security hole? "Our new systems are hardened against such threats, so why not upgrade?"

I don't have much time on PLCs, but my understanding is that these systems are generally supposed to be airgapped vis-a-vis Internet-facing systems. They shouldn't need to be patched.

<conjecture>I should also add that given the number and diversity of these things that are in place within, say, an automobile assembly plant, patching them is a nontrivial proposition. It isn't just that there are thousands of them, it's that patching and testing them requires downtime. Given the potential cost of that downtime in a large-enterprise manufacturing scenario it's easy to see why customers aren't beating the doors down at Siemens/ABB etc for security patches.

If GM knocked on Rockwell/ABB's door tomorrow and said "we want security patches on all our systems" they would probably get started.</conjecture>

Also, all the 'Ice Cream Sandwich' folks need to take that shit elsewhere.

ICS has meant "Industrial Control Systems" for DECADES before Google came along and created Ice Cream Sandwich and the rest of the Internet shortened it to ICS. Just because there are fanboys does not make ignorance OK, and does not make the article a troll. Join the rest of the responsible world that does not live solely in the land of consumer electronics.

Maybe but I've been involved in industrial controls for years and seldom see systems referred to by that designation.

I was in them several decades ago and indeed they were ICS. Intel Corp actually sold a chassis called the ICS-80. Back then factories hated computers so the solution was to sell ICSs as they were not computers, except for the CPU board, memory boards, I/O boards, etc..

The funny thing about all these journalist commenting on the issues is the lack of understanding the actual problem. The systems are sold in the thousands. They are expensive and have an expected term of life. They get past support time and everybody moves on. If they make it through the warranty period it is considered like an appliance and will run until something else breaks or needs to be upgraded.

These problems occurred in the ICS business when people started using Windows products to access their devices. Using the least secure OS on the planet to allow remote access to a system that has no security built in as a default is ... well, short sighted. If you recall the original scare tactics regarding the Y2K problem it was Nuke Plants running DOS or Windows that failed Y2K and wouldn't boot the plant.