David Kennedy, CEO of TrustedSec, first announced Heartbleed's involvement on Fox News and in a TrustedSec blog post Wednesday afternoon. Kennedy says he had this confirmed by three people close to the incident, who remain anonymous. CHS has not yet publicly confirmed this and could not be reached for comment as of press time.

"The initial attack vector was through the infamous OpenSSL 'heartbleed' vulnerability which led to the compromise of the information... Attackers were able to glean user credentials from memory on a CHS Juniper device via the heartbleed vulnerability (which was vulnerable at the time) and use them to login via a VPN," Kennedy said in yesterday's blog post.

"This is why we put up Heartbleed.com," says David Chartier, CEO of Codenomicon, which discovered the Heartbleed bug, in conjunction with Neel Mehta of Google Security. "We were very passionate about telling people how to recover from this."

Codenomicon advised vulnerable parties to not only upgrade to a new version of OpenSSL, but also to revoke encryption keys, issue new encryption keys, and make users change their passwords.

Trouble is, not everyone followed those recovery steps (if any), nor in a timely manner. Chartier says that revoking and re-issuing keys, in particular, took some time for security managers to do. The Heartbleed vulnerability became public April 7. CHS believes the intrusion in their systems occurred in April and June. Chartier says he will be "not at all surprised" if it turns out that Heartbleed was part of how attackers obtained the records in this attack.

The CHS breach is being hailed as the biggest security incident Heartbleed has caused so far -- at least the biggest one we know about.

"Although it is irrational to think that Heartbleed hasn’t already been utilized in even larger-scale attacks prior to the Community Health Systems breach, this is just the largest thus far where the investigation proved Heartbleed was what got them the data," says Evan Keiser, security analyst for SilverSky. "Taking into account how long many major corporations, hospitals, government, and healthcare servers were vulnerable to the Heartbleed bug I would be extremely surprised if this was the biggest attack in terms of scale."

Regardless of whether or not CHS has suffered the worst blow from the bludgeoning of the OpenSSL bug, some caution that we should not focus too much on one unpatched vulnerability; there are plenty of other mistakes CHS made.

"It seems pretty obvious to me that Heartbleeed could have played a role, but it's not how [attackers] got their hands on the data," says Dr. Vincent Berk, CEO of FlowTraq.

Berk compares Heartbleed to sonar pings. Just like most of the pings a submarine sends out aren't going to be echoed with anything interesting, Heartbleed hacks can, but rarely will, come back with valuable data like privileged access credentials. "There's no way really to tell what kind of data you [are going to] get, and most of it is trash."

As Berk explains, because of the extensive attention in both industry and consumer media, everyone knew about Heartbleed, and surely competent, caring security professionals would have made sure all systems were adequately patched. "Either [CHS] didn't care or they're just not qualified" to manage their systems, says Berk. Assuming they did care, and just failed to patch for such a well-publicized vulnerability, says Berk, "I can only imagine how many other ways to get in there are... If it wasn't Heartbleed, it would have been something else."

One of CHS's biggest offenses, says Berk, is that they were only watching for what came into the network, not for what was going out of it -- like 4.5 million customer records. Exfiltrating such a large set of data would take time and computing resources that should have raised red flags.

Similarly, just like the large-scale exfiltration of protected data is abnormal, there are other kinds of uncharacteristic behavior that was probably exhibited by the attackers using a legitimate employees' credentials.

"It's very easy to get your hands on valid credentials," says Nir Polak, CEO and co-founder of Exabeam. Whether it's a leak through Heartbleed or merely spearfishing, Polak says, legitimate logins will be lifted and borrowed by attackers.

Trying to prevent that from happening is a worthwhile endeavor, says Polak, but organizations need to act under the assumption that sometimes it's going to happen anyway -- and start looking for ways to tell if a legitimate user is behaving strangely.

"It's been happening in the fraud protection industry for a long time," says Polak, noting how banks now routinely monitor for inconsistencies in customers' purchasing activities. If an expensive purchase is made at a point-of-sale machine in Italy, when the legitimate customer is still buying groceries in New York as usual, the account may be flagged and frozen. "[Those methods] just need to be applied to information security."

New technologies are being created to do just that; for example, the BioCatch "passive biometrics" solution DarkReading has covered before.

Patching a highly publicized vulnerability in open-source code is just one small task companies must do to fulfill a bigger responsibility to lock down all the computing tools they use. Chartier points out that some companies may not have realized that OpenSSL was part of their environment, because it might have been located in embedded systems.

"It is part of an enterprises’ obligation to their customers and stakeholders, that they understand their supply chains," says Veracode CTO Chris Wysopal. "To help, they must look into third-party code auditing programs so they can rapidly assess their deployed environment for software vulnerabilities, even in the case where the software is on an appliance.

"Vendors need a way to quickly understand where they have built products with open-source components," Wysopal says. "All products should use Software Composition Analysis with an alerting mechanism for rapid response when a new vulnerability is made public in an open source component. And if the software component analysis tool integrates with a SAST solution, all the better so they can instantly find all locations components in their previously scanned code."

Another weakness in CHS's infrastructure is the apparent lack of segmentation in their customer database. CHS owns, leases, or operates 206 hospitals in 29 US states, none of which served 4.5 million patients all on their own.

Heartbleed's role in the CHS breach is noteworthy, but more noteworthy are the pile of best practices (and common sense practices) that the hospital chain failed to do.

"While Heartbleed was how the credentials were stolen this time," says Joshua Roback, security architect for SilverSky, "this could have just as easily been a common spear phishing attack, similar to the Target attack earlier this year. The real concern is the ability to hack into the database once logged in, and then exfiltration [of] the data."

Sara Peters is Senior Editor at Dark Reading and formerly the editor-in-chief of Enterprise Efficiency. Prior that she was senior editor for the Computer Security Institute, writing and speaking about virtualization, identity management, cybersecurity law, and a myriad ... View Full Bio

And if the CC data had been encrypted in the POS terminal the attack would have yielded ... nothing. We all agree it's inevitable. What we need to be doing is 1) defending against the obvious and known, and 2) deploying defense in depth, so that it doesn't matter if they get through. Encrypting data at rest and in motion is part of #1.

I disagree with your assessment of the Target breach. Ram scraping malware wasn't created for Target, it has existed and been used in many POS breaches. that wasn't the point anyway. Ultimately, preventing breaches is damn near impossible. No one said "do nothing". What I'm suggesting is that even if you do everything right (which is actually impossible in any sufficiently large organization), you still might get breached. Anyone who thinks their environment is perfectly secure either has a non-functioning business or is wrong. these are going to keep happening and in fact it will get far worse IMHO.

Target did NOT lose encrypted data; they lost clear text because of a specification error that didn't immediately encrypt it and because they didn't use whitelisting. Eventual vulnerabilities are no excuse to do nothing. Agreed that encryption is not a magic bullet. Yet encryption of data in flight and at rest needs to be the bare minimum starting point and we aren't even there yet.

While nobody is immune to these attacks indeed - there is really no excuse for not being able to update your security barriers once you know the nature and breadth of the breach on table. Heartbleed has certainly told you some serious markers on how not to handle data. Ultimately though, no amount of data security can account for the security habits and practices that a security manager of a big enterprise can put in process. Enterprise security (as outlined here goo.gl/a67V8i) may never be foolproof, but a lot of it is a matter of internal discipline and keeping an eye out. You must care about sensitive data your customer trust you with, for your own sake!

it's symptomatic of every single large IT environment in existence. NO ONE is immune. Not the banks. Not the Gov't, or their contactors. Not security companies. And, not your organization. Encryption isn't a magic cure for breaches. Target encrypted the data that was stolen at resta and in transit. But at some point the data has to be in clear text to be useful, and whatever processes involved can be subverted.

This is just symptomatic of no defense in depth. Data needs to be protected in flight AND at rest. And hiding behind a single Curtain Wall didn't even work in castles 600 years ago. Just like the NSA Snowden fiasco, CHS owes it to their patients, stakeholders and regulators to take irrevocable corrective action.

I agree with your last statement. But the risk can be lowered substantially through a strong security posture. However, as attacks evolve so must the protections and unfortunately the attacks are evolving faster than the patrol.

Neither would have netflow, unless their network traffic patterns are very predictible and/or the attackers were very stupid. Often neither is the case. Let's assume each record was average 200 bytes and 10:1 compression. Even a single file with all 4.5 million records would not be very big in the grand scheme of what often flows across a network. And that assumes the file wasn't split into smaller parts.

Sadly, the reality is that breaches are now just a part of life on the Internet. Even organizations with relatively strong security can become victims if there is sufficient motivation.

@aws0513 writes that "the only question that remains is in how transparent the organization leadership is in notifying share holders, law or regulatory entities, and the general public when a breach is identified."

Sadly I think that unless organizations are required by law to be transparent, the status quo will continue for a long time for all but the most security-focused businesses. It's a tragedy-in-the- making for healthcare not to be counted among them.

I would also like to point out that simply standing up a IDS would not have prevented or even alerted this type of attack. Based on what I have read, the only way to have caught this would be to monitor netflow data and look for spikes.

Most organizations, not just healthcare, do not have someone assigned to monitor the netflow data 24x7 so I think this would have gone unnoticed in many organizations.

Published: 2015-03-03Off-by-one error in the ecryptfs_decode_from_filename function in fs/ecryptfs/crypto.c in the eCryptfs subsystem in the Linux kernel before 3.18.2 allows local users to cause a denial of service (buffer overflow and system crash) or possibly gain privileges via a crafted filename.

Published: 2015-03-03** REJECT ** DO NOT USE THIS CANDIDATE NUMBER. ConsultIDs: none. Reason: This candidate was withdrawn by its CNA. Further investigation showed that it was not a security issue in customer-controlled software. Notes: none.

How can security professionals better engage with their peers, both in person and online? In this Dark Reading Radio show, we will talk to leaders at some of the security industry’s professional organizations about how security pros can get more involved – with their colleagues in the same industry, with their peers in other industries, and with the IT security community as a whole.