Posted
by
timothyon Friday July 29, 2011 @08:14AM
from the best-practices-are-best-practice dept.

Lucas123 writes "After dozens of hard disk drives were stolen from a leased facility in Chattanooga, potentially exposing the personal data of more than 1 million customers, BlueCross decided to go the safe route: they spent $6 million to encrypt all stored data across their enterprise. The health insurer spent the past year encrypting nearly a petabyte of data on 1,000 Windows, AIX, SQL, VMware and Xen server hard drives; 6,000 workstations and removable media drives; as well as 136,000 tape backup volumes."

But no one would ever guess "damnyouratbastardstohellihopearabidbadgerchewsyourballsoff" as the password for such a well loved and respected institution as a medical insurance company... So no worries!

Most insurance companies these days, are far more concerned with getting bonuses to the executives.

You don't honestly think that the executives will end up with smaller bonuses as a result, do you? We all know that isn't how this game works.

The company will cover these costs by raising premiums and/or reducing payments. It is very likely that the executives will see larger bonuses after this, as a self-congratulatory measure for "proactively correcting the situation".

I mean, as a customer, first you get screwed over by having your medical records out in public. Then the company gets fined and leverages that fine on its customers, thusly getting screwed a second time. Finally, costs are incurred for getting up to standards, and guess who is paying for those costs?

Which is a win for all. Executives can't get bonuses if there isn't a decent amount of income to the company, there can't be a good amount of income to the company if there aren't high revenues. There can't be high revenues if there isn't a supply of something people want that can be produced for at least slightly less than they are willing to pay for it. They wont be willing to pay for it if it's cost is higher than it's value to the individual.

You seem to be forgetting the financial companies who were so fucking broke we had to give them money or the world would end. You know, the same companies who took our money, turned around and gave billions in bonuses to their employees, presumably for doing such a wonderful job

And why we should have much lower taxes and smaller government and insure the government is never able to do a bailout like this ever again. Perhaps instead of taxes we could have bailout bonds issued so people could feel they were doing the right thing by buy the bank bailout bonds if they felt it was the right thing to do.

Why? Some states require that companies notify people when their data is stolen, as well as sometimes requiring identity theft protection (e.g., credit reports or alerts) or somesuch. This can get pretty expensive, and so it's probably cheaper to just encrypt everything. They're not being altruistic -- they're saving money. It wouldn't surprise me if some executive got a bonus for saving the company money...

This entire effort might be useless if they're not using good encryption. Is there one master passphrase to bypass all of the encryption? Also, they make no mention of how they plan to prevent physical theft of data again just that 'Well this time I put a password on my data, take that thieves!'

"We searched the country and were unable to find another company that has achieved this level of data encryption," Michael Lawley, vice president of technology shared services for BCBS, said in a statement.

He certainly did not search very hard. Less than 1PB encrytpted, we do more than that every single day. And I doubt we are unique.

It is a pity that the data was stolen before adequate protection was put into place, but it seems to me TN BCBS took the right steps afterwards:

1. They sent out alerts to those affected, both current and former members

2. They now encrypt all their stored data

Of course, this will not prevent all possible leaks, but at least it shows they are taking protection of their customers' data seriously, and have put in serious work to protect that data. I wish more organizations did that. Way to go, BCBS of Tennessee

"I know I already shit on the floor, but I'm wearing a diaper now so it's all good!"

where is badanalogyguy?

so you're saying that one mistake (data loss; floor shitting) will render every countermeasure (encryption; diapering) invalid? nah, I don't think so. The insurance company handled the data loss quite competent - they disclosed it early (afaik) and implemented a regime that will make future data losses much harder.

It wasn't a perfect analogy, but I don't think they should be congratulated for closing the gate after the horse already bolted. They're just doing what they should have been doing all along. Really, they shouldn't let anything even get stolen.

no, it makes data losses just as easy as they were before. It prevents data theft as the records are now (theoretically) protected. Without proper off-site backups they are still screwed if someone steals their drives again.

"I know I already shit on the floor, but I'm wearing a diaper now so it's all good!"

where is badanalogyguy?

so you're saying that one mistake (data loss; floor shitting) will render every countermeasure (encryption; diapering) invalid? nah, I don't think so. The insurance company handled the data loss quite competent - they disclosed it early (afaik) and implemented a regime that will make future data losses much harder.

Does the insurance company have insurance for their data?

If the jelly does not cover the peanut butter on the PB&J pizza, then the PB gets hard and difficult to eat.

I'm by no means a security expert but isn't $6 million a bit excessive for the effort?

TFA says "The company said it spent more than 5,000 man-hours on the encryption effort, which encompassed about 885TB of at-rest data." That equates to around $1200/hr. Perhaps I should become a security expert.

The 5000 man hours may only reflect actual labor and not reflect all the hours of planning/scheduling etc. What ever hourly rate for labor double it for overhead, the cost of a person is about twice their salary, at $100/hour that's $1M in labor. Another 500K in planning. I have no clue what software they used but I'm pretty certain it wasn't a single package. Each system may well have required a different package + licenses + contractor time from the vendor. For example they may have had to out source the voice call recordings to who ever provides their phone system. I kind of doubt they slap all the recordings onto a single box and mass encrypt.

They're a very distributed organization so there's going to be a *lot* of duplication of effort, they may have had to do the phone bit at hundreds of sites.

I don't know if it could have been done for $3M or if $6M actually represents a relatively reasonable price compared to a lot of the $XXX Mllion dollar utter failure projects. It strikes me as fairly reasonable considering the scope of the problem and usefulness of the result (assuming it's not a $6M whitewash).

Now we need to factor in an encryption scheme that works across Windows, AIX, etc with enterprise support backing it up say $1.2 million to licence for all servers and locations (seem low but hey) and we have $1.8 million to spend.
Now we gotta pay people some prices to do that work so lets say $.5 million (500,000) so about $100 per man hour (bout right) and we have $1.3 to spend.
Now pay t

Other people did a breakdown before me of the costs. Lucky thing: it's expensive to start but cheap to keep it, just remind people every 6 months that they should use the software. Oh, and check very often that you can restore your backups: there's nothing funny in working your whole weekend because an encrypted backup has locked itself in.

I don't think the barn door saying means what you think it does. It suggests pointless action taken after the event. The original data was stolen but encryption to hinder future theft of data seems sensible.

Even with the best commercially available encryption if someone steals the hardware storing the encrypted data they have all the time in the world to try and access it. The disks were in the possession of a 3rd party at the time of the theft so a security audit of their premises and security procedures might be in order to help raise awareness and prevent future incidents.

Your analogy, while not perfect has a valid point. However, remember that they now have a new horse in that barn. (all the customers that have since the data loss) What would you say about the farmer that lost his horse, got a new one, and still leaves the door open?

Perhaps the lesson here should be to all the IT people (does anyone in IT still read slashdot?) take this type of preventive action BEFORE you have data stolen. (yes, i know it's really up to the C-something-O to fund and order such an operation

$6 million is pocket change to a company that has $5.2 billion in annual revenue. However, the true cost is really higher, as encrypting everything means that things like disk corruption are no longer repairable, lost passwords can't be reset without losing data, and the like. It'd be interesting to see just what the ongoing costs are.

That said, I would like to compliment Tennessee BC/BS for doing the right thing, in spite of it costing money.

My personal experience with a couple of mainstream commercial enterprise solutions, is their data recovery tools leave a LOT to be desired and seem to only work for us about a third of the time. Features and management tools get the attention; auditing and recovery are after-thoughts in most products.In a few instances where we had to engage a data recovery service, they charge quite a bit more when they find out that they're dealing with an encrypted disk (i.e. when we're going after a specifc folder or a

$6 million is pocket change to a company that has $5.2 billion in annual revenue.

Right, but any money spent on IT is a waste to the stuffed shirts, until something blows up, which, inevitably, gets them off the fence. Telling the COs in a meeting, "our worst possible downtime with the current allotted budget might be as bad as 3 days," makes them all look at each other with satisfaction and approval, seemingly, ok with being down 3 days in theory. Then, after 3 hours of downtime, they are talking about

I work for a company where data is subject to HIPAA (United States' Health Insurance Portability and Accountability Act - a law whose provisions also address the security and privacy of health data). Our data has been encrypted -- at rest and in transit -- for years. The loss of private health information, like what Blue Cross did, is a serious crime under HIPAA and subject to major fines (in this case, at least tens of millions of dollars, probably, given how large the breach was). The initial cost to encr

It sounds reasonable on the surface, since people think of drive theft as very exceptional and something you can physically defend against. But then.. these people never had a drive fail and then RMAed it? Am I supposed to believe that when there's a mechanical failure and they're unable to erase the drive, they destroy it rather than mailing it back to a vendor or manufacturer?

leased facility = cloud so this is what you get from going to the cloud the data can be in a place that can range from a nice data center to a small room in a office building. Also the people ruining the cloud can just have real low prices and then sell data to the highest bidder.

Leased facility != cloud. In a leased facility, you can find out the operational conditions and the level of physical security. You can make them part of the lease contract if you care enough. You can't do that in a cloud.

... even if it is far too late. And of course, the customers will pay for the cost of the failure, plus the cost of the fix. The company made a bad choice, and the consequences of that bad choice will be born by.. the customers. The executives will still get their usual multimillion dollar "performance" bonuses as if nothing was ever wrong.

It's the government. They "encrypted" the drives. What do you think that means? Do you really think they did it properly? Or do you think they Bought some licenses form Symantec and clicked next next next? Randomly generate passwords? Seriously?

If they were properly secure in the first place, the would not need encryption. Encryption is for data that leaves your network. If physical media is leaving your network, you're doing it wrong.

These drives were likely part of various RAID volumes. Doesn't that mean they're pretty well useless outside their hosts? Is someone really going to go to the level of forensic data recovery to elevate from property theft to identity theft? That stuff isn't cheap, so the ROI is probably going to be really low.

Should be, but generally isn't. Security costs money, and most companies have been in a cost cutting mode for years. Security is one of the first things to go since it's invisible until you're compromised.

Generally I think most companies don't need it. Some only need the basics. You got my personal information, or credit cards? Just securely encrypt those sources. Sure some might slip out here and there, but you won't lose your whole database of 300,000 customers or whatever.

I just mean if your a bank, financial institution of some description, or someone that handles my medical information, get on the encryption boat and set sail. Seriously. I mean it is one thing if someone gets my VISA number... its usual

Looked around the stories including their "infographic", not clear what they are using and how they've implemented it.

Do servers have pre-boot enabled? How did they change they operational processes? Are these HW-encrypted drives? What is the failure rate on the process?

Details like this are important. As it stands, they spent the cash and a lot of time, but no indication that they've implemented it properly. I wouldn't feel much safer.5,000 hours is nothing to be honest for even a mid-size company. T

when one of their machines reboots, where does the key come from? such sites usually spend as much money as possible on the theory that mauve is better, which in this case probably means FC SANs. but at which level does the encryption happen? and doesn't disk encryption just mean that you need to take the enclosure or client box too?

This is complete bullshit. Even if for some reason the company held each number in an individual file rather than documents, spreadheets, databases, etc. you could encrypt the drive. You could also encrypt the individual files if you wanted to.

And then they can pay me again to switch to TrueCrypt when BitLocker falls off the Microsoft upgrade treadmill:-P

Firstly, as someone else has already said, not everything is based on Windows.

Secondly, I cannot think of a product I should be less inclined to use than TrueCrypt to deal with such a problem. Reason I say this is simple - in every large business you always have the occasional helpdesk call to reset a forgotten password - usually when someone's just come back off holiday. How exactly are you going to deal with the problem when the answer to a helpdesk call for a lost TrueCrypt password is "please send the l

What is ironic that any enterprise tool has encryption built in if it was made in recent times:

The EMC devices have Powerpath encryption for LUNs. Someone hacks the SAN, nothing available on the server other than trashing the LUNs.

IBM storage arrays check if they can boot off a key server, and then unlock their encrypted drives in hardware. If this isn't enabled, AIX has EFS (different from Windows's EFS) to ensure that only the user with the right key can attach a directory.