Introduction To Database Encryption - The Reboot!

Updated June 4th to reflect terminology change.

This is the Re-Introduction to our Database Encryption series. Why are we re-introducing this series? I’m glad you asked. The more we worked on the separation of duties and key management sections, the more dissatisfied we became. Rich and I got some really good feedback from vendors and end users, and we felt we were missing the mark with this series. And not just because the stuff I drafted when I was sick completely lacked clarity of thought, but there are three specific reasons we were unhappy. The advice we were giving was not particularly pragmatic, the terminology we thought worked didn’t, and we were doing a poor job of aligning end-user goals with available options. So yeah, this is an apology to our audience as the series was not up to our expectations and we failed to achieve some of our own Totally Transparent Research concepts. But we’re ‘fessing up to the problem and starting from scratch.

So we want to fix these things in two ways. First we want to change some of the terminology we have been using to describe database encryption. Using ‘media encryption’ and ‘separation of duties’ is confusing the issues, and we want to differentiate between the threat we are trying to protect against vs. what is being encrypted. And as we are talking to IT, developers, DBAs, and other audiences, we wanted to reduce confusion as much as possible. Second, we will create a simple guide for people to select a database encryption strategy that addresses their goals. Basically we are going to outline a decision tree of user requirements and map those to the available database encryption choices. Rich and I think that will aid end users to both clarify their goals and determine the correct implementation strategy.

In our original introduction we provided a clear idea of where we wanted to go with this series, but we did adopt our own terminology in order to better encapsulate the database encryption options vendors provide. We chose “Encryption for Separation of Duties” and “Encryption for Media Protection”. This is a bit of an oversimplification, and mapped to the threat rather than to the feature. Plus, if you asked your RDBMS vendor for ‘media encryption’, they would not know what they heck you were talking about. We are going to change the terminology back to the following:

Database Transparent/External Encryption: Encryption of the entire database. This is provided by native encryption functions within the database. The goal is to prevent exposure of information due to loss of the physical media. This can also be done through drive or OS/file system encryption, although they lack some of the protections of native database encryption. The encryption is invisible to the application and does not require alterations to the code or schema.

Data User Encryption: Encrypting specific columns, tables, or even data elements in the database. The classic example is credit card numbers. The goal is to provide protection against inadvertent disclosure, or to enforce separation of duties. How this is accomplished will depend upon how key management is utilized and (internal/external) encryption services, and will affect the way the application uses the database, but provides more granular access control.

While we’re confident we’ve described the two options accurately, we’re not convinced the specific terms “database encryption” and “data encryption” are necessarily the best, so please suggest any better options.

Blanket encryption of all database content for media protection is much easier than encrypting specific columns & tables for separation of duties, but it doesn’t offer the same security benefits. Knowing which to choose will depend upon three things:

What do you want to protect?

What do you want to protect it from?

What application changes and management tasks will you tolerate?

Thus, the first thing we need to decide when looking at database encryption is what are we trying to protect and why. If we’re just going after the ‘PCI checkbox’ or are worried about losing data from swapping out hard drives, someone stealing the files off the server, or misplacing backup tapes, then database encryption (for media protection) is our answer. If the goal is to protect data in the event of compromised accounts, rogue DBAs, or inadvertent disclosure; then things get a lot more complicated. We will go into the details of ‘why’ and ‘how’ in a future post, as well as the issues of application alterations, after we have introduced the decision tree overview. If you have any comments, good, bad, or indifferent, please share. As always, we want the discussion to be as open as possible.

Comments:

If you like to leave comments, and aren't a spammer, register for the site and email us at info@securosis.com and we'll turn off moderation for your account.

By reppep on 06/03 at 10:09 PM

Adrian,

I don’t like this usage of ‘Database Encryption’. The term implies something handled internally by the database software’s encryption features, but you include external whole-disk/FS encryption under this heading.

It might be worth adding a third category: External Encryption, for things outside the DBMS or closely coupled software. Then ‘Database Encryption’ could become ‘Whole Database Encryption’.

By Adrian on 06/03 at 10:18 PM

I see your point. Like the “Whole Database Encryption” suggestion. Rich?

-Adrian

By Rich on 06/03 at 10:36 PM

That doesn’t totally work because sometimes it’s only a column, even though it’s not data level.

I’m struggling myself to find something better.

By Clerkendweller on 06/05 at 08:56 AM

Thanks for the re-introduction. I’d perhaps like to add something to the “three things” list, but it may be out-of-scope. How about:

“Does it need to be there?”

The encryption issue may be reduced or avoided by considering:

* whether the data should have been collected at all
* if the data is past its retention period and should be disposed of
* could the data be removed (since it is stored elsewhere)
* could a tokenisation system be used instead?

I’m sure you will have already thought these things through, but some people won’t have.

By Arshad Noor on 06/05 at 05:55 PM

May I humbly point out that encrypting data in any layer but the application layer leaves your data insecure. Storage media, operating system, database - no matter which layer you encrypt, as long as it is NOT the application layer, you leave gaps between the application and encryption layers that attackers will surely exploit.

The advantage of encrypting in the application layer is that you only have to do it once. Neither you nor the attackers can go higher than the application layer in the technology stack, so you’ve concentrated your defenses, while ensuring that no matter where the data does after it leaves the application layer, the data is always encrypted. You also avoid the need to buy self-encrypting drives, database encryption modules, etc. All you need is a robust, secure key-management system with the libraries to integrate into your application and handle the crypto work for you.

While you may argue that you do not have access to the source-code of the application and cannot modify it, I humbly disagree. As long as you’re paying the bills to your ISV/internal developers for coding the application, the application CAN be modified. All it requires is a will to do it.

By Mostafa Siraj on 06/08 at 11:54 AM

Hello,

very nice (re)post for the article, I was wondering about the “User Encryption” native support for the main DBMS providers (MSSQL and Oracle for example), is it supported by these providers if yes, can you provide me with links so that I can know more about it?, if no, is there any guidelines to do it or third party support

Thanks.

By Adrian on 06/08 at 09:34 PM

@Arshad - It really depends upon the threat. In principle I agree, but if the customer is only worried about lost or stolen media, then modification of the application is an unnecessary burden. Self encrypting drives are way cheaper than several thousand man hours to re-design, re-code and retest an application, underlying database and any management/disaster recovery processes that need to adjusted to accommodate for the change in data or columns being encrypted.

@Mostafa - All of the database vendors have links for column, table and in some cases cell encryption. <a href=“http://www.ibm.com/developerworks/db2/library/techarticle/benfield/0108benfield.html#N1008C”>IBM</a>, <a href=”(http://download.oracle.com/docs/cd/B28359_01/network.111/b28531/data_encryption.htm”>Oracle</a>, and <a href=“http://msdn.microsoft.com/en-us/library/ms179331.aspx”>Microsoft</a> will have a lot of published material but you will need to do some digging to find what you want. A majority of collateral will will steer you towards ‘Transparent’ or ‘TDE’ references, and this is not what we are talking about. There is a ton of practical third party users and authors who have good explanations but code snippets as well, so you may prefer a google/yahoo search as you can tune it to what you are interested in.

-Adrian

By Arshad Noor on 06/11 at 04:11 AM

@Adrian - I am not sure that a company can only worry about lost media. It might be their most visible problem, but let’s not forget that the two largest breaches in IT computing (TJX and Heartland) did not involve a single lost tape, disk or laptop. Both were attacks of online systems and Heartland was certified as PCI-compliant).

In both cases, the attackers compromised layers of software above the media layer (see Page 12 of this slide-deck at http://csrc.nist.gov/groups/ST/key_mgmt/documents/June09_Presentations/Arshad_Noor_KMWJune09_StrongKey.pdf), thus allowing them to gain access to unprotected data even if the disk-drive, OS or database had performing the encryption.

In the long-term, modifying the application and fixing the code to encrypt data in the application layer gives you the highest probability that you will NOT have to revisit this problem again. Witness Heartland’s and TJX’s problem - despite spending millions on PCI compliance, hundreds of millions in settlements for TJX (still unknown for Heartland), they’re having to spend money all over again on a problem they could have taken care of correctly the first time around. Isn’t there a lesson in this for everyone else?

By Adrian on 06/12 at 01:06 AM

@Arshad - Yep, understand where you are coming from. Look at the next post in the series and look at Rich’s post on June 11th regarding Database vs. Application encryption.

Thanks,
Adrian

By michael o'keefe on 06/11 at 04:37 PM

@Arshad - One clarification - according to the WSJ, TJX was not PCI compliant.

Also, application encryption is important, but despite high profile cases like the TJX hack and others, which get all the publicity, the majority of fraudulent activity is from internal sources. In other words, if I encrypt at the app level, but not the database, that is of little use if the bad guy doesn’t need to use the app to access the data.

By Arshad Noor on 06/11 at 04:47 PM

@Michael,

I’m not sure I understand what you mean, Michael. If the application is encrypting the data, the implication is that at all layers below the application - including the database layer - the data is already encrypted. So, unless, the bad guy had authorized access to the plaintext data, how would he get to it?