.NET Developer, Microsoft MVP

Caveats in ASP.NET Core Data Protection

ASP.NET Core provides a built in Data Protection mechanism to let us encrypt or decrypt sensitive data. There are some limitations that can cause problems while bringing convenience. I met some problems these days.

My Scenario

My blog system has a feature to send email notifications, so you need to configure an email account to let the program use that account to send mail to an administrator or users. This involves the question of how to securely store your account password. As programmers with morals, we certainly can't store plaintext passwords to databases like many platforms in China. In this scenario, we also can't store passwords with HASH, because sending messages is done by the system in the background, and the user is not required to enter a password to compare the HASH with the database stored values. Therefore, my first thought is to use the symmetric encryption algorithm such as AES, store the encrypted ciphertext in the database, the program decrypt the values according to key, and then use the account to send mail.

Don't Reinvent the Wheel

Before designing a feature, I usually check the internet first to see if there is already a framework feature that comes with the functionality to complete the requirements. As a result, ASP.NET Core's own Data Protection APIs is brought to my attention.

The benefits of using Data Protection APIs are:

Replacement for the classic MachineKey

No need to design encryption algorithm on your own, just use the most secured framework provided algorithms.

Without having to manage the key yourself, the framework will automatically store and manage the key for you.

The key is refreshed every 90 days by default

API is easy to use without requiring to know deep about encryptions

Retain flexibility and extensibility, allowing steps such as custom algorithms, key storage, and more

I used this method to encrypt the email account password and store it in the database. Then change the code to successfully decrypted the data, run on my own machine and complete the function of sending mail without problem. So I deployed it to production.

Production Blow Up!

An exception occurred in the production environment while decrypted the ciphertext in the database:

System.Security.Cryptography.CryptographicException: The key {bd424a84-5faa-4b97-8cd9-6bea01f052cd} was not found in the key ring.

After research, this is because ASP.NET Core generates different keys to encrypt data when it is running on different machines, and the ciphertext in my database is encrypted with the key of the developer machine, not the key of the server. So when you try to decrypt, you can't find the key for encryption, and this exception blows up the server sky high.

ASP.NET Core can store Key to Windows Registry, IIS User Profile, Azure KeyVault, Azure Storage Account or File System, etc. In Azure App Service, the Key is stored in %HOME%\ASP.NET\DataProtection-Keys folder, it will automatically sync into other instances in your App Service. You can see the folder with Kudu:

Therefore, to solve the problem of key inconsistency in different environments, only need to find a consistent storage location. But that's not going to solve my problem. Because by default, a new key is regenerated every 90 days, so that the ciphertext in the database will fail to decrypt again if it is not updated.

In addition, the AntiForgeryToken in ASP.NET Core is also using the same Data Protection APIs, which in case when you deploy multiple web servers on your own (not using App Service to scale out), it will result in different keys in every server, and the user will fail on submitting form data.

The Solution

Although we can do this by saving key in a unified location, we can also specify an automatic refresh cycle, but I do not recommend it. Because this mechanism applies only to encrypted, short-aging data, it is not designed for data that is persisted into the database. So in this scenario, we still have to write our own service of encryption and decryption.

First (very faceless) copy a pair of AES encryption and decryption functions from Microsoft's official documents:

The key and IV can be obtained by KeyInfo's parameterless constructor. After saving these values, you can always use this pair of key, to ensure that the subsequent encryption and decryption data are consistent.

Conclusion

ASP.NET Core comes with a Data Encryption API that is secure, easy to use, and flexible. However, that key storage and timed refresh apply only to short-aging encryption. For long-time saved static ciphertext, you can implement a encryption and decryption service on our own.

Thanks, your comment is pending approval now.
It will show up once blog administrator approved your comment.

A

asozyurt 2019-1-15 13:56

Thanks for this useful article Edi.It's a good and informative blog, I'm glad to follow.
Wanna ask you smt about Rss/Atom feeds.I have a blog in .Net Core too and recently added Rss/Atom support. (http://blog.asozyurt.com/rss) But there is a problem with the prolog Encoding declaration(<xml version="1.0" encoding="utf-16">)Your feed is ok with encoding utf-8. I've solved it by replacing the string but it doesn't seem right to me.Codes are in myGithub. Can you share your solution or guide me? Thanks again.

A

asozyurt 2019-1-15 14:06

I think I found, https://github.com/EdiWang/Edi.Blog.OpmlFileWriter. Is it true?

Many thanks. We where test crawling our website and noticed a difference in the URLs. Now we know why :) It's already hard to keep consistency when developing applications in a team. Now even need to be more careful with selecting which UrlEncode to use by default.