I am developing an Encryption/Decryption Utility using Adobe AIR. I have chosen the AES algorithm and currently use the as3crypto library for this. My query is in what practical ways can I detect if the password provided at the time of decryption valid or not.

As of now I thought of encrypting a constant string with the password and storing the encrypted string in the header of my encrypted file. For decryption, I can check if the correct password is provided by decrypting this string.

This is a temporary work around as storing a ciphertext of a known plaintext (The string is hard coded as a constant inside my application) is not a proper way as I know.

Since I don't have much experience, I am looking for a proper and safe way to accomplish this. My application will be a standalone desktop application without a server.

Do you really have to check if the key is correct? You could simply decrypt the file with the wrong key and then let the user handle the garbage :-)
–
Paŭlo Ebermann♦Jul 28 '11 at 13:15

lolz, you don't want to be spending a couple of mins decrypting some 100Mb+ file to find out the file is broken, with no info that the password is wrong. Plus the original file's extension is also encrypted and stored.
–
midhunhkJul 29 '11 at 5:29

3 Answers
3

First of all, you should definitely be using some sort of an authenticated encryption solution. If you can't use AES-GCM or some other mode designed specifically for authenticated encryption, you should probably pick AES-CTR and HMAC-SHA256.

Next, you should encrypt the actual data with a completely random key you have generated and store this key encrypted to the password (using a password based key derivation function, such as PBKDF2). If you want to be really proper, using a standard key-wrapping algorithm for this would be best, but using authenticated encryption works. Do not encrypt the key without authenticating it.

This gives you two benefits:

You can change the passphrase to a file without re-encrypting it fully

You can detect if the passphrase was typed wrong

And included the standard disclaimer: if this isn't for a hobby project - please don't. There's no need to reinvent the wheel - you can implement standard OpenPGP symmetric encryption, for example. And even if you do use a standard - cryptography is hard to implement correctly and it takes a long time to build confidence in an implementation.

Here's what I understood, correct me if I am wrong. For encryption, create a random key and encrypt the PlainText. Then encrypt this key with the user entered password and store it. So while decrypting, I have to decrypt the saved key using the User provided password. So how can I know the decrypted key is same as the one generated during Encryption. It is a random key, right?
–
midhunhkAug 17 '11 at 5:55

2

You got that part right, but you missed one part: "Do not encrypt the key without authenticating it." The HMAC-SHA256 part will tell you if the key is correct or not with overwhelming probability. This is almost the same as encrypting a fixed string - or encrypting a crc - just that the authenticator is cryptographic and covers the key as well.
–
NakedibleAug 17 '11 at 8:15

After the header is decrypted, the last 1 or 2 bytes in Buffer
should be the high-order word/byte of the CRC for the file being
decrypted, stored in Intel low-byte/high-byte order. Versions of
PKZIP prior to 2.0 used a 2 byte CRC check; a 1 byte CRC check is
used on versions after 2.0. This can be used to test if the password
supplied is correct or not.

The known encrypted string is CRC of zipped file after compression. You can use any hash function instead of CRC.
As you mentioned CRC string stored in plaintext and encrypted to be used if the password
supplied is correct or not.