Im new to OpenSSL, Can anybody give me a hint in how to initialize AES CTR mode from a C file. I know this is the method´s signature but I am having problems with the parameters, there´s not many documentation neither a clear example how to make a simple encryption. I would appreciate if somebody could exemplify a call to this method. Thanks in advance!

Hi Caf I really appreciate your quick answer it has been really useful, and defenetly the best example I have found on the web. I am trying to open a file with undetermined length, encrypt it and write another file with the ciphertext generated, then open the ciphered file and recover the plaintext. I need to use a file of a considerable amount of MB cause I would like to benchmark the performance of the CPU. However Im still having a problem while decrypting. Somehow when decrypting a considerable txt files (1504KB)it wont decrypt it complete, and I get half of it in plaintext and the other half still ciphered. I think this might be related to the size of the iv or the way I am calling the counter. Here is what I have so far:

Your problem is that you are reinitialising the counter after each block. This is wrong - move the init_ctr() call outside of the while() loops in both encryption and decryption. indata and outdata also don't have to be of AES_BLOCK_SIZE length - they can be considerably larger.
–
cafAug 6 '10 at 0:24

2 Answers
2

Usually, you will be intending to call AES_ctr128_encrypt() repeatedly to send several messages with the same key and IV, and an incrementing counter. This means you need to keep track of the 'ivec', 'num' and 'ecount' values between calls - so create a struct to hold these, and an initialisation function:

(msg_in is a pointer to a buffer containing the plaintext message, msg_out is a pointer to a buffer where the encrypted message should go, and msg_len is the message length).

Decryption is exactly the same, except that you do not generate the IV with RAND_bytes() - instead, you take the value given to you by the other side.

Important:

Do not call init_ctr() more than once during the encryption process. The counter and IV must be initialised once only prior to the start of encryption.

Under no circumstances be tempted to get the IV anywhere other than from RAND_bytes() on the encryption side. Don't set it to a fixed value; don't use a hash function; don't use the recipient's name; don't read it from disk. Generate it with RAND_bytes() and send it to the destination. Whenever you start with a zero counter, you must start with a completely fresh IV that you have never used before.

If it is at all possible that you will be sending 2**64 bytes without changing the IV and/or key, you will need to test for the counter overflowing.

Do not omit error-checking. If a function fails and you ignore it, it's quite possible (even likely) that your system will appear to be functioning normally, but will actually be operating completely insecurely.

Let me add a detail that escaped me when I used this: the num argument is how many bytes into a block you are, not the counter. If you are encrypting packets (for instance), always set state->num to zero and put your counter into the high bytes of iv.
–
Mike ElkinsNov 9 '10 at 18:51

@Mike Elkins: Indeed - you can treat both num and ecount as opaque internal state of the OpenSSL CTR implementation. In most cases it should not be necessary to directly alter them.
–
cafNov 9 '10 at 21:35

It looks like the basic problem with your test program is that the mode values of the fopen calls is not correct. I think you need to change your fopen calls in encrypt to this:

fp=fopen("input.txt","rb");
op=fopen("output.txt","wb");

And the ones in decrypt to:

rp=fopen("recovered.txt","wb");
op=fopen("output.txt","rb");

One other thing worth pointing out is that ckey should probably be declared as a 32 byte (256 bit) buffer. It is true that the 128-bit encryption only uses 16 bytes of the data from the key. But the OpenSSL function AES_set_encrypt_key (at least in the version I am using) reads 32 bytes from that buffer. It only uses the appropriate number of bytes, but the read does occur. That means that if the buffer is only 16-bytes and happens end at the end of a page that is adjacent to a non-readable page in memory, it would result in an access violation.

Oh - and I just noticed that there is an extraneous call to free in there. The free(buffer); call is not valid since buffer was never allocated. I realize your code is just a simple test, but ... well, we are programmers and can't help ourselves.