An encryption function is supposed to obscure the plaintext to the point where no information can be obtained about it. If the underlying encryption algorithm can even be determined, that goal has not been achieved.

Is this correct? Can double-encrypting (with either the same or separate algorithms) weaken security?

$\begingroup$I suppose it's possible that there might be a second key that would cause the second encryption operation to cancel the first, resulting in plaintext output.$\endgroup$
– user9070Mar 18 '16 at 13:28

$\begingroup$Only if you mess up (e.g. by using a cipher that leaks information about the plaintext into the length of the ciphertext.)$\endgroup$
– CodesInChaosMar 18 '16 at 20:37

4 Answers
4

Can double-encrypting (with either the same or separate algorithms) weaken security?

If you do not assume that the algorithms and keys are independent, then it certainly can.

The example of ROT13 from the other answer illustrates the point even if it is not real encryption. Similarly, a synchronous stream cipher applied twice with the same key will undo itself. Something like that could be possible with other kinds of ciphers, e.g. you could have block cipher such that $E_k = D_{k'}$ if you choose the keys wrong, or you could define AES' where the encrypt and decrypt directions are switched. A combination of these would be secure with independent keys but not necessarily otherwise.

If you assume keys are independent, the result can still be weaker compared to the stronger of the two algorithms. For example, assume you have CBC encryption that is vulnerable to a padding oracle attack. Even if you add a secure stream cipher on top, the padding oracle attack still exists. With some of those assumptions you can say that the combination is no weaker than the first cipher (pdf).

$\begingroup$But if double encrypting can lead to it being easier to obtain the plaintext, than can't an attacker simply encrypt a single-encrypted ciphertext they're trying to break so that it is not double-encrypted and therefor easier to break?$\endgroup$
– ShelvacuMar 19 '16 at 9:31

$\begingroup$ROT13 is "real encryption", providing you consider a Caesar shift to be "real encryption". In general, a double encryption with Caesar shifts will be a null operation if the two keys (= shifts) chosen add up to a multiple of 26. With ROT13: 13 + 13 = 26.$\endgroup$
– rossumMar 19 '16 at 10:44

$\begingroup$@shelvacu, if the attacker tries to encrypt again, then their key is independent (or they probably know how to decrypt anyway), so it's the situation of the final paragraph and the encryption is no weaker than the initial one.$\endgroup$
– otusMar 19 '16 at 13:09

$\begingroup$@rossum, ROT13 is Caesar with a constant key. I wouldn't call AES-with-constant-key real encryption either. Anyway, whether you consider it encryption or not has no bearing on the real answer.$\endgroup$
– otusMar 19 '16 at 13:11

The answers and comments here are good, but I think that it's worth tidying it all up a bit. The question is broad, and this is exactly expressed in the answers. There are multiple questions here. Before I begin, I note that when we talk about the keys not being "independent", we need to define what we mean. I am only going to relate to the keys being the same. If you let the keys be related (e.g., one is the inverse of the other), then it's easy to come up with counter-examples showing nothing is secure. However, in reality we are interested in the same key versus independent keys. We will start with these three questions:

Is double-encryption necessarily secure when the algorithm and keys are the same?

Is double-encryption necessarily secure when the algorithm is the same but the keys are independent?

Is double-encryption necessarily secure when the algorithms are different but the keys are the same?

What attacks are we considering as well? If we only consider eavesdropping then it's different to the case that we consider chosen-plaintext attacks. We also need to ask whether we are considering the case that both algorithms are secure or not. (Of course, if they are both secure, then why bother double encrypting. So we'll consider the case that it's only guaranteed that one is secure later.)

Let's start by answering the above three questions:

When double encrypting with the same algorithm and the same key, and when the scheme is secure under chosen-plaintext attacks, it is easy to see that security is preserved. This is easy to prove: a CPA attacker can double encrypt by itself by querying the ciphertext back to the oracle. Therefore, if one can break the double encryption then one can break the single encryption. However, if the scheme is only secure in the presence of an eavesdropping adversary, then double encryption can break (for security, define that the adversary outputs two vectors of plaintexts, gets their encryption, and needs to determine which vector was encrypted). In particular, take any eavesdropping-secure encryption scheme and modify it so that an encryption of an encryption of 0 outputs the secret key. This will still be secure in the presence of eavesdropping adversaries (easy to prove; will leave it to you). However, this will be completely broken when using double encryption.

When encrypting with the same algorithm and independent keys, it's easy to prove that security is preserved. The reduction can generate the second key itself. So, this is secure even if the scheme is only secure in the presence of eavesdropping adversaries.

When encrypting with different algorithms and the same key, you're in big trouble. This should be obvious since you are never allowed to reuse keys for different schemes. I'll leave this again to you to come up with a concrete counter-example (just make sure that both schemes are secure).

A more interesting/important question is whether double encryption is secure when one of the schemes may be insecure. In general, constructing a scheme that is secure from a number of schemes where only some of them are secure has many advantages. The first construction of HMAC used a combination of SHA1 and MD5 for this reason. Formally, we call such constructions robust combiners. Regarding double encryption, this is the cascade research here.

$\begingroup$Regarding same algorith same key, what about OTP? Encryption and decryption are the same process, so OTP with the same pad will give you the plaintext again. I guess theres also the problem of the pad no longer being one-time.$\endgroup$
– ShelvacuMar 21 '16 at 15:53

$\begingroup$@shelvacu Yes, OTP is an example of a scheme that is secure against eavesdropping only. So, this is another counterexample. However, it is unsatisfying in the sense that it is anyway only secure for a single encryption (but is still mathematically a valid counterexample). In contrast, my counterexample works even for schemes that are secure for multiple encryptions (with eavesdropping adversaries only).$\endgroup$
– Yehuda LindellMar 21 '16 at 18:03

1

$\begingroup$"When encrypting with different algorithms and the same key, you're in big trouble. This should be obvious" It is not obvious.$\endgroup$
– ShelvacuMar 21 '16 at 20:35

$\begingroup$@shelvacu I assume you tried to find a counterexample. See the answer saying that you should take $E'_k = D_k$. This is a problem as is, since no-one says that E' will be secure. In addition, it's a problem to define this for probabilistic encryption. But, you should be able to do something similar that works.$\endgroup$
– Yehuda LindellMar 21 '16 at 20:40

$\begingroup$Adding a key that's ignored by ROT13 produces a cipher with a key. You appear to be specifying some requirement more involved than just a key, but it's unclear what.$\endgroup$
– OlatheMar 19 '16 at 4:44

I would say that the answer is yes, if the first and second encryption algorithms are the same algorithm but with different keys.

Most probably in the above case, a third encryption key would do a single decrypt of the result.
If the algorithms are disjoint (like aes and twofish) and each uses a different key, I would say no, it is more robust.
I would do a AES encryption followed by a Twofish encryption, along with cypher block chaining between the first and the second. (if using CBC make it three algorithms, then drop the CBC).