For instance, what makes MD4 a bad choice for an HMAC? In this case I am asking about MD4 because its less than ideal. I know that a preimage attack can be used to undermine the system, but why? What is the attacker calculating?

1 Answer
1

Actually, HMAC might still be secure for a hash function that is broken (with respect to the requirements of a cryptographic hash function, such as primary preimage resistance, secondary preimage resistance and collision resistance), but it must not be too badly broken. If you read the original paper, you see that the authors assume things such as the hash function being "weakly collision resistant", or more precisely that the inner hash function is collision resistant only to adversaries that see the hash value only after it being hashed again with a different secret key.

Also, since HMAC, contrary to NMAC, doesn't use two independent keys, but a single key that is combined with two different pads using XOR, additional assumption about the pseudo randomness of the hash function are made.

I believe MD4 does not even meet these weaker assumptions, and that is why HMAC-MD4 is considered to be broken, while e.g. HMAC-MD5 is not (yet).