It’s not a matter of what the word “literally” means. (And since the end meaning of words – particularly in Japanese and Chinese – depends largely on context, “mu” does not have a literal meaning. It simply implies a negative condition.) It’s a matter of how the word is used. And how the word is used is that no answer can exist with the conditions given. Answering “mu” is the same as saying “I can’t answer that question.”