1 Answer
1

This may have to do with the range and the angle of the knife attack. In order to get a takedown you need to come from either side or the back of the enemy. If you come from the front you will most likely just slash your opponent. You will slash the air if you happen to be out of range.

A second explanation is more likely though: You don't hold the button down long enough. A knife takedown required you to hold down the button instead of tapping it once. Source
EDIT: This counts when using the melee button, when having the knife in hand (as the second commenter points out and is said in the source) you'll see a change in the way the knife is held when a takedown is ensured (vertical for takedown vs. horizontal for slash).

+1, though I've had the knife slash the air even when directly behind the enemy and holding the button. Knife just seems to be broken in this game.
–
BlueRaja - Danny PflughoeftAug 1 '12 at 19:50

The knife mechanics are a little off. In most cases, if you try to stab as soon as you reach them, you will slash. There's a fairly small animation in which you look as if you're tightening your grip on the knife (noticeable if you pay attention as you sneak up on them... barely). That will signify a one hit kill.
–
FluttershyAug 1 '12 at 19:58

1

@Assile - nope, it's a glitch. A missed swipe, or sometimes just random factors causes the knife to be unusable, and will be forced to equip another weapon and re-equip the knife to use again. It's a known bug - commonly referred to as "The Rubber Knife glitch"
–
SSumnerAug 1 '12 at 22:24