Now after hundreds of tests I can only confirm that HC-SR04 isn't really a sensor to rely on.
Yes it shows something and it is somehow correlated with reality but... the deviations are so big!
In my case it shows stable results if you point it onto a wall... if you put a book, hand or try to dance in front of it... so unreliable measures! I even tried to apply statistical methods to filter outfitters silly errors etc... too bad!

I've had some success with this but not using the HC-SR04. As others have commented this sensor is rubbish at anything except detecting a good flat surface like a wall for example.

I instead used the Sharp GP2Y0A02YK0F. You have to re-write some of the code to use this as it just provides a sense voltage you have to convert in to a distance. There is lots of useful information on the comments section here https://www.sparkfun.com/products/8958. Which is where I managed to come up with the function below.