What works for me is standing infront of a ledges and just press space, no movement buttons and it usually works, also on places that are not marked as climbspots.

Sometimes it doesn't work, you have to stand high enough to reach the ledge, Geralt then grabs it and climbs it.have to keep trying at some times.

And sometimes there are other entrances too, under water for instance.Yesterday i was in a cave, dove a bit and after a (long) swim underwater (made it using the health key a lot) i ended up in a cave with 2 Gargoyles, wasn't even marked on the map.