What I expect from this is that I make all calculations in world coordinates, then use InverseTransformDirection to make it local coords and set constraints, then apply the result. But what I get is chaotic movement, jumping etc. I tried both InverseTransformDirection and InverseTransformPoint, but at some point it worked when I set the result to transform.position, not localPosition. So I must be missing something about those transforms.