There is probably a problem with either the shift linkage or sometimes a cable is used in the shifter and that might need replacement. The shifter itself could be binding up.
My best advice is to take it to a transmission shop and not a regular repair facility. Automatic tranmission, especially in a RAV4, are difficult to work on and take special experience and tools to work on them. So, take the extra time to take it to a transmission spoecialist for repair.