I love Denon, I have a 3808. Yes, when you use 5.1 it'll only use the surround left and right speakers, not the surround back left & right speakers. As far as I know, there's no way to manually have the receiver take the side channels to the rear speakers behind you. The rear back channels are for 7.1 only.

In a 5.1 setting the surrounds are supposed to be to the sides - at or a little behind and above the sitting area. This should give you a good surround experience. If you feel you want the sound to come from behind just move the speakers back a little. The Amp setting should not matter. ( the back setting are meant for 7.1 - I think your Amp makes it easier so people do not confuse back with surround )

I'm using a new Denon receiver and was wondering if there's any way to fix this so the rear sound actually comes from the rear and not the side. I could also move my sofa back a bit during 5.1 movies.

Does your Denon have DPLIIX? If so, choose this mode for 5.1 movies to hear sounds from both the side and rear surrounds. If the movie is 7.1, then the receiver should automatically detect it and utilize all 4 surround speakers.