I worked with VIDEO ONLY. No pictures.
Noticed, that setDisplayOrientation does affect right display orientation only during preview.
But if to play any video captured either in landscape mode or portrait mode it is always in landscape mode.

I set in AndroidManifest.xml

But used to debug in 4.0.1 device. Maybe that is a problem.
However i noticed that default camera app works fine and capture portrait movies in portrait mode.

I've stuck with this problem before, too. I found that you can use the function setOrientationHint (API 9). Call this function before you call MediaRecorder.prepare(). You can setup the orientation degree for your output video.