Hmm....trying to think of a place you wont get a lot of hassle. Maybe the Distillery. I know you are outside, but you could pop in and out of the cafes to get warm. My daughter had her engagement shots there, and they are killer. No, I did not take them. Thats why they looked so good.
Off the top of my head, I cant think of a place that is interesting and inside, that could have some cool backdrops. BCE place has some interesting walls and windows. But has a new look to it , where as the Distillery has that really old look. Will keep thinkin on it.

It's a tough question. You want inside on a Friday evening, yet many buildings will be closed after working hours are over. And others may want a permit of some sort.

I would think many of the University buildings will be open after hours. Lots of nice old buildings, and again somewhere to warm up in between poses etc. I have no idea if permits are required though. Victoria College has a great staircase in its main building. University College-and-Hart-House has that great quad and several (outdoor) corridors with picturesque columns. Kensington has lots of great graffiti-ed alleys (if they're into that sort of thing), but it will be busy on a Friday night.