In what sort environment could he battle him where he could really cut loose though? because if the battle is on Earth, then Thor is not going to endanger innocent people by cutting loose, and it's similar if they encounter each other on a battle field, his comrades in arms may die as a result of his cutting loose.

Sorry, but I think Avengers disproves this.

Even though it's not shown or implied, Thor *had* to have done a helluva lot of collateral damage in that movie. Knocking cars all over Manhattan, blasting thunderbolts everywhere, smacking down jetbikes and leviathans into crowds below, busting open the Helicarrier in his duel with Hulk....