It's not as easy as you guys make it believe. If you switch the mouse buttons, you ARE going to get in trouble with menu's. But by this time there's a mod that exactly addresses this most annoying thing. It's called: On-screen Interface Key Fix, and I use it myself with joy. You can change whatever you want but the keys in menus will not be affected.

It can be downloaded for free and without subscribing at skyrim nexus.

As Mikalichov says, it's easy to change in the menus on PC. I was irritated by this too and made the switch early in the game (around 20 hours, yes that's early).

This is only for combat. It doesn't effect menus, alchemy, enchanting, smithing or trading at all.

But there is not option to change the "preferred hand". For instance a single handed weapon always gravitates* to your right hand and spells/shields gravitate to the left hand. This happens even if you switch the mouse buttons and as far as I can see, there is no way to swap this.

If I'm wrong, PLEASE say, because I may be right-handed, but I'm index-fingered (that sounds bad).