Reformatting seems excessively ridiculous. If drivers and OS's were so incompatible as to always cause problems, reformatting would be an entire sub-industry of the PC market.
I've done a lot of video card switching with the simple steps mentioned above and have no stability or performance issues. Problems show up, fine use a driver cleaner. Still screwed? Consider reformatting. Given the time it would take someone like me to do (tons of re-installs to do), its a last resort.
If your switching sides (AMD/Nvid), just make sure uninstall all software (Control centres, physx, etc) in there, and run a registry cleaner.