Javascript is not even remotely as big an issue as browser plugins (java/flash/silverlight/adobe reader/etc) when it comes to security. To me, noscript is like browsing the web with images turned off to avoid jpeg or gif vulnerabilities.

You do understand that you need to whitelist the pages that you actually want to see? Most of the time if a page doesn't work correctly without flash and javascript and I am only going to visit this site once I don't even bother.