Choose a different distrobution then if you don't like how Cononical's distributions do it. Your data is not locked into one distribution and there are many very good ones to look at. Check out a full install distribution like Mandriva 2008.1 or PCLinuxOS. If you need to stick with the apt/deb based distributions, Debian is very nice and not much more complex than Ubuntu so it may cover all your needs.

The difference here is that if you don't like Windows, you don't have a different distributor or different version of Windows without a particular browser. If you don't like Ubuntu, you can easily look at other distributions; some even provide better hardware support than Ubuntu.