There are either too many possible answers, or good answers would be too long for this format. Please add details to narrow the answer set or to isolate an issue that can be answered in a few paragraphs.
If this question can be reworded to fit the rules in the help center, please edit the question.

Installing the Tor Bundle includes the TorButton Firefox extension, which won't allow plugins like Flash - these plugins could leak your identity, even when using Tor. For example, a Flash app could determine the local IP and send that information back to the server, so even if it's being sent over Tor, you're still hosed. So use Tor, Tor is good, but don't use any unsafe plugins.

To be certain that browsing history isn't saved on a hard disk, you can boot from a Linux Live CD and run Tor from there.

I would imagine you mean not keeping your information from being stored on your computer, but being stored on everyone elses. For the former, just use private browsing mode, which is now in every major browser.

Not creating any evidence of your activities at all is practically impossible in most circumstances, however, using throwaway accounts, disabling tracking cookies, and not using services known to have poor privacy policies (or help governments illegally spy on their users) is a good start.

Also avoid using public terminals you can't be sure are clean of keyloggers, spyware, etc. Need to use a public terminal? How about booting up a live CD/SSD? If you're even more paranoid, bring your own USB keyboard.

Anonymous proxies and anonymity networks like Tor are good tools since your physical location (and thus your physical identity) is never connected to your online activities. Though you can still leave an online trail and potentially do something that reveals your offline identity, such as logging into an account tied to your offline identity.

hide yourself using encryption

If you can use end-to-end encryption, and especially if even the service provider at the other end is unable to access your data, then there may be evidence that you were online, but no one can tell what you were doing. Here I'm using the term "encryption" generally. You can even disguise your data so others can't tell what you're really doing. But don't do something stupid like use HTTPS everywhere but then transmit your password in plain text when using FTP or send an email over an unencrypted connection.

You can even use encryption to hide your activities offline by using full-disk encryption. Even if you don't save browser cookies/histories, you probably still keep bookmarks and so on. If you use a password manager for your online passwords, it might be a good idea to keep the password database in a separate hidden volume.

cover your tracks after the fact

Obviously you can't break into a data center to erase your online tracks, but you can still use private browsing/incognito mode so that your browsing history/cache/cookies/etc. are erased as soon as you close the browser. (Note: this doesn't work with Flash cookies. And although many companies, including Clearspring, Quantcast and Hulu were sued for using Flash cookies for tracking, it's still used by many, including Hulu.)

hide yourself in the crowd

Most website operators enable basic logging out of necessity. However, the sheer volume of traffic they receive can provide a certain level of privacy if you don't stand out from the crowd. That's why Panopticlick measures your browser's uniqueness. Likewise, if you use a public internet connection that lots of people use and your usage patterns are ununique, then it'll be very hard for others to identify your tracks amongst the noise.

Shared public accounts can be handy for this as well. Usually such accounts are simply a way to bypass compulsory registration, but if thousands of people are sharing a single account, it's hard to tie one person to a specific activity.

Different anonymity services exist for what type of anonymity you are looking to achieve.

Want to change your IP?

Use Proxies and Tor

Free Public internet hotspots

Want to surf on a machine without leaving a trace?

Use the measures that MiffTheFox mentioned

Also something that has not been mentioned yet is VPN services. These usually cost $$$ and money can always leave a trail back to you depending on how you spend it. It all depends on how much you trust your VPN provider.

There are some other ways of getting anonymity but these cross the lines of what most countries consider to be illegal.

You can use Tor, change your IP all the time, delete all data left behind, but given enough time, it can always be traced back to you. So yes, for simply hiding one or 2 sites you visit, a proxy and all these other good idea work fine.

But if your not supposed to be doing it, legal reasons or whatnot, just remember, they can always fun you given enough time and enough subpoenas will always find you.

You use an anonymous connection like open wireless networks or prepaid mobile internet cards, strong encryption, and secure services (service providers that either don't keep or anonymize their logs) so that there's no one to be subpoenaed. Perfect security may be impossible, but good enough security is possible, even against governments.
–
Lèse majestéApr 13 '12 at 11:25

1

Even with that, browser plugins, IP address, fonts even can provide a unique identifier for almost any computer in the world. Yeah it might be anonymous for the most part but if someone can test your computer versus a known ID, your screwed.
–
alpha1May 21 '12 at 15:37

Use TAILS, The Amnesic Incognito Live System from https://tails.boum.org/ via either burning the ISO file to a Live CD or a USB flash drive. It incorporates all apps such as email for use over Tor, and appears to release every couple of months. It uses a Debian Linux release with a Firefox derivative (version 10+) that is heavily modified to protect privacy along with the Ad Block Plus and NoScript Firefox add-ons and uses Vidalia, a GUI controller for Tor. They are working on producing a persistent storage (USB related) in a future release.

Privoxy is a non-caching web proxy with advanced filtering capabilities for enhancing privacy, modifying web page data and HTTP headers, controlling access, and removing ads and other obnoxious Internet junk. Privoxy has a flexible configuration and can be customized to suit individual needs and tastes. It has application for both stand-alone systems and multi-user networks.

No. A virtual machine, by itself, will do nothing to hide your ISP (ISP = Internet Service Provider). It will also do nothing to hide your public IP (Internet Protocol) address, although it will change the one used internally on your local network. The most that would do is allow you to tear down the virtual machine to erase any traces of activity on your hard drive, but it would have to be used in conjunction with other methods to hide your IP (such as Tor or UltraSurf).
–
ContangoJan 31 '13 at 9:54