The Truth Behind Web Cookies

The Internet is predominantly built on a technology called HTTP. It turns out that HTTP doesn't hold any notion of "state" between the browser and the server application. Therefore, when writing applications for the Web, developers need to pass a token between the browser and the client. This is how the "cookie" was born.

Cookies come in two forms, persistent and nonpersistent. Persistent cookies stay with your browser even after you've exited it. The next time you open your browser and access the website that placed the cookie on your browser, it will find the cookie still there. Nonpersistent cookies, as their name suggests, do not stay around after you exit your browser.

Typically, persistent cookies are used for personalization and keeping track of a user's behavior across multiple sessions. Nonpersistent cookies, on the other hand, are mainly used for session management.

How do cookies work?

Cookies can be set by the server in response to a request from a Web browser. If you visit a website, the Web server that hosts the website can send a cookie to your Web browser. Once the Web browser gets the cookie, it sends it back to the Web server each time it makes a request to the same URL. This is a mechanism by which a server hands something to the browser that's handed back the next time the user makes another request.

Note that the cookies that one server creates are not sent back to another website. For instance, a cookie that http://www.techtv.com sets is never sent to http://www.yahoo.com. This is one of the reasons why it's hard to have single sign-on between multiple websites on the Internet (although systems such as Microsoft Passport and Oblix NetPoint can enable such single sign-ons using something called HTTP redirect). On the other hand, the fact that cookies from different sites are not shared promotes privacy for users who do not want websites to interfere with each other or to track users across the Web.

What's in a Cookie?

The information stored in a cookie allows the browser to know which Web servers to send the cookie back to. The cookie also includes the payload, as well as information about when the cookie is valid. Normally, a site will encrypt the information contained in the cookie. When you ask a website to save your user name and password, it will either take those values and encrypt them before storing them on the file system or create an encrypted token that takes the place of your login. In many cases, when you examine the cookie information you'll notice a long string of characters that most probably doesn't make any sense to you.

Here's a cookie that TechTV.com set for me when I visited the website:

Here is what the different parts of the cookie mean:

The website that created the cookie, and the one that the Web browser will send it back to.

Flag (TRUE)5

A flag that tells the Web browser whether all the machines within TechTV.com or only specific ones can get the cookie.

Path (/)

This is the URL's paths within TechTV.com that can receive the cookie. Setting / as the path tells the Web browser to send it to all requests to .techtv.com.

Secure (FALSE)

Tells the Web browser that the cookie can be sent over HTTP instead of a secure HTTPS connection.

Expiration (1108418573)

Number of seconds from January 1, 1970, when this cookie will expire.

Value (Visitor 80c59448.484ec627.216.200.223.239.1013810251524)

This is the actual value that the TechTV.com Web server wants back when it receives the cookie. Just looking at it tells us that it's most likely using this to track repeat visitors. The site has assigned me a visitor number that it will use to look me up in some database of repeat visitors.

Where and How Are Cookies Stored?

Cookies are stored in different places by different Web browsers.

In the Netscape browser, cookies are stored in a file called cookies.txt that's with your user information. If you open that file, you'll find a separate line listing each cookie.

In the Microsoft Internet Explorer browser, cookies are stored in the Temporary Internet Files folder with all the other temporary Internet files. Each cookie is stored in a separate file with a particular format. The file is stored as "Cookie:<username>@URL", where <username> is the user who is logged into the Windows machine, and the URL is the address of the Web server that set the cookie. Opening each cookie file will give you the details about that specific cookie.

Single sign-on across multiple websites

Fundamental cookie technology hasn't changed much since they first came into existence with the first Web servers and Web browsers. What is interesting is how much has been built using such basic technology. In this section we'll talk about how single sign-on is accomplished between websites that can't share cookies.

This is accomplished by using HTTP redirect, where servers can redirect the browser to different URLs. Here's how it's done:

User goes to a website, say http://www.expedia.com. Since this website accepts Microsoft Passport authentication, you can click on the URL that says "Login With Passport."

When you click on the link, you're sent to http://www.passport.com. Expedia also appends some extra information to the URL that it uses to link with the Passport site.

The Passport site asks you for your user name and password.

Once it gets your user name and password, it'll authenticate you and create its own cookies for http://www.passport.com. These cookies are nonpersistent.

It then redirects you back to Expedia and attaches some extra information to the URL in the return.

Expedia uses this extra information to log you in, and also creates its own cookies for http://www.expedia.com.

At this point, the user has two sets of cookies, one for Passport and one for Expedia. If the user now goes to any other site that accepts Passport authentication, the user will not be prompted for authentication because the Passport cookies are already present with the Web browser.

How to Manage Cookies

Some of you may have experimented with managing your security by turning off cookies. You may have found that it's extremely hard to turn them off completely. There are many legitimate uses of cookies, and it's almost impossible to use certain sites without cookies.

It's risky, but you can clear out cookies manually by editing the cookies file or deleting cookie files from your temporary Internet folder. Internet Explorer, for instance, lets you completely clear out all cookies through an option in the Tools menu. In most cases, if you clear out the cookies, you'll have to log in again to websites where you had saved your login information.

You can also download and use programs, such as CookiePal and CookieWall, that can help you manage and filter cookies.