Wednesday, October 7, 2009

Most people just couldn’t fathom how complicated the web is. From the time a user types the URL into the browser’s address bar and hits Enter, to the time the web page opens – a period barely a few seconds long, a thousand different things happen. Your request goes through thousands of kilometers of undersea optical fiber cable, across several continents, through hundreds of computers performing millions of calculations till it reaches the destination, and then makes the same trip back home. All this happens before you can even say “Internet”.

Now, it wouldn't be possible to espy on the whole journey, but we can certainly find out what data goes through to and fro using a web proxy. Fiddler is a great tool in this regard. It’s particularly useful when we have reasons to be suspicious about a website. Where is a page retrieving it's content from? Is it sending back data to places it isn’t supposed to? Is the page doing funny things on your computer?

Fiddler is a free web debugging proxy which logs all HTTP/HTTPS traffic between your computer and the Internet, and allows you to inspect and “fiddle” with the incoming and outgoing traffic.

Fiddler is available as a plug-in for Internet Explorer, meaning you can only use it to debug sites opened in IE. Traffic via other browsers are not intercepted by Fiddler. Just discovered, it works with Firefox too.

Once you open the site you want to debug, launch Fiddler using the icon in IE’s toolbar or via the start menu. Here you can see all HTTP requests sent from your computer. You can click on any request and get a detailed report on it – bytes transferred, time taken for DNS lookup, time taken to reach each server on it’s way etc. If you want to dig further, click on the “Inspector” tab. Here you can view the headers, authorization data if any, or the hex codes. The data is available in various different formats like text, XML or raw.

This is useful if you want find out if the website behaves suspiciously, like sneaking in a cookie or communicating with another site or server. The response from the server is displayed in the bottom half of the window. Here too you can inspect the headers, see what external files are being transmitted etc.

The amount of information is actually quite overwhelming. So there is a solution – filters. You can filter out traffic you are not interested in. Since a single page can output so much data, the filters will come extremely helpful when you are actually debugging. For instance, you can filter out the images and style sheet and concentrate only on JavaScript or ActiveX elements. You can even intercept the data, modify it and see what response you received from the server.

Fiddler also supports plug-ins and there are a handful available that brings various features like Syntax highlighting, gallery for images, content blocking where you can altogether block certain contents from appearing on the site.

On Fiddler’s website you can find a couple of very demonstrative videos which will walk you through each and every tool in the program. There are also a few documentations and a discussion board.