Gigi Labs

Monday, May 20, 2013

HTTP Requests in Wireshark

Hi everyone! :)

In yesterday's article, Network Programming: Networking Theory, we discussed what happens when a message is sent over a network, and when it is received. Today, we're going to see a practical example of that, by observing the HTTP requests sent by a web browser.

The first thing you should do is download Wireshark. This program will allow you to monitor network traffic going into and out of your PC. After installing it, run it, and you will see the following main screen:

Click on "Capture Options" and tick the checkbox next to the network interface listed. The network interface is basically a network card or, more commonly, the networking hardware on your motherboard. Wireshark can monitor traffic passing through the Ethernet port.

Click the "Start" button to start capturing packets. Immediately, you will start seeing stuff going in and out of your PC. You will know whether it's incoming or outgoing depending on whether your PC's IP address is in the "Source" or "Destination" column (in the screenshot below, my IP address is hidden):

In the "Filter" field at the top, type "http" and press ENTER. This filter allows you to concentrate on a specific type of network traffic - in this case, we are focusing on HTTP traffic which is used by web browsers.

In the Capture menu, Restart capturing, since there is a lot of traffic that doesn't interest us. From a web browser, visit http://www.programmersranch.com/. Soon after, Stop capturing in Wireshark from the Capture menu.

You can now find various HTTP requests to various parts of the page at programmersranch.com, including the page itself and various images. The screenshot above shows the HTTP request for the main page.You can expand the sections towards the middle of the window to view more detail about various parts of the transmission. In this case, I've expanded the HTTP section, where you can see the whole HTTP request. You can do the same for TCP, IP, etc.

When you click on a particular section (such as HTTP), the relevant part of the hex view (at the bottom of the window) is highlighted. This is useful because it sometimes shows you things that you might otherwise miss. In particular, you'll notice that the last four characters are represented by hex values: 0d 0a 0d 0a. In decimal, this becomes 13 10 13 10, which map to the ASCII values of CR LF CR LF (carriage return, line feed, carriage return, line feed). In short, you have two blank lines at the end of the HTTP request. They are important because HTTP requests won't work without them.

You should also be able to find the HTTP response coming from the server, which contains the HTML arriving at your browser (shown above).

Finally, in Wireshark you can right click on a particular transmission and select "Follow TCP Stream":

This allows you to view all the relevant requests and responses on the same connection without having to find the packets one by one:

Be aware, however, that following a TCP stream like this will change the filter from http to something else. This means that you won't be seeing all incoming HTTP packets. Be sure to change the filter back in order to continue viewing HTTP traffic.

Very well. You now know how to use Wireshark to sniff packets going into and out of your PC. In code, you can create the same messages and send them out in a socket in order to achieve the same behaviour that browsers, email clients, etc. have. In tomorrow's article, we will be working with HTTP in code. So stick around. :)