The ravings of a SANS/GIAC GSE (Compliance & Malware)
For more information on my role as a presenter and commentator on IT Security, Digital Forensics Statistics and Data Mining;
E-mail me: "craigswright @ acm.org".

Dr. Craig S Wright GSE

Followers

My Profile

Share it

What is happening

BooksI have a few books and another is on the way for 2012. Firstly, I have to plug the first in the Syngress Series of books on IT Audit. This is a comprehensive compliance hand governance handbook with EVERYTHING (from the high level to the hands on for the expert) to get you started in IT compliance and systems security. The main book is "IT REGULATORY AND STANDARDS COMPLIANCE HANDBOOK". This is the first in a series I have planned and more will follow in time. There will be electronic updates to this book over time to maintain it to a current level over time.

I will be working on co-authoring a book on CIP (Critical Infrastructure Protection) - but more on this later.

On top of this I recycle computers. To do this I take 1.5 to 2 year old corporate lease computers and refurbish them so that they can run the most current programs.

The question is - what do you do to help?

If you do not have the time, have you though about a donation?

This blog has been monetarised. This is where the money goes. By clicking and purchasing on this site, you help Burnside and Hackers for Charity. All monies earned here are split 50/50 between these two charities.

Who I am...or what...

Visitor locations

Tuesday, 7 July 2009

DnsTop is a libpcap application. Like NGrep and tcpdump (covered in previous posts), it is an application that captures and analyses network traffic. DnsTop can be used to display various tables of DNS traffic on your network. As the site states, dnstop displays tables of:

Source IP addresses

Destination IP addresses

Query types

Response codes

Opcodes

Top level domains

Second level domains

Third level domains

etc...

Compiling and installing varies (with some RPMs available). Generally this is a standard "./configure && make && make install" process. This is of course dependant on your system's installed libraries.

Running dnstop is simple. the '-4' option sets IPv4, the '-l 9' option sets the application with up to 9 tables. Remember, the more tables, the more process memory and CPU used. Usually this is not an issue. Finally, the interface or file is set. In this case eth0. A PCAP format file can also be used as the input.

dnstop -4 -l 9 eth0

This means you can capture a PCAP file using TCPDump, Snort or any other sniffer of your choice and feed the data to dnstop for analysis. For instance, we can review the file, "save.pcap" and display the dns query results from this.

dnstop -4 -l 9 save.pcap

If you start with level 9 (-l 9) as an option, you have a far greater range of options.

The program is interactive. Select the level and it will display the various output styles. For instance, in the saved data we have the output for level 1 (TLD):The domains at level 2:Servers at lower levels and we get to a level with source, hosts and counts displayed:Using the type option (enter t) you can drill down into a summary of the DNS query types recieved:or the summary of the opcodes (enter o in the interactive screen).

In this post I will go over a dry run on using NGrep. We will start with capturing a file using TCPDump into a PCAP format and then reading this into NGrep to extract the information we seek.

Step 1The first stage is to capture all of the data we need into a PCAP file. For this we will use TCPDump. To do this, we are going to capture all traffic (with the complete payload, '-s 1500') to a file called capture.pcap.

tcpdump -nn -i eth0 -s 1500 -w capture.pcap

The command above and in the image below is what we have used for this. See the posts on TCPDump for more details on the TCPDump command settings. In the image below, I have used the 'date' command before and after the capture so that you can see this has run for a number of minutes.Note that we are not translating the IP address and protocols. This means that we have to learn what the protocols usually are and also we are less likely to make assumptions (for instance, TCP 80 is not always web traffic).

Step 2Now that we have our PCAP capture, we can start an examination. It is always a good idea (in incident response and forensic situations) to make a hash and a copy of the file.You can see how we have done this and validated the copy as well in the image above.

Step 3We can see the contents of the entire capture with NGrep by looking at our capture file.

ngrep -xX -I working.pcap

The command above and in the image below as output displays the packet capture in Hex and ASCII. This is of course far too detailed and we need to filter the results to make any detail from this.In this instance, the packet output displayed shows a HTTP GET request and associated data. The Proxy used is clear inside the packet as is the web site called (http://www.msnbc.com). From the GET request data we can see that the user is making a search request (the page is - /search?).

Step 4Now, say we are looking for a specific incident that occured using HTTP on TCP 80 (the default) to the server http://www.msnbc.com, we could first restrict the output to HTTP on TCP port 80 only (port 80)

ngrep -W byline -I working.pcap port 80

The output for this command is displayed below:Notice that the output contains the details of the HTML recieved from the web server. This is from any web server however and we wish to restrict this to a single host. We will also save this output to a file using the followin g command (where we have the server, www.montrealgazette.com):

The command and output is displayed below.We have now saved the selected traffic we wish to investigate. We can verifiy that we can read this using another PCAP compatible program (in the instance in step 5 TCPDump).

Step 5Here we are validating that we have saved the data capture we wanted using TCPDump to read the extract capture file (/tmp/traffic.pcap):

/tmp/traffic.pcap

The output is displayed below showing us that the packet capture has worked.Here you can see we have a limited set of traffic to a specific host with a specific request (HTTP GET). NGrep is a powerful tool and well worth taking the time to learn how to use. It can be far easier to have a small extract of a capture to work with or even to determine if a file contains information work investigating before you go into details.

NGrep can do this for us.

Next, we will look at some other tools (including Wireshark and DNSTop).

Monday, 6 July 2009

You can monitor and search for about any packet combination you can think of with NGrep. For instance, you can set it up to watch TCP port 25 (SMTP) and monitor and record email traffic.Using RegEx filter expressions, you could even filter this for selected source or destination hosts, email addresses or content. An example of this is the use of such a filter to protect intellectual property. A small string could be embedded into documents (e.g. Word files in the templates) and this could then be used as a search term.As from the image above (FTP) and that below (Telnet - in this case for Cisco and similar devices) we could look for insecure authentication. This example looks for username and password combinations that are sent over clear-text pathways.In addition, the "-W byline" option will display data is a manner that is better presented for us humans. The example below shows us collecting HTTP (TCP 80) traffic. This example allows us to see the actual HTTP calls to the server and the ASCII data contained within this process.We could even restrict this to selected web pages that we want to monitor.

Sunday, 5 July 2009

NGrep will either treat the search input as case sensitive (the default) or with the use of the '-i' option will treat it as being case insensitive. This is, you can search with or without capitalisation in a standard string. Of course, this does not matter if you are using a well formed RegEx.

TCPDump is simpler and as a result makes a better capture engine then NGrep (not that you can not do this with it). As such, it is better to use TCPDump to capture to a PCAP format file, and then to analyse the saved data with NGREP. The '-I' flag is used to select the PCAP format file that you want to read.

By default, NGrep will display the ASCII of a file and insert a '.' in place of unprintable characters. The '-x' parameter is used to print both Hex and ASCII in the output (recommended).

More on NGrep again tomorrow. Tomorrow we will go through a dry run of using NGrep.