Ouch! There are so many logs here and how should I start? Just like sguil where you must speak mysql pretty well to perform better analysis, you must speak shell scripting enough to perform smooth analysis flow on Bro logs. The power of Bro lies in it's protocol anomaly detection and therefore having different protocols logging in different files(prefix with its protocol) do make things clear. However you can't consider its logs as alert and indicator of malicious event(different approach comparing with snort) even though alarm, notice and weird logs do give some clues about suspected network event. Another problem should be when the file grows larger in a day, it is pretty hard to crawl the file to check on malicious event. For example look at one of the file -

It could be deep pain if you want to read them, don't you? Therefore it becomes unclear when one first learn about Bro and want to proceed to examine the logs but don't know where to start. Here I will give you some insight of how I perform analysis using Bro but I need another powerful tool to assist me, I vote ourmon!

Bro only generates network statistical data in its daily report therefore we don't have real time view of current network statistic, therefore ourmon plays the main role for you to access to the network statistical data via its well crafted graph. But where's our flow data to understand each connection? Bro does store connections flow data, all its connections flow stored in the file with the prefix of conn.*. The only data we don't have is full content(pcap), you can either use tcpdump, daemonlogger or Bro alone to do the job for you however I would like to ignore this part for the moment because generally you can have idea of each connections because most of the widely used and popular protocols such as http, smtp and so forth are decoded and stored in its corresponding file such as http.*, smtp.* and so forth, other protocols that are not decoded(provided no decoder for it) still can be seen in conn.* to understand its nature. Here's the example when you find unconfirmed or suspected malicious activity in certain time frame via notice log -

You may see I pipe to cf, cf is small utility to translate unix epoch time format to human readable form, therefore it's more easy for you to read and correlate the event based on time frame. The format of connections flow should be read in this way -

Here you go, you are now having full understanding of the network connections(all those http requests with same tag number), this is kind of batch analysis to identify network events. I don't explain how I make use of ourmon here or maybe I will make it for future post. Ourmon generates the graph by plotting such as TCP flags ratio, network errors, irc stats and some other graphs that you can correlate with the data provided by Bro to fully understand your network and readily countering any network threats.

Below I demonstrate some of the tips I use before I examine Bro logs, it's good catch if we want to read the separate kind of logs in timely manner, for example I want to check out the alarm log only, I just execute -

I use ls with -c switch so that I can sort them by file creation time, that way making us easy to check out any daily log we want by date especially searching for connections tag. In the alarm log, you may see the time stamp of every network event started with t=, that render cf tool unable to convert the time stamp from unix epoch time to human readable form correctly -

Just discard t= and parse it to cf. Bingo! Piping to less command is also great because you can perform certain function such as search by regular expression, easy log navigation and so forth(vi style functions).

Both Bro and Ourmon provide you great context for most of anomaly network events, I have spent my time to figure out the better way to utilize them and hopefully this help anyone who uses them because I did face difficulty when first dealing with Bro that it seems to require more man power to examine them. One more tip, remember to turn on dynamic protocol detection(DPD) when dealing with stealthy attackers.

Some of you may wonder why not examining the application protocol log(http.*, ftp.*) and instead of digging the connections log first, the answer is pretty simple, if you examine the connections log and found no data transfers(Orig Bytes Sent | Res Bytes Sent), it's pointless to look at the application protocol log and may indicate some kind of network probing or scanning activities. However this doesn't apply to every condition and sometimes you need to tinker of how to perform analysis effectively. I draw the simple diagram for better illustration -

I use color depth to indicate the understanding level of the network event. You may see the color in the entity becomes lighter and lighter from left to right when you have better understanding of network event through out the structural analysis process.

What's the lacking in Bro? Clearly enough that it really needs better way to organize and manage its logs and perhaps OSSEC can fill the gaps with additional log parser. Using this mechanism with the deployment of Snort NIDS, I'm pretty confirmed that you can identify known and unknown(0 days) network attacks.