handling 150mb pcaps

i'm looking for feedback or best practices, or just approaches the community takes with dealing with a whole days worth of pcaps. i'm using security onion. it collects full packet captures for 2 LANs x.x.x.x/21. A lot of traffic. It collects 150MB per packet.

what does anyone do when trying to sift, coordinate, 'get the ful picture.' Currently I am using IDS tools to find a specific time to investigate with the pcap, but i'm doing them one by one. what if you want to expand to an hour of traffic? what if you wanted to look for a trend in traffic that would not be noticed in individual pcaps? how would you handle information for a stream that extends out of the 150mb limit?

4 Answers

I usually throw them all into TraceWrangler at once (as @NJL suggested already), and use the Tools/Communication Details menu option to look at the conversations I need. Double clicking a row extracts all packets of the conversation to a new PCAP and runs Wireshark to open it for investigation.

It's also possible to use extraction tasks to extract only those conversations with a Snort alert to inspect them specificially.

Comments

You can merge pcaps by simply dragging all of them onto Wireshark or use the "File->Merge" dialog (requires that you have one PCAP already opened in Wireshark).
You can also use the mergecap command-line tool and finally you can also use TraceWrangler to merge captures into one.
You could also use tshark to do a commandline analysis of your large capture files, filter what you want and export it to separate files and then merge everything together.
Depending on the amount of data this might be the preferred method and simply leave your machine "cooking" overnight etc., but of course it requires you to do some form of scripting (bash/python/BATCH etc.)

I can recommend Riverbeds Packet Analyzer.
It's very snappy and makes it very easy to work with multi-GB capture files.
You can filter and graph almost anything and it's easy to export a selection of traffic to Wireshark for detailed packet-by-packet analysis.

If you're after following specific TCP sessions across multiple capture files I can recommend TraceWrangler.
It has it's limitations, bugs etc. but when it works it's great and makes it very easy.

Do you know what you're looking for in your captures or are you looking for something that's just "odd"?

The best thing is,when you know for what you are looking for and when it has happened. Otherwise there can be a lot of approaches. But one thing that can work is the graphical analysis. ( Looking for peaks)