If this is your first visit, be sure to
check out the Forum Rules by clicking the
link above. You may have to register
before you can post: click the register link above to proceed. To start viewing messages,
select the forum that you want to visit from the selection below.

Help with extracting tcpdump data...?

I'm using tcpdump to monitor the traffic on my router - very slick! Anyway, right now I'm using wireshark with a filter like this to pull yahoo chats:

data.data contains "Command=\"6" || data.data contains "Command=\"11"

This gets me the right packets, but it's very tedious to extract the actual chat session - ie, I have to do a 'follow stream' in WireShark and then copy and paste all of the chat texts to another document. In addition to that, it seems like the follow stream only pulls the current session so if the session had ended and another one started later I need to find a packet from the next session and follow that stream...

Is there a utility out there that will pull yahoo chats from these files in a nice format? Something like this:

user1: blah blah
user2: blah blah blah
...

I have dozens of these files with huge amounts of chat data that I want to archive.

Yeah OK. My first thought when you mentioned that you werent seeing all the packet contents was that you may not have captured all of the packet using tcpdump. Use of the snap length switch as you have done above should take care of that though.

Thx guys, tried out msgsnarf... not sure it's going to do it for me, so I'll check out the chaosreader too.

Here's what my problems are... up until around the end of June yahoo chats showed up with a protocol of YMSG. msgsnarf does pull data out of these files, but there are two problems - 1) it doesn't grab all of the chat exchange - ie, I can see more than what it pulls by looking via WireShark and 2) not a big deal, but it throws all kinds of junk chars around the texts of the chats.

The bigger problem is that around the end of June, the yahoo chats stopped showing up with a protocol of YMSG and instead show up as TCP. I can still check these out in wireshark, but I can't use the YMSG filter - I have to use the data contains filter listed in an earlier post. For caps with this kind of chat data, msgsnarf doesn't pull ANY of the chat activity out of the file

xplico looks really cool, but according to the status page it doesn't do yahoo chats yet:

w ww.xplico.org/status

(not being a dick - all help is super-appreciated! Just putting the info here in case others are trying to do the same

NetWorkminer is also badass - you should check it out, small exe, no install... not good for the chat extract I'm looking to do though.

(not being a dick - all help is super-appreciated! Just putting the info here in case others are trying to do the same

No its OK, I wouldn't have taken that personally

Originally Posted by ajf3ajf3

NetWorkminer is also badass - you should check it out, small exe, no install... not good for the chat extract I'm looking to do though.

Yes, Networkminer is on my list of tools to check out when I have time or a specific need. Its a long list so I haven't got around to it yet.

So it basically sounds like they have changed the chat protocol sometime during your monitoring period. There's no real requirement that Yahoo is trying to meet regarding interoperability so there's nothing to really stop them doing this unfortunately.

Depending on what data you want out of the chats you might have some luck in doing a straight string extraction (using the strings command) - but this may result in you missing out on various metadata in the communication. If you were after files (binary data) transferred using the chat client you could try tcpxtract.

Other than that though you may be stuck with writing your own parser. You may be able to find the spec for the chat protocol online somewhere or perhaps you may be able to reverse engineer this from the source of an open source chat client that can work with Yahoo.

Hope that helps somewhat. If you find a good method come back here and share the knowledge!

Capitalisation is important. It's the difference between "Helping your brother Jack off a horse" and "Helping your brother jack off a horse".