Packets, pcaps, Python and Maltego

Projects

Disclaimer

This is my personal blog, all data and information provided on this site is for informational purposes only. The views expressed on these pages are mine alone and not those of my employer.

I will from time to time post something that might be slightly or massively inaccurate, this is not due to laziness but merely to the fact that I'm not perfect and let’s face it neither are you, otherwise you wouldn't be reading my blog (unless Google lied to you..).

I welcome all comments and emails, which are presented in a positive and constructive manner, however I withhold the right to delete or not publish any comments that I feel are "negative". After all if you are taking the time to read and then comment why not do it in a positive manner.

Site Admin

General

So it’s been a while since I’ve blogged anything not because I haven’t been busy (I’ve actually been really busy), but more because a lot of the things I work on now I can’t share (sorry). However every now and again I end up coding something useful (well I think it is) that I can share.

I’ve been looking at Domain Squatting recently and needed a way to codify whois lookups for domains. There are loads of APIs out there but you have to pay and I didn’t want to, so I wrote my own.

It’s a lightweight Flask application that accepts a domain, does a whois lookup and then returns a nice JSON response. Nothing fancy, but it will run quite happily on a low spec AWS instance or on a server in your internal environment. I built it to get around having to fudge whois on a Windows server (lets not go there).

In order to run the Flask application you need the following Python libraries (everything else is standard Python libraries).

Flask

pythonwhois (this is needed for the pwhois command that is used in the code)

To run the server just download the code (link at the bottom of page) and then run.

python whois-server.py

The server runs on port 9119 (you can change this) and you can submit a query like this:

Recently I was asked to see if I could create some Maltego transforms to provide a quick analysis of Netflow data. Always up for a challenge (and to feed my Maltego addiction) I created gotFlow, which is based on the Canari Framework (for rapid Maltego transform generation).

gotFlow is designed to support (currently) nfdump and should still be classed as an “early release” (meaning more to come). It’s a nice simple transform set with only 3 transforms, 3 entities and 1 Maltego machine.

The transforms process works as follows:

nfdump file -> source ip -> destination ip -> destination port

The source and destination IP’s are the Maltego IPv4 Address entities allowing you to run additional transforms against them.

To get started you can either add a single nfdump file or import nfdump files from a directory.

From here you can run the ‘[NF] – Import Files’ tranforms that will import all the nfdump files from the chosen directory.

Once that’s run you should (depending on the number of nfdump files) get something that looks like this.

You can now either run the Maltego machine against the files or run the transforms seperately. For the purpose of this blog post I’ve cheated and used the machine.

The Machine runs the following transforms, feeding off the return entities generated by the transform before it.

The thickness of the line between the source IP and destination IP is the size of the flow. The returned value is in bytes which I convert to kilobytes (bytes / 1000). If the line is thin (the default) it means its below 1 kilobyte.

The only configuration change you need to make before you run gotFlow is to define the location of the nfdump executable which needs to be added to:

The WordPress.com stats helper monkeys prepared a 2014 annual report for this blog.

Here’s an excerpt:

The concert hall at the Sydney Opera House holds 2,700 people. This blog was viewed about 28,000 times in 2014. If it were a concert at Sydney Opera House, it would take about 10 sold-out performances for that many people to see it.

Hello reader(s), I hope you are well and enjoying the onset of spring (here in the UK that means rain.. and lots of it)..

So a few weeks ago I released “Project Watcher” which were my wireless transform for Maltego. It was a bare bones release intended to get them out there and see what people thought. I’m pleased to say I’ve received no constructive feedback and as such I’m following the same motto as always.. “Sod you, I’m having fun ..”

While I was writing the first transforms one of the things that I found was it there weren’t any open source Wardriving databases with a nice HTTP based API, so I thought as the next stage of Project Watcher I would create one..

Today I finished the prototype of a what I hope soon to release to the public. Basically it will allow people to use a simple HTTP GET request to query the Watcher database and see if a wireless access point has been collected and stored. This is a “no frills” solution, there isn’t a web page to look at just an API (I’m not going to call it a RESTful API because it’s not, well not yet).

This was all new to me as I don’t code (other than python) but the API will use MongoDB and a Node.js front end to allow people to query the database. To be honest if it doesn’t work I will just turn it off, I’m running this at my own expense so it’s more about learning new things but if it takes off that would be awesome.

I’ve written some python that allows me to take the watcher.db (sqlite) database and import into MongoDB and my next piece of work is to write some code be able to import kismet files into the database via a web page (so you guys can have a go).

Longer term I hope to get a web site up to allow people to search and a couple of other bits that I’m keeping secret…🙂

Hello readers, I hope you are well and this blog post finds you all in good health and excellent spirits.. Well enough about you, this is my blog after all so on to me..🙂

The last few months have been challenging, my initial high of InfoSec learning and drive has seemed to dropped and instead I’ve been left with a sense of emptiness in terms of what and where to go next. If you remember I started this journey nearly 2 years ago with the sole purpose of doing more “security stuff” and overall I have to say I’ve achieved my goal. He’s a quick recap of what I’ve done (yeah I know I’m blowing my own trumpet but lets face it, if you could, you would).

OSCP – Done

OSWP – Done

Malware course – Done

SANS course – Done

Wrote some cool code (well I think it’s cool) – Done

Wrote the “Very Unofficial Dummies Guide to Scapy” – Done

Met some really cool people and even got to see a bit more of the world out of it – Done

So where to go from here?? A few people who I have a great deal of respect and time for suggested that instead of my scatter gun approach to learning I focus more on one or two areas, which to be fair makes perfect sense. The problem is on what, I needed to understand my “bliss”, the thing that you love the most and are passionate about. You know that thing that can consume hours of your time without you even realising (no not Christmas shopping).

It’s taken me weeks to work out what my “bliss” is, and in the end it turned out to be quite simple. Throughout my career I’ve built things, designed things, devised solutions to problems that other people have struggled with. One of my greatest assets is my imagination, my desire to learn new things and to push the boundaries of “the norm”. It’s what I enjoy, it’s my bliss.

So what does this mean, I hear you ask. Well throughout 2014 I’m going to take the 16 years infrastructure knowledge I have and the 2 years of InfoSec skills I’ve developed to build things. I have no idea what yet but with my new (and oddly strange) love for coding it’s more likely to be taking an idea that randomly pops into my head (very random at times) and turning it into something, always with a security twist. I want to see what focusing on creating things can lead to. I’ve already experienced it with my sniffMyPackets work, and I want to see what else I can do.

For me, that’s the true meaning of “hacker”, not these Hollywood hackers that take down systems with a single keystroke but someone who builds something, that can take an idea and make something out of it (whether it’s a bad idea or not), or takes an idea from someone else (giving full credit to the original creator) and tweaking it for new and interesting mischief.

I already have a few ideas locked away in the attic that is my brain and it’s time to dust off my IDE and start making things go boom (not really boom if you are reading this Mr NSA).

So if I don’t get a chance before, I wish you all a very merry Christmas/New Year etc etc. and may you all find your bliss in 2014.

So at the moment I’m working through Coursera’s “Malicious Software and its Underground Economy: Two Sides to Every Story” 6 week course. It’s actually quite good and I’m learning new things as well as reinforcing things that are stuck in my head somewhere.

The second lecture is all about Malware Static Analysis and they give you a quick overview on Assembly, which needless to say made my head hurt. So I reached out to my Twitter followers and asked for recommendations of books/videos/websites that will provide a “dummies intro to assembly”.

Like the awesome community this is they provided me with some awesome suggestions/links so I thought I would share:

So I don’t think I’ve ever done a rant blog post, and to be fair there is no real reason behind this I just started thinking about it on the way into work (which is about a 10 minute drive). Shall we begin??

DISCLAIMER: I apologise in advance for any bad language used during this rant or the excessive use of “”.

A couple of weeks ago I had reason to tell someone (over email) a little about myself in an attempt to “sell” myself. It’s not something I like doing but sometimes you just have to. It made me realise that during the last 18 months that I’ve been “trying to get into Security” that I’ve actually achieved a lot so I hope this rant will help people who are in the same situation as me.

Community is King???

This time 2 years ago I would spend most of my downtime playing computer games, call it a lack of motivation, laziness or whatever but that’s what I did, then with some gentle pushing from my nearest and dearest I decided to start using my time to learn and develop. When you start with the goal of “breaking into Security” many people point out that the key to success is “the community” and it’s true but that can be the hardest challenge. If you don’t work in Security then some people will tell you it’s just a hobby and maybe they are right or maybe that’s just bollocks, it’s for you to decide and ultimately turn it into anything you want.

I’ll let you into a secret, I started this blog for 2 reasons, the first was to keep a record of what I’ve done and allow me to pat myself on the back for the number of visitors I get, the second was because I wanted to get noticed, I hoped that over time people would read my blog, follow me on twitter and allow me into their circle of InfoSec friends and maybe if I was lucky I might end up with a job out of it. Then I realised something, and some people might disagree but its my blog not yours..

“You don’t have to work in Security, to be in Security”

Not really groundbreaking is it but it’s important because well it’s the point of this post. Over the last 18 months I’ve done a fair few bits and pieces for “the community” I’ve met some awesome people, done some awesome things and have even more awesome things on the horizon and 98% of that was from the community. If people tell me Security is just my hobby my first reaction is to tell them to “do one” because I have hobbies and they don’t consume the amount of time I put into projects, blogging, helping with events. Hobbies don’t consume your time like this does, they don’t push you to go further, learn more, make yourself better and give you that feeling that you can make a difference. This isn’t a hobby, it’s not my career either but doesn’t make it any less, its part of who I am and always will be.

So if you are just starting in Security and find yourself a little unmotivated because you can’t find that dream Security job or you are finding the community a bit “cliquey” here are my top tips:

1. Write it and they will come – Remember that awesome blog post (not this one) you read about the latest exploitation technique? Or that tool you used? Someone took the time to write that and then out of the goodness of their heart gave it away for free to YOU. Don’t you think it would be nice to repay the favour?? Seriously if you just start writing code, making videos or writing articles people will find them, share them and slowly over time you will find yourself more involved in the community than you ever expected.

2. Twitter isn’t just about your latest bowel movement – Follow people on twitter, it’s a good way to find people who post all that useful stuff you read. Interact with them by all means but remember this.

To start with they will probably ignore you, won’t follow you and generally see you as noise on their timelines, but give it time and slowly you will get there. I get more followers from Twitter from blog posts/code release than just by talking to people, and just accept that some people are very picky about following back or even replying back if you mention them in Tweets.

3. You’re never alone – In the UK there aren’t a lot of conferences, CTF events and only limited events, if there isn’t anything in your area then start something, you want to be part of the community then sometimes you have to make it happen. If you want to organise a monthly Security focused meeting in your area then do it, don’t let people tell you can’t, because well you can. Even if only 1 other person turns up that’s 1 person you didn’t know who shares the same interests as you (unless it’s your mum).

4. It’s up to you – If you want to make Security just a hobby, then that’s fine. If you want to make it a career that’s awesome but it’s up to you to decide and more importantly it’s up to you to make it happen. Don’t let other people label what your passions, dreams or ambitions are, they are yours and no one elses.