Now if you need to start from the get go start reading here again as I will explain exactly what this thing does. If you are new to Node.js then please read my article here before continuing as it might help with the above syntax and understanding Node.js better.

To run this simply put it into a text file and run it by running: node <filename>.js

Pre-requisites

Before running this there are a few commands you should run first. These are to install the needed modules for this code, they are pretty self-explanatory. They are:

1

2

3

npm install ntwitter

npm install util

npm install mongodb

You will also need to setup a Twitter account with an application registered for this. If you are unsure of how to do this read this document. It’s a great intro that I found when doing mine since it is not really that intuitive so maybe Twitter will make this easier to find in the future.

Notes on MongoDB Hosting

If you don’t want to host your own MongoDB instance for whatever reason I will give a huge recommendation on a hosting provider that I use. It is called MongoLab and these guys are awesome. They give you a free instance for up to 512MB of stored data and in using them for a few months they are very reliable with no downtimes so far. Plus their editor works well for seeing JSON data as will be inserted by this program.

The Code Broken Down and Dissected

Now let’s do a break down on the code itself and explain what each section does.

This is pretty easy to look at and see what it does. It creates an instance of MongoClient from the just installed module and tries to connect with it. Makes sure you change my URL string to whatever it is for yours. If it fails then it will throw an error for you to see on the command line. Next it tells the Mongo module which collection to use (if you are new to Mongo this is a rough equivalent to a database from the SQL world). You will get this from your Mongo instance.

JavaScript

1

2

3

4

5

6

vartwitter_client=newtwitter({

consumer_key:'AAAAAAAAA',

consumer_secret:'AAAAAAAAAAA',

access_token_key:'AAAAAAAAAAAA',

access_token_secret:'AAAAAAAAAAA'

});

This is the part where we connect to Twitter. You will need to log into Twitter and find the application you registered earlier and on that page is a list of keys you will use above. Just match up the labels with the keys in the code. Simple as pie (well sometimes pie isn’t that easy).

JavaScript

1

2

twitter_client.stream('user',function(stream){

stream.on('data',function(data){

The first line of this initializes a user stream from a Twitter account and registers a callback function. The second line sets up an event-handler for when data is received from the Twitter stream and registers a callback function passing the data from the tweet into this function.

JavaScript

1

2

3

4

5

collection.insert(data,function(err,docs){

collection.count(function(err,count){

console.log(util.format("count = %s",count));

});

});

The first line of this code inserts the data (which comes from Twitter as JSON) into your Mongo collection and registers a callback function. The code inside of that callback function simply counts the number of records in the collection and spits it out to the console. The nesting of multiple callbacks can get confusing, but after writing a few it becomes pretty easy to see and if you use a good editor it will show the indentation and syntax coloring for you. I use vim as an editor, but there are plenty more out there.

Conclusion

This was a very simple example of how to insert tweets into a MongoDB collection. Just be warned if you run this for any extended amount of time it will take up a LOT of space. For example I inserted 124 tweets and that takes up about 360KB. So just scale that and you can see it will take up a ton of space if you let it go for an extended amount of time. I recommend breaking down the JSON and inserting only what you need if you want to run this for any significant amount of time. I will probably touch on a way to do this in a future posting once I get a design created and some code created for it.