Working with http requests in node.js (similar to curl)

Today I wanted to test out how to fetch data from one of the http APIs and save it to Mongo.

A quick example: get Twitter profile data for a given Twitter username.

In languages like PHP, we’d call curl and pass the URL. With node.js it’s actually really simple, we just call the GET method.

We are setting our request object that will call “GET” with our specified URL and wait to receive the response.

Request also has another method “on” that will listen for response data to come back (‘data’ event).

NOTE: It is not enough to just detect the ‘response’ event, as it looks like data coming back in “chunks”. I created a variable called “completeResponse” that concatenated all chunks into the final complete response (before I added that, my app would throw errors because of incomplete json that I tried running through JSON.parse()). Then wait for the ‘end’ event before actually saving the data into mongo.

Working with http requests in node.js (similar to curl)
ON JUNE 8, 2012 BY NTANYA54IN TECHNOLOGY
Today I wanted to test out how to fetch data from one of the http APIs and save it to Mongo.

A quick example: get Twitter profile data for a given Twitter username.

In languages like PHP, we’d call curl and pass the URL. With node.js it’s actually really simple, we just call the GET method.

We are setting our request object that will call “GET” with our specified URL and wait to receive the response.

Request also has another method “on” that will listen for response data to come back (‘data’ event).

NOTE: It is not enough to just detect the ‘response’ event, as it looks like data coming back in “chunks”. I created a variable called “completeResponse” that concatenated all chunks into the final complete response (before I added that, my app would throw errors because of incomplete json that I tried running through JSON.parse()). Then wait for the ‘end’ event before actually saving the data into mongo.

Detailed info on http request and this issue

After you get your final parsed data object, it’s ready to be saved into mongo.