More prosaically, the high performance asynchronous design of node.js and the tunable latency/consistency of MongoDB mean a high throughput application can be assembled in a compressed timeframe and with standard tools you probably have laying around the home. Since MongoLab runs managed hosted MongoDB instances on Joyent’s Cloud near a node.js SmartMachine, you get world-class operation of both environments.

Below, we’ll take a quick spin setting up a MongoLab database and a no.de account. We’ll build a minimalistic Web Server that can do some data inserts and queries and display it through a gratuitous 3D guestbook demo.

For the impatient

Sign up at mongolab.com and create a MongoLab database on Joyent Cloud and note database name,hostname, port, database username/password

For the really impatient

What is MongoDB?

First a quick word about MongoDB for the newly initiated. MongoDB is a non-relational database system that emphasizes horizontal scale across multiple servers, tunable consistency, and high performance. Being a document-database, it uses JSON notation to describe data and sports a rich query language with indexes to enhance query speed. It also has a map-reduce framework for more intense data analysis and transformation. There is growing adoption of MongoDB for large stores of documents like in a Content Management System or in data analytics, for feature-rich Web 2.0 sites and games, and for persistent stores for mobile applications. Its code is open source licensed under the Gnu AGPL v3.0 and is commercially licensed from its author, 10Gen. Large corporations and smaller outfits are using MongoDB in production today. New users, you are in good company.

MongoLab and Joyent

MongoLab is a cloud database service specifically for optimized production of and development on MongoDB. MongoLab hosts instances in Joyent’s SmartMachine cloud for high performance connectivity to SmartMachines and no.de. MongoLab takes care of operational tasks of MongoDB like replication, backup, and redundancy. Precious development hours can be focused on unique features and less focused on unproductive database maintenance. MongoLab has unique ad hoc Web query and data manipulation tools that inspect and manipulate data easily. Of course, standing up a MongoDB instance on a SmartMachine is also straightforward. Free accounts up to 240MB at MongoLab make a compelling case to get started quickly. MongoLab makes using MongoDB as easy as belting out a U2 tune in the shower, or a Katy Perry one if you’re into that.

The node.js Appliance is practically instantaneous for launching development, test, and production servers running node.js. Joyent also hosts a free (as in beer) node.js little brother (128MB of RAM little) at no.de. You can get started in experimental development and try out new design vignettes with no cost or risk. Joyent’s SmartMachine infrastructure that underpins the Appliances and no.de is a beauty of a design, having been built on OpenSolaris and decades of real-world stress testing. Cloud Analytics and the magical DTrace mean you can find bugs and anomalies with minimal thrashing.

Don’t fret if you don’t have a browser that supports the right version of WebGL. The 3D stuff is pure eye-candy. It's gratuitous! The main idea can be demonstrated with any browser and two URLs detailed below.

If you do have a WebGL browser there's an instance of the 3D guestbook running on the Joyent Cloud: nodejs.mongolab.com The server is a node.js instance that returns some static content and also handles two dynamic requests: One for querying a MongoLab database and another for inserting new messages.

The client side uses the GLGE Library to render a 3D scene and animate some scrolling text. The camera view can be rotated by dragging the mouse across the canvas. The text is pulled from a URL mongolab.no.de/getentry. Each request returns a different message from the MongoLab database. If you don't have WebGL, you can point your browser right to that directory and get the next entry.

The URL mongolab.no.de/addentry?entry=testmessage does some validation, truncates to 20 characters and inserts it into a MongoLab database for later recovery. That also works in an non-WebGL browser.

Back in the 3D interface, the text field in the front of the canvas lets a viewer of the scene to sign the guestbook with a 20 character message by invoking the /addentry URL.

MongoLab setup

Fill out the signup form. Note that this “first” account name and username will allow you to create multiple databases on MongoLab. Each database will have its own users independent of the MongoLab account username.

Click Add for a Database

Select “JoyentCloud” for the Cloud Provider and the Free Plan for 240MB of space.

Fill out a database name. Pick something unique. The system will tell you if there's another database with that name.

The DB username and DB password are the “second” credentials and these are used by the driver to connect to the database. Be sure to note them down.

Click on the row that depicts your new database. You'll see a box that has the exact command line and URI info to connect to your database. Note them down.

Great, you’re set on the MongoLab side. The next steps populate the database with a sample entry. They're optional.

Now click Add in the Collections section.

Name this new collection “guestbook” in the popup and click Create.

In the new row under “Collections” click on “guestbook” and well now add a couple of guestbook entries.

Click Add under the Documents / Objects section.

Type “entry”:”hello” in the curly brackets such that you get ‘{“entry”:”hello”}’ in the large black input field. Click “Create and go back”.

No.de setup

Onto signing up at no.de. Remember that no.de is a free service for development. Joyent’s public cloud SmartMachine node.js appliance is what you’ll want for more capable deployments (available at https://my.joyent.com/)

Click “Order a Machine”, provide a unique machine name, click Provision and one will be provisioned for you in a few minutes. The Status column will go from “provisioning” to “running” when it’s up and running.

While you’re waiting for the status column to change, click on the name of your machine and you’ll get some instructions to add the following to your $HOME/.ssh/config file. It’ll look something like this:

Once it’s running, you should “ssh” into it by typing ssh <your no.de machine>.no.de SSH is important because we’ll need it to move the code from your workstation to the no.de SmartMachine instance. You’ll see a warning that your machine hasn’t ever connected to that server and the fingerprint is unknown. That’s expected only the first time you connect to the machine. You can type “yes” to accept.

Making config.js changes

config.js in the root of the git repository demo-node-01 is where you’ll put the info we collected from the MongoLab database creation. There's a snippet below. The exports.databaseUrl should look like “dbh.mongolab.com:27007” and are chosen by the MongoLab system. The exports.databaseName, username, and password are provided by you when you created the database. Yes, the username and password are the “second” ones. Save your changes to disk.

High level overview

Before we commit the changes to the no.de server, let’s look at the high level stack here. At the top is the 3D browser client in Javascript that calls to the no.de instance running node.js. That node.js instance runs the server.js program from our git repository that itself connects to a MongoLab hosted database running on a Joyent SmartMachine.
The node.js server connects to a MongoLab instance running in Joyent’s cloud and makes queries and insert requests.
Node.js itself is covered in detail nodejs.org. Download and installation instructions are there if you want to get them to run on your own workstation. But for the rest of this discussion, we’ll be wallking through a free no.de instance.

server.js major functions

Diving in deeper into the server.js running in node.js, the major functions are:

The server queries the MongoDB database and sends back the result to the client. The server also tracks a little meta-data to keep remember what guestbook entry was most recently sent. Clients would see a different entry most of the time. Yes, this is not idempotent, but we’re not looking for pure REST semantics; we’re making a demo not a banking app.

The server inserts new guestbook entries. On insert requests, the server does a little data validation and cleansing before connecting over the MongoDB wire protocol to do an insert into the database.
Let's take a look at server.js internally. There are three steps that the server goes through when it's loaded.

The first is to parse a single command line option which allows you to override the default network port (port 80). That’s useful if you’re running it locally and don’t have / want authority to bind to those system level ports. Be aware if you fiddle with the port number that there may be firewall issues in your network that block non-standard ports.
The second step is to initialize the connection to MongoLab. The third step is to start the server listening on the designated port. Most of the work is done in a function called dispatch() each time a new request is made.

MongoLab connectivity

In initializeDb(), you can see the configuration parameters, the database’s network hostname and the authorized username and password, are pulled from the file config.js. The file is ‘require’d in the preamble of server.js.

Note there are programming language specific drivers for connecting to MongoDB. We use an enhanced driver package called mongoose that works with node.js. As a bonus, it provides a measure of object-document mapping prettiness wrapped around the driver, even if we don’t use those features in our demo.
The package.json file specifies to no.de that we need mongoose to run the demo. If you want to get mongoose directly, there’s an installation program called “npm” which you can remember as “node package manager” even though it stands for something slightly different and less memorable. npm can be found at npmjs.org.

dispatch handler

In the third step to start the http server in startServer(), where the serverCallback gets called when a new http message is received from the network. In turn the dispatch() routine is the called that handles the two special URL paths “/getentry” and “/addentry”. Failing that, it sees if the path is one of the allowed filenames. allowedFiles is also a configuration entry in config.js. If the pathname matches then the file is served to the requesting client.
If the path was “/getentry” the server makes a query to the database and returns the next guestbook entry (from most recent to oldest) to the client. Before it does this, it sees if the guestbook had any new entries added since the last query, in which case, it returns the newest and starts counting down from the top again.

Stop for a moment to look at the query. The call to query() has as parameters the name of the collection: “guestbook” followed by the query: {“entry”:{“$exists”: true}}. A collection is a MongoDB database’s set of documents. The documents in a collection get searched and indexed as a group. The query is just a JSON document with the name of the field: “query” being used to test for a filter: {“$exists”: true}. There’s a nice set of query predicates in MongoDB: conditional operators, regex’s, array matches, etc. Details can be found here: http://www.mongodb.org/display/DOCS/Querying
If the path was “/addentry”, the server looks for a query parameter “entry=” and after doing some data validation, inserts it into the database. The inserted document in our demo is a trivial key-value: {“entry”:<data>}. In a more real-world use case, the entire rich, nested constructs of JSON would be used.

Pushing code to no.de

So we’ve made the necessary changes to the config.js file and we now can push the code to the no.de instance. We’ll use the git command line below. If you’re using a visual git tool, the steps are similar. Git uses ssh to authenticate from your workstation to the no.de SmartMachine instance, so again it’s important that it’s set up properly, including the .ssh/config file above.
If you haven’t used git before, it’s a distributed source code manager and it has become popular for deploying to remote cloud machines. The basic idea is to push a local repository from your workstation to the remote Joyent cloud repository.
So first we have to commit the config.js changes to the file to the repository:
git commit -a -m “Updated config.js.”
Next we have to tell your local workstation to set up the remote repository.
git remote add mongolabdemo <your no.de machine>.no.de:repo
That creates a remote alias ‘mongolabdemo’ that points to your new no.de SmartMachine in a directory ‘repo’. The directory name “repo” is special because there are scripts expecting your code to be dropped there.
git push mongolabdemo master
That pushes the code to the alias you created. Master is a special git keyword meaning the latest changes. You should see something like the screenshot below.

Debugging

You can point your browser to <your no.de machine>.no.de and the 3D screen should appear with scrolling guestbook entries. Remember to add your own entries in the text field. Congrats!
Now if things don’t go exactly well, there are a few places you can look to debug. If you’re ssh’d into the no.de server, there are a number of script commands that are helpful. They’re highlighted in the greeting when you first ssh in. Notably node-service-log connects your ssh session to the console.log output of the node.js server. Run it and you should see “http server open for business.” if the http server starts successfully. And “Connected to MongoDB successfully!” appears if the database connection is alive. Once you point your browser to the URL, the server emits log entries for each URL request.
You should also be able to point your browser directly to the URL http//:<your no.de machine>.no.de/getentry and get back one of your guestbook entries. If you’re getting something else, check that the database credentials and hostname are correct in config.js.
Each time you make changes to the code, do a
git commit -a -m “<your new message>”
and a
git push mongolabdemo master
to update the no.de instance.

Conclusion

So that was a whirlwind tour of creating an http server on Joyent’s node.js SmartMachine service with dynamic data from MongoLab. We didn’t review any of the 3D client side code, which is in the webroot subdirectory. There are a lot of interesting things that could be done on the client side. It could be a mobile or tablet application, or a rich HTML5 client (if you peek in the webroot code, you’ll see some jQuery!). I hope that you learned a bit about node.js and MongoDB and how to use them in the cloud. In the process maybe the gratuitous 3D client sowed some creative seeds in the the fertile terraced valley of your mind. I’m hoping to seeing your magum opus on no.de and mongolab.com!