Another Great Example of AWS Fidelity – Neo4j, Cloud-Init and Eucalyptus

I recently ran across a blog entry entitled Neo4j 1.9.M01 – Self-managed HA. I found the concept of graph databases storing data really interesting and reached out to the guys at Neo4j to get some insight on how to deploy their HA solution on Eucalyptus. Amongst the resources that they provided, they shared this little gem – how to deploy Neo4j on EC2. In order to run first, you need to know how to walk – so before going down the path of standing up HA Neo4j, I decided to be influenced by the DIY on EC2 article provided by Neo4j and deploy Neo4j on Eucalyptus – with a little help from Cloud-Init. The follow-up blog will show how to use the same setup, and deploy an HA Neo4j environment.

Ubuntu Cloud Image – 12.04 LTS Precise Pangolin

Ubuntu cloud images come pre-packaged with cloud-init, which helps with bootstrapping the instance.

I wanted to have the solution work on AWS EC2 and Eucalyptus; since Ubuntu cloud images work on both, its a great choice.

Registering the Ubuntu Cloud Image with Eucalyptus

In order for us to get started, we need to get the Ubuntu Cloud image into Eucalyptus so that we can use it for our instance. To upload, bundle and register the Ubuntu Cloud image, ramdisk and kernel, do the following:

Cloud-init Config File

Now that we have the image ready to go, we need to create a cloud-init config file to pass in using the –user-data-file option that is part of euca-run-instances. For more examples of different cloud-init files, please refer to the cloud-init-dev/cloud-init repository on bazaar.launchpad.net. Below is the cloud-init.config file I created for bootstrapping the instance with an install of Neo4j, using ephemeral disk for the application storage, and installing some other packages (i.e. latest euca2ools, mlocate, less, etc.). The script can be also accessed from github as well – under the eucalptus/recipes repo.

Next, we need to create a security group for our instance. To create a security group, use euca-create-group. To open any ports you need for the application, use euca-authorize. The ports we will open up for the Neo4j application are SSH (22), ICMP, HTTP( 7474), and HTTPS (7473).

Create security group:

# euca-create-group neo4j-test -d "Security for Neo4j Instances"

Authorize SSH:

# euca-authorize -P tcp -p 22 -s 0.0.0.0/0 neo4j-test

Authorize HTTP:

# euca-authorize -P tcp -p 7474 -s 0.0.0.0/0 neo4j-test

Authorize HTTPS:

# euca-authorize -P tcp -p 7473 -s 0.0.0.0/0 neo4j-test

Authorize ICMP:

# euca-authorize -P icmp -t -1:-1 -s 0.0.0.0/0 neo4j-test

Finally, we use euca-run-instances to launch the Ubuntu Precise image, and use cloud-init to install Neo4j:

Thats it! The cool thing about this too, is that you can find an Ubuntu Precise AMI on AWS EC2, use the same cloud-init script, use euca2ools, and follow these instructions to get the same deployment on AWS EC2.

As mentioned before, the follow-up blog will be how to deploy the HA solution of Neo4j on Eucalyptus. Enjoy!