Using apache-libcloud to provision cloud servers

Apache’s libcloud is a Python library that allows you to write code that interacts with numerous cloud service providers. From the big players like Amazon AWS, Google and Rackspace to even smaller local providers like the New Zealand based RimuHosting. A full list of supported providers can be found here

The library is not limited to provisioning servers and supports a bunch of things like load balancing, storage and even DNS but here we’ll just focus on cloud server provisioning.

Installation

I am assuming you are running Ubuntu or another Debian variant. if not check official installation instructions here. Normally installation is as easy as:

sudo apt-get install python-pip
sudo pip install apache-libcloud

Getting started

The library is designed to provide a common interface to the different cloud providers, the general process is.

Choose a provider

Request a Driver for that provider

Set credentials

Run operations

Continue or complete

As an example, if I wanted to see the EC2 instance sizes that were available in Sydney I could do the following.

Another helpful method is list_images() which returns a list of all the OS images that are available for our current driver (in this case just EC2 AMI’s in Sydney). Simply adding the following to the bottom of our script and running it again

images=driver.list_images()forimageinimages:printimage

Will output a the size list then a big list of the available node images like following (trimmed)

One thing to note, is that these images and sizes are objects we need to pass through to the create methods, so if you know the size of Node you want unfortunately to make the library portable you cant just parse in the ID, you still need to get the NodeSize object you are after from the list, for example if you knew you wanted to create a ‘m1.small’ instance you’d have to do something like this:

The same is true for your host’s NodeImage so lets update our script from before to specify a NodeSize and a NodeImage.

For this example I am going to use an AMI for an official Ubuntu Instance image in Sydney. This is for m1.small instances and is available in the Sydney pool of AMI’s. ami-934ddea9. You can see all the official Ubuntu ones here.

This script will create a EC2 small instance, in Sydney, add the Public key to the authorized_keys file for root, and make sure that port 22 is open for connection.

You can now SSH into your server, check the AWS EC2 console for the publicly assigned DNS name to connect to.

ssh -l root ec2-54-252-210-134.ap-southeast-2.compute.amazonaws.com

NOTE: since we are running Ubuntu, you can’t SSH into the new box as root you need to use the user account ubuntu. If you do login with root you will be instructed about this.

So there we have it. Apache libcloud makes it really easy to provision servers.

There is a whole bunch of additional functionality we haven’t covered here. Executing commands on the server during provisioning for installation of software, adding public keys via the library and running other arbitrary shell commands. If you’re interested in knowing more feel free to leave a comment.