Preparing and deploying experiments on CONFINE testbed

Preamble

This tutorial is the fourth part of a series of four part tutorial. Many concepts and ideas shown here are described on previous parts. So, make sure that you has finished the third part of this tutorial before start this. Again, we assume that all the next commands will be executed from the ~/confine/utils/vct directory.

After completing this tutorial, we will be able to deploy and execute any experiment on CONFINE testbed (real or virtual). However, as CONFINE is under development, some topics may not be covered in detail in this tutorial. If you have more questions and/or suggestions, take a look at CONFINE Wiki webpage.

What have we missed?

Well…. we have some CONFINE research nodes configured, two with a transmission client and one more with a tracker. We tested them and now we know that are properly configured and run our experiment without problems. However, there are two basic points missed:

Scalability: We configured each node manually, who served us to verify our idea but is not exportable to larger (say, 1000 nodes) slices. we need to create a CONFINE sliver template sefl-deployable. Additionally, we have to include some mechanism to execute the experiment automatically without human intervention.

Logging: We ran our experiment, but we did not measure anything on it. We need some method to get information about nodes/network behavior, maybe in real-time.

We are going to address both problems in next sections.

Preparing the sliver template

One of CONFINE current limitations is that, on each sliver, a researcher can only allocate and deploy a single template. Is not posible to have connectivity from one slice to another, or deploy two different OS images inside the same slice. It is consistent with SFA concept of intra-experiments' connectivity and extra-experiments' isolation.

Technically speaking, a sliver template is nothing more, nothing less, than a filesystem directory structure that contains a complete OS. There are a lot of tutorials on Intenet that explains how to create this templates from scratch, but the fastest way to do it is taking advantage of our previous created slivers.

However, our fd02 and fd03 slivers are almost full. So, before start working we need a largest research node template - slices' size are only limited by nodes' free space. If you don't know how to generate it, we suggest you to visit our short tutorial at the end of this page.

We increased the fd01 node size up to 1G, enough for our purpose; but not too large.

Remember that all slices on your sliver shares the same template. So, you have to assure that you can find the desired number of CONFINE research devices with the enough space to allocate both templates: sliver plus experiment.

If at some point, it is impossible to reach the sliver try to stop it and start again (with the proper attributes update between each operation). It happens some times when the node has been stopped before stop the sliver.

In our case, we need to install wget, vim and transmission packages. The original file to be downloaded by clients will be configured on the experiment template instead been provided by the sliver or experiment template. It makes more sense, because is an experiment-specific configuration parameter (e.g. we can imagine that some researchers wants to reproduce the experiment using other kind of files, with different sizes or maybe test corrupted files) and we are saving slivers space.

Our sliver is now complete. It has an opentracker (that we will execute only on one sliver) and a transmission client that will start as a daemon by default on all slivers. It is time to create the template.

CONFINE slivers are simple linux containers filesystems, which means that can be packed and exported whenever you want simply compressing its rootfs directory as a tgz file. We will call our template debianbt32.tgz

Finally, you have to “download” it from the sliver to some accesible place. In our case, we uploaded the sliver on CONFINE's images server.

Preparing the experiment template

Experiment templates preparation are formally explained in details on Node system hack page. Here we are just desacribe our experiment preparation.

The experiment template is an archive file that is extracted on top its sliver root file system. The archieve should provide at least an init script and links to start the related init function in the corresponding OS format. Basic directory structure

~$ mkdir etc
~$ cd etc
~/etc$ mkdir rc.d init.d

(NEED MODIFICATIONS)

A researcher must provide an experimentation archive file that is extracted on top its sliver root file system. The archieve should provide at least an init script and links to start the related init function in the corresponding OS format. For example for OpenWrt this would be:

./etc/rc.d/S94confine-experiment (which is a link to ../init.d/confine-experiment)
./etc/init.d/confine-experiment
start() Function
stop() Function
During livetime of the sliver, all slice attributes could be accessed either in uci format or bash environment variables:

This example experiment simply pings the public IP of all other slivers of its slice and stores measured round trip times in the directory /root/confine/data/ . It contains of two files:

./etc/rc.d/S94confine-experiment (which is a link to ../init.d/confine-experiment)
./etc/init.d/confine-experiment

Running the experiment

Last step is to assemble all the pieces together and run our bitTorrent experiment. First of all, we need to modify some configurations on VCT. We are going to create a new file named vct.conf.overrides on vct directory to easely track the changes.

First line defines the research node template to use. As you can see, we are downloading a public CONFINE image as before, but with 1GB hard disk space. You can fins below how to create larger CONFINE research node images.

Next, we have been defined a new template based on previous debian hello-world experiment:

fs_template_url is a public bitTorrent sliver template available on our images server. You can use it if you want.

exp_data_url is the new experiment template.

exp_name is the new identification number for this experiment. It is not necessary to change this iD, but will help us to track the slice status.

If you followed the whole tutorial, probably you noticed that the slice creation process is straigh. So, it's a good idea to create your own slice creation script.

~$ vim create-slice.sh

It takes three arguments and allocates, deploys and executes the desired experiment.

Generating larger CONFINE research node images

Creating a new image

This method is simpler than the other, but it is also less flexible because you have to do it at the begining. The idea is generate a new CONFINE OpenWRT image, with the desired final space and use it as VCT user template.

We start, of course, with our SONFINE SDK. Go to confine directory and backup the configuration files.

Increasing previous image

This method (inspired by [1]) is prefereable if you already have a slice with something installed on it and you want to save some time. We are going to increase the size of a CONFINE node (named fd01) from 512MB to 1024MB.

First, stop the node.

~$ ./vct_node_stop fd01

VCT images are inside VCT_VIRT_DIR/images directory.

Notice that when images are stopped, VCT system returns the user and groups ownership to root user (you can change the user on vct.conf file). It is a good thing, because allow us make any change desired. ;)

Create a blank qemu file, this is the file we will be adding to our VM image. Create it with a size equal to the amount of extra disk space you want added to your VM. In our case we want to add an additional 512MB of space.

/var/lib/vct/images$ qemu-img create -f raw addon.raw 512M

This is what the addon file looks like after it is created.

VCT is very rigurous with image filenames, so we need to rename the original image file febore change it. After the whole process, we will delete all this extra files. Then, we wil combine both files into a single one.

The partitions, however, are not partitioned to take advantage of this extra space. We are going to fix-it loading a live gparted image (because we need to work from an unmounted filesystem) and resize the partition using that tool. First, download the live image and start our node using qemu tool.