Sunday, April 3, 2011

This tutorial provides step-by-step instructions on how to install Xen 4.0 on a Debian Squeeze (6.0) system (AMD64) and create paravirtualized guests (don't mix this up with fully virtualized guests, i.e. hardware virtualization (HVM)).
Xen lets you create guest operating systems (*nix operating systems like Linux and FreeBSD), so called "virtual machines" or domUs, under a host operating system (dom0). Using Xen you can separate your applications into different virtual machines that are totally independent from each other (e.g. a virtual machine for a mail server, a virtual machine for a high-traffic web site, another virtual machine that serves your customers' web sites, a virtual machine for DNS, etc.), but still use the same hardware. This saves money, and what is even more important, it's more secure. If the virtual machine of your DNS server gets hacked, it has no effect on your other virtual machines. Plus, you can move virtual machines from one Xen server to the next one.
I do not issue any guarantee that this will work for you!

1 Preliminary Note

I'm using a Debian Squeeze system (x86_64) with the hostname server1.example.com and the IP address 192.168.0.100 as the host system (dom0). (The setup might differ slightly if you are on an i386 system.) I will use Debian Squeeze for the virtual machines (domU) as well (but also show the changes you need if you want to install an Ubuntu Maverick guest).
This guide will explain how to set up image-based virtual machines and also LVM-based virtual machines.

2 Installing Xen

Afterwards we open /etc/modules and make sure that we have the line loop max_loop=64 in it (this step is needed only if you want to create image-based virtual machines - you can skip it if you want to create LVM-based virtual machines):

vi /etc/modules

[...]
loop max_loop=64

Next we open /etc/xen/xend-config.sxp...

vi /etc/xen/xend-config.sxp

... and uncomment/add the line (network-script 'network-bridge antispoof=yes') and comment out all other (network-script ...) lines (antispoof=yes enables the Xen firewall which prevents that a VM can use an IP address that it is not allowed to use, like, for example, your gateway IP). Also make sure that the line (vif-script vif-bridge) is enabled:

... this means you are just running on a Xen-ready kernel, but not the Xen hypervisor itself (which means you will not be able to boot virtual machines). To change this, change the boot order of your kernels as follows and update the GRUB bootloader:

mv /etc/grub.d/10_linux /etc/grub.d/50_linux
update-grub2

This makes sure that the next time you boot, the Xen hypervisor gets loaded.
Reboot again:

3 Creating Image-Based Virtual Machines

We will use xen-tools to create virtual machines. xen-tools make it very easy to create virtual machines - please read this tutorial to learn more: http://www.howtoforge.com/xen_tools_xen_shell_argo. We've already installed xen-tools in the previous step (chapter 2).
Now we edit /etc/xen-tools/xen-tools.conf. This file contains the default values that are used by the xen-create-image script unless you specify other values on the command line. I changed the following values and left the rest untouched:

The dir line specifies where the virtual machine images will be stored. Make sure that at the same time, lvm is commented out (these settings are mutually exclusive!). dist specifies the distribution to be installed in the virtual machines (if you run the command

xt-guess-suite-and-mirror --suite

on the command line, you will see that it translates to squeeze, so in this case Debian Squeeze would be installed unless you specify anything else on the command line).
To find out which distributions you can install in a virtual machine, run:

The passwd = 1 line makes that you can specify a root password when you create a new guest domain.
The kernel and initrd lines specify the kernel and ramdisk that get installed in a virtual machine. If you want to install Debian Squeeze, please use the settings as shown above (otherwise the Debian Squeeze guest might not boot). If you want to install Ubuntu Maverick, for example, you can use the same settings, or you comment out both lines, in which case the default Ubuntu kernel would be used (yes, Xen 4.0 allows you to use non-Xen kernels in a guest!).
Make sure you specify a gateway, netmask, and broadcast address. If you don't, and you don't specify a gateway and netmask on the command line when using xen-create-image, your guest domains won't have networking even if you specified an IP address!

The mirror line specifies the mirror to use (the command

xt-guess-suite-and-mirror --mirror

translates to a Debian mirror by default. Of course, you can specify another mirror, e.g. as follows:

mirror = http://ftp.de.debian.org/debian/

ormirror = http://archive.ubuntu.com/ubuntu (for Ubuntu; in this case make sure that you specify an Ubuntu version in the dist line, e.g. dist = maverick)
)
It is very important that you add the line serial_device = hvc0 because otherwise your virtual machines might not boot properly! disk_device = xvda makes that the virtualized disks are named xvda1, xvda2, etc. This is the correct setting for a Debian Squeeze guest; Ubuntu guests, especially when using a non-Xen kernel, might not boot with this setting, but you can override it with the --scsi switch that you pass to the xen-create-image command - in this case the virtualized disks will be named sda1, sda2, etc.
To summarize, the above settings are perfect for Debian Squeeze guests; if you want to install Ubuntu Maverick instead, you will have to override some of the settings in /etc/xen-tools/xen-tools.conf (--dist, --mirror, --scsi switches; maybe also comment out kernel and initrd) - I'll come to that in a moment.
Before we go on, we must create the directory where the virtual machine images should be stored:

mkdir /home/xen

Now let's create our first guest domain, xen1.example.com, with the IP address 192.168.0.101:

Options that you specify on the command line override the settings in /etc/xen-tools/xen-tools.conf. Options that are not specified on the command line are taken from /etc/xen-tools/xen-tools.conf. Please make sure that you add --role=udev, or your virtual machine might not boot properly!
(To learn more about the available options, take a look at the xen-create-image man page:

man xen-create-image

)
The xen-create-image command will now create the xen1.example.com virtual machine for us. This can take a few minutes. The output should be similar to this one:

(To use the default Ubuntu kernel instead of Debian's Xen kernel in the guest, you can also comment out the kernel and initrd lines in /etc/xen-tools/xen-tools.conf.)
There should now be a xen1.example.com configuration file - /etc/xen/xen1.example.com.cfg. Take a look at it to become familiar with virtual machines configuration files:

to log in on that virtual machine (type CTRL+] if you are at the console, or CTRL+5 if you're using PuTTY to go back to dom0), or use an SSH client to connect to it (192.168.0.101).
To get a list of running virtual machines, type

If you want xen1.example.com to start automatically at the next boot of the system, then do this:

mkdir /etc/xen/auto
ln -s /etc/xen/xen1.example.com.cfg /etc/xen/auto

Here are the most important Xen commands:xm create -c /path/to/config - Start a virtual machine.xm shutdown - Stop a virtual machine.xm destroy - Stop a virtual machine immediately without shutting it down. It's as if you switch off the power button. xm list - List all running systems.xm console - Log in on a virtual machine.xm help - List of all commands.
A list of all virtual machines that were created with the xen-create-image command is available under

4 Creating LVM-Based Virtual Machines

This chapter explains how you can set up LVM-based virtual machines instead of virtual machines that use disk images. Virtual machines that use disk images are very slow and heavy on disk IO.
In this example I'm using a Debian Squeeze host with the LVM volume group /dev/vg0 that has about 500GB of space. /dev/vg0 contains two logical volumes, /dev/vg0/root and /dev/vg0/swap_1 that consume about 100GB of space - the rest is not allocated and can be used to create logical volumes for our virtual machines:

We will use xen-tools to create virtual machines. xen-tools make it very easy to create virtual machines - please read this tutorial to learn more: http://www.howtoforge.com/xen_tools_xen_shell_argo. We've already installed xen-tools in chapter 2.
Next we edit /etc/xen-tools/xen-tools.conf. This file contains the default values that are used by the xen-create-image script unless you specify other values on the command line. I changed the following values and left the rest untouched:

Make sure that you uncomment the lvm line and fill in the name of your volume group (vg0 in my case). At the same time make sure that the dir line is commented out (these settings are mutually exclusive)! dist specifies the distribution to be installed in the virtual machines (if you run the command

xt-guess-suite-and-mirror --suite

on the command line, you will see that it translates to squeeze, so in this case Debian Squeeze would be installed unless you specify anything else on the command line).
To find out which distributions you can install in a virtual machine, run:

The passwd = 1 line makes that you can specify a root password when you create a new guest domain.
The kernel and initrd lines specify the kernel and ramdisk that get installed in a virtual machine. If you want to install Debian Squeeze, please use the settings as shown above (otherwise the Debian Squeeze guest might not boot). If you want to install Ubuntu Maverick, for example, you can use the same settings, or you comment out both lines, in which case the default Ubuntu kernel would be used (yes, Xen 4.0 allows you to use non-Xen kernels in a guest!).
Make sure you specify a gateway, netmask, and broadcast address. If you don't, and you don't specify a gateway and netmask on the command line when using xen-create-image, your guest domains won't have networking even if you specified an IP address!
The mirror line specifies the mirror to use (the command

xt-guess-suite-and-mirror --mirror

translates to a Debian mirror by default. Of course, you can specify another mirror, e.g. as follows:

mirror = http://ftp.de.debian.org/debian/

ormirror = http://archive.ubuntu.com/ubuntu (for Ubuntu; in this case make sure that you specify an Ubuntu version in the dist line, e.g. dist = maverick)
)
It is very important that you add the line serial_device = hvc0 because otherwise your virtual machines might not boot properly! disk_device = xvda makes that the virtualized disks are named xvda1, xvda2, etc. This is the correct setting for a Debian Squeeze guest; Ubuntu guests, especially when using a non-Xen kernel, might not boot with this setting, but you can override it with the --scsi switch that you pass to the xen-create-image command - in this case the virtualized disks will be named sda1, sda2, etc.
To summarize, the above settings are perfect for Debian Squeeze guests; if you want to install Ubuntu Maverick instead, you will have to override some of the settings in /etc/xen-tools/xen-tools.conf (--dist, --mirror, --scsi switches; maybe also comment out kernel and initrd) - I'll come to that in a moment.
Now let's create our first guest domain, xen1.example.com, with the IP address 192.168.0.101:

Options that you specify on the command line override the settings in /etc/xen-tools/xen-tools.conf. Options that are not specified on the command line are taken from /etc/xen-tools/xen-tools.conf. Please make sure that you add --role=udev, or your virtual machine might not boot properly!
(To learn more about the available options, take a look at the xen-create-image man page:

man xen-create-image

)
The xen-create-image command will now create the xen1.example.com virtual machine for us. This can take a few minutes. The output should be similar to this one:

(To use the default Ubuntu kernel instead of Debian's Xen kernel in the guest, you can also comment out the kernel and initrd lines in /etc/xen-tools/xen-tools.conf.)
As you see from the output, xen-create-image has created a new logical volume for our VM in the vg0 volume group, /dev/vg0/xen1.example.com-disk, for the VM's root filesystem. Take a look at

lvdisplay

and you will see that it has also created a second logical volume, /dev/vg0/xen1.example.com-swap, for the VM's swap:

There should now be a xen1.example.com configuration file - /etc/xen/xen1.example.com.cfg. The disk line contains physical devices (the two logical volumes created by xen-create-image) instead of disk images:

)
(Please note: if you have a dual-core or quad-core CPU and want the virtual machine to use all CPU cores, please change the vcpus line to vcpus = '2' or vcpus = '4'.)
To start the virtual machine, run

to log in on that virtual machine (type CTRL+] if you are at the console, or CTRL+5 if you're using PuTTY to go back to dom0), or use an SSH client to connect to it (192.168.0.101).
To get a list of running virtual machines, type

If you want xen1.example.com to start automatically at the next boot of the system, then do this:

mkdir /etc/xen/auto
ln -s /etc/xen/xen1.example.com.cfg /etc/xen/auto

Here are the most important Xen commands:xm create -c /path/to/config - Start a virtual machine.xm shutdown - Stop a virtual machine.xm destroy - Stop a virtual machine immediately without shutting it down. It's as if you switch off the power button. xm list - List all running systems.xm console - Log in on a virtual machine.xm help - List of all commands.
A list of all virtual machines that were created with the xen-create-image command is available under