Wednesday, January 30, 2013

RHEV upgrade saga: Creating VMs on Open vSwitch

In last week's post, we discussed how we created our network by integrating Open vSwitch into RHEL KVM.
Now we need to create some virtual machines to run the workloads. (VMs
are required to run within a virtual environment, so we need an easy way
to create them.) Once more we will approach this from running on a RHEL
6 and RHEL 5 box, as the steps are somewhat different.

The libvirt that comes with stock RHEL 6
(and RHEV actually) is version 0.90.10-21, which, lucky for us,
contains support for Open vSwitch, however the libvirt for RHEL 5 is
version 0.8.2, which does not contain support for Open vSwitch. This
means that for RHEL 5 we have to take some extra steps to manage our
networks and implies that we can't use virt-manager to create our VMs.
It also means on RHEL 5 that we can't import our Open vSwitch networks
into virsh to make using virt-manager and other tools easier.
Even so, I feel that libvirt v1.0.1 is a better way to go, so I
downloaded the source RPM from libvirt.org and rebuilt it on my RHEL 6
machine. This did require me to rebuild libssh2 (needed >= v1.4) and
sanlock (needed >= v2.4) to get the proper versions of those tools to
support libvirt 1.0.1.

While this upgrade works for RHEL 6, it will NOT work on RHEL 5 as it
would require installing so many new packages that it is far easier to
just upgrade to RHEL 6. So if you are using RHEL 5, you should continue
down the path to use libvirt 0.8.2.
Without a tool to manage multiple KVM nodes, it is very hard to do a
rolling upgrade of libvirt. I am still looking for a good tool for this.
RHEV may be the only usable interface, but I could also use OpenStack
-- a discussion for another time.

For RHEL 6

Once libvirtd has been restarted, we can import our networks into
libvirt for use, to do that we need to write a proper libvirt network
XML file. Here is the one I used named ovsbr1.xml

ovsbr1

The key lines are the name of the bridge, the virtualport type, and
portgroup. While I do not use VLANs, we want to make a default portgroup
that includes all VMs, etc. This has no VLANs defined. So we need to
define it in libvirt, verify it is defined, start it, and then verify it
is active.

Building VMs

Before we make some VMs we need to place the VMs on our storage.
There are multiple types of storage pools we can use: physical disk
device (disk), pre-formatted block device (fs), logical volume manager
volume group (logical), iscsi target (iscsi), multipath device (mpath),
network directory (netfs), SCSI host adapater (scsi), or directories
(dir). For our example we will be using a directory. However, for best
performance a logical storage pool is recommended.

In general, we do not want to use the default location because it
ends up being in an inconvenient location within the root filesystem.
You may wish to delete it, so that VMs don't accidentally end up there.
Use of a block storage device as a disk type such as iSCSI would be a
better performer than a file system approach if the iSCSI server is
running over a high speed network such as 10G. If all you have is 1G
your mileage may vary.
I did this using a simple script that will assign the proper values
for my VMs. Specifically the base memory, number of vCPUs, disk to a
pool, the networks to use (in this case two Open vSwitch bridges), where
to find the installation media, and finally the use of VNC to do the
install.

This makes it an easily repeatable process and the script takes two
arguments, the vmname and the size in Gigabytes of the disk. Once I have
a VM installed, I could then clone it as necessary. Run as such for a
12G VM named vmname.

# ./mkvm vmname 12

During the install you will have to configure your networks, to
determine which Mac Addresses go with which you should use the following
command:

# virsh dumpxml vmname
…
…

What you are looking for is which interface goes with which bridge
via its Mac address as the Linux installer lists network adapters via
Mac addresses not bridges. It does not even know there is a bridge
there. Using the above script works on RHEL 6 and RHEL 5 and does not
require you to go into and edit any XML files.
If you do have to edit the XML file containing the VM definition you can do so using: