Information technology, applied.

Why Xen as Hypervisor?

The choice of hypervisors was easier than the choice of OS because there are fewer choices. I need virtualization to to create multiple environments. Virtualization will allow me to create a production server group, a testing group (identical to production but entirely separate), and a development group. Using VMs I can easily create and change multiple servers and environments.

Virtualization, of course, means an environment based on software instead of hardware. Software gives us enormous flexibility and plasticity. VLANs are the same: software defined LANs in the switch. I think the testing potential in a software defined environment is tremendous. Our test environments become essentially simulations of our production environment.

Anyway, my criteria for a hypervisor were:

It runs on my somewhat obsolete hardware. My Xeon CPUs support only the early Intel virtualization extensions.

It is free or low cost.

It will run CentOS and Windows. I will need to run Windows clients in my testing environment and want to have the option of running a Windows server if needed.

It is stable and performs well.

Has advanced feature such as pooling and live migration.

Is well documented.

Has a Windows console. Not very important but a helpful tool as one is learning. My goal is to eventually use scripts to manage the VMs but that is step two.

I tested three hypervisors in depth over the course of several weeks:

SmartOS

VMWare ESX 5.5

XenServer 6.2

In my tests and for my needs, XenServer was the clear winner. As with the OS choice, it simply had the fewest negatives. I have used it for a few weeks I have grown to like Xen a lot (UUIDs and all). I give Citrix a lot of credit for the truly excellent documentation they provide. Their documentation is so good, I almost made the choice strictly on it alone! See for example, this guide to network design using XenServer. It is thorough, practical, and well written. After the XenServer manuals, reading the SmartOS wiki documentation felt painful and frustrating.

My experience with each was as follows. SmartOS had some very interesting features: it booted from a USB drive (there is no install disk), it used the ZFS file-system, and it used Solaris zones. I wanted to choose SmartOS. The problem was that it didn’t support my server hardware. It would not run KVM and produced some weird networking errors I wasn’t able to fix. The abysmal documentation didn’t help. I did like the idea of using zones for VMs. I thought the performance would have been the best.

ESX 5.5 worked a bit better. I thought this one would win because it is the market leader. It ran fine on my hardware, and installed easily. The primary documentation was OK. I did have two big problems that caused ESX to lose. The first was that I had a lot of trouble getting NIC bonding to work with it. ESX 5.5 (the free version at least) does not seem to support the LACP protocol. On one server I couldn’t get it to work at all. I tried everything on my switch but to no avail.

The biggest problem, however, was that the free version of ESX provides only the barest feature set. No pooling, no live migration, no ability to use the same Windows console to manage all servers at once, and not even the ability to see the real time graphs. I know its free but maybe they could provide something a bit better for purely personal or non-commercial use? Compared to what Citrix offers, ESX’s feature set appear scrooge-like and miserly.

XenServer, on the other hand, provides a pile of advanced features for free. It provides pooling, live migration, iSCSI support, high availability, disaster recovery, and advanced networking options. It runs great on my hardware. Plus it has a nice CLI API. I have already written scripts in Ansible to do most of the basic administration tasks (but I get ahead of myself). Even creating a PXE boot install process was fairly easy (easier than with SmartOS that’s for sure!)

As with all my software choices I will continue to reevaluate as I go. Stability and reliability will be key. So far, XenServer has met all of my priorities for a hypervisor and I am very pleased with it. I recommend it if your priorities match mine

Related Articles

5 replies

Any reason you didn’t try or mention trying some kind of Linux/KVM solution like Proxmox, OpenNode, oVirt… ?

Working with Proxmox today and also reevaluating periodicaly, I stumbled again at Xen no so long ago discovering XenOrchestra was writen by a French crew living nearby. I was impressed by the feature list but hadn’t time to dive deeper since. Do you have specially informative pointers or, again, knowledge to share, about HA and DR features/setup ?

Good question. I haven’t tried Proxmox but looking at their web page I see they support containers. At first I wanted to go with Solaris Zones via SmartOS because I think it is more resource efficient. However, as a practical matter Linux makes more sense for me. I also tried VMWare which was fine but I found their free version had fewer features than Xen.

I’ve been very happy with Xen. The features I like:
– Stability (zero crashes on any host after 6 months)
– Para-virtualization approach.
– It has an API that is easy to use with automated CM tools like Ansible (** this is a must have for me)
– Great documentation
– The VM pool (I create VMs using Ansible and Xen just assigns them to a host. Super easy!)
– The HA is great because it will move VMs automatically (I tested it a bit and it worked)
– For DR I replicate my primary SAN storage to another SAN (it could and should be off site but isn’t in this case). I use the SAN for my VM storage rather than the host disks so all their data is replicated and Xen can switch to the DR storage if the primary goes down with no loss to the VMs.
– Live VM migration works well too as does shallow and deep VM cloning.

All of these are fairly standard I guess. The Xen documentation and command line API made the difference for me. The management console is nice and shows useful stats. I’m planning on using another monitoring tool in its place eventually. I do all the admin on Xen via Ansible scripts. No manual operations allowed!

As a follow-up I think I’m going to give Proxmox a try based on your comment. From the site it seems to have the features I want. have you used their containers? I’d be interested to hear about your experience with them.

Sadly I can’t tell you about the containers support from my personnal experience (I tend to favor mainstream kernel projects) but it seems to work well considering what I’ve seen on the forum. IIRC, there may be feature not supported compared to KVM. Recently published v3.2 adds really exciting features like Ceph and OpenvSwitch, with Spice already included.

On the container front, as a Linux user, my choices are clearly LXC and Docker which I’m hardly waiting to find time to play with.

About Xen HA, is VM failover handled by corosync/pacemaker cluster stack or does XenServer monitors itself and restarts guests after a host failure ?

DR wise, you only mention SAN replicaton, but you also have to have a XenServer running on your DR site don’t you ? How’s the failover handled in case of primary site unavailability, is it automatic or manual ?

Finally, as I understand it, you manage the whole VM lifecycle with Ansible… any code and/or playbooks to learn from somewhere ?