Menu

We have virtualization on servers, desktops, laptops and what’s next? Naturally, the “smart phone”. Today’s mobile phones have become quite powerful that one can easily virtualize them and run multiple operating systems. But,

Why do you need smart phone virtualization?

Similar to desktop virtualization, the use-case is not that clear, but there are some benefits that can be gained.

Reducing vendor lock-in. Can you run iPhone applications on an Android phone? Currently, no, but with virtualization this might be possible.
There are issues with Apple (or other vendors)’s mobile OS licensing that make it illegal to do so, right now. You can only run iOS on Apple approved devices, but this may change in the future. Many old phones have legacy applications and libraries that can be run in virtualized sandboxes as well.

Mobile cloud computing. You can clearly access the so-called “cloud” from a smart phone, but if smart phones are virtualized, mobile applications can be migrated to a cloud and back seamlessly. This is
still a pipe-dream and a few research platforms (MAUI
and CloneCloud) are proposed to make this a reality. The benefits of doing this are enormous: one can potentially access infinite compute and storage resources on the smart phone.

Security. This is probably the strongest argument for mobile
virtualization. The popularity of smart phones is making them a prime target for
malicious hackers. Malware, Viruses and other nasty stuff are making their way into smart phones, which currently don’t have much protection against them. It’s not easy to run Anti-virus software on a mobile phone, as the current AV software is very resource hungry making it unsuitable for smart phones.

Isolation. This is related to security. It will be possible to have different user profiles with different levels of access (say work and home) with mobile virtualization. VMware recently demonstrated, how multiple user profiles can co-exist in the same phone at Mobile Congress 2011.

Ease of development. This is an indirect benefit for mobile phone
developers. Virtualization can make the development of mobile applications
easier by removing the need for maintaining multiple versions of the same
application for different operating systems.

Challenges

Before you start seeing virtualized mobile phones, there are still plenty of
challenges that need to be solved.

Limited resources. Though, the CPUs on mobile phones are getting more
and more powerful, they still have limited battery and storage space.
Virtualization usually adds more burden to the resources and is not necessarily
energy-aware.

Poor connectivity . Mobile cloud computing sounds like a great idea,
but mobile phones often experience variable connectivity. It’s not easy to
implement disconnected
operations for storage and for seamlessly migrating computation.

Poor performance. Virtualization adds performance overheads, which
are still being researched in server virtualization environments. It’s not
clear whether the overheads are worth the effort on a mobile phone.

These challenges can be overcome and it’s only a matter of time before
smart phones are virtualized. Some current players in the market are: VMware’s
mobile virtualizaiton platform (MVP) and OKL4
MicroVisor. OKL4 claims to be the first open source mobile hypervisor, which
is an exciting development that can lead to advances in mobile
virtualization.

If you are hosting a web server at home and using your router’s port
forwarding to redirect requests from WAN port to WebServerHost:80, there’s
an annoying problem in some routers: you cannot access the web server from
inside the network. There are multiple reasons why this might happen.

The router assumes that all traffic from internal hosts to port 80 are for
the router admin interface. This is annoying, as you just get to the
router’s web admin page. Solution: you can port forward from
another port (say 8000) to your WebServer’s port 80. Now, you can access
the website with http://my.site.net:8000. You are out of luck, if your router
does’t allow port forwarding to a different port (say
WAN:8000->WebServerHost80 instead of WAN:80->WebServerHost80).

You have two or more routers with different subnets and the routing is
messed up. This is a difficult problem to diagnose and often the router’s
admin interfaces are not that amenable to debugging routing problems. I suggest
changing your firmware to DD-WRT, if you can.

Port forwarding doesn’t work for incoming traffic from local ports on
the router. I think this is probably the most common reason. Without DD-WRT,
there’s no easy way to fix this. Solution: One simple way to solve
this is to set a hard-coded DNS entry in /etc/hosts (Linux) orc:\WINDOWS\system32\drivers\etc (Windows) as follows.

Virtualization is a hot topic and the field is growing at a rapid pace. There
are hundreds of papers on each sub-topic, and this list is intended to be a good
starting point for someone starting in virtualization research.

Surveys/Books

These are good starting points, if you are just learning about
virtualization

Survey of virtualization machine research. Robert P. Goldberg. IEEE
Computer, June 1974, pp 34-45. [PDF]. One of the oldest surveys about virtualization research.

A Survey on Virtualization Techniques. Susanta Nanda and Tzi-cker
Chiueh [PDF].
A more modern survey of virtualization techniques. The paper lists many of the
techniques, but doesn’t really explain how they are different etc. Yet,
this is a good read for budding virtualization researchers.

Virtualization Overview

These are the papers that will help in understanding fundamentals and
concepts of virtualization.

Xen and the art of virtualization. Barham et.al. This is the
classic SOSP paper on para-virtualization. [PDF].

When Virtual is Better Than Real. Peter Chen and Brian Noble. This
is a great short article explaining the benefits of virtualization. ACM
link.

Disco: Running Commodity Operating Systems on Scalable Multiprocessors. E.
Bugnion, S. Devine, and M. Rosenblum. This paper is considered to be the first
paper that revived the virtualization concepts pioneered by IBM. ACM
link.

Running multiple operating systems concurrently on an IA32 PC using
virtualization techniques. Kevin Lawton. This is a great article on the
difficulties involved in virtualizing the x86 platform. link.

Formal requirements for virtualizable third generation architectures. Popek
and Goldberg. The classic paper explaining the requirements for virtualizing a
specific ISA. ACM
link. If you want light reading, check the Wikipedia
explanation of the requirements.

Scale and Performance in the Denali Isolation Kernel. Andrew Whitaker,
Marianne Shaw, and Steven D. Gribble. Denali is another example of early
virtualization papers that rejuvenated the research. Denali shows the containers
(or vservers) concept, which is used in OpenVZ, KVM and VServers.

The Exokernel Operating System Architecture. Dawson Engler’s thesis.[PS].
This is more of OS research, but a great read for understanding some of the
techniques (thin hypervisor, pass through etc.) that are used in modern
virtualized systems.

CPU Virtualization

Virtualization of CPU is provided by a CPU scheduler that provides the
illusion of multiple CPUs (or VCPUs). Scheduling has a long and rich history.
Below are a few links in relation to virtualization.

Memory Virtualization

Memory Resource Management in VMware ESX Server. Carl A. Waldspurger. [PDF].
Classic paper on memory virtualization. This paper introduces memory ballooning,
which is a great technique used in commercial platforms as well.

Difference engine: harnessing memory redundancy in virtual machines. Diwakar
Gupta et.al. [PDF].
A great follow up paper that talks about sharing memory across VMs.
Incidentally, both these papers have received best paper awards.

Migration/Cloning

Live Migration of Virtual Machines. Clark et.al. One of the first academic
papers discussing how to do live migration, which is implemented in Xen. VMware
has supposedly implemented vmotion before this was published. [PDF]

The design and implementation of Zap: a system for migrating computing
environments. Osman et.al. Zap predates Xen paper and talks about sandboxing
processes that can be migrated. This is more like container virtualization
rather than full system virtualization. ACM
link

Resource Management/Automation

Carl Waldspurger’s PhD thesis is a great place to start for
understanding resource management. MIT link.

Automated control of multiple virtualized resources. Padala et.al.
Disclaimer: this is my own paper It’s a good place to start if you want to
learn about automating resource management for virtual machines. The control
theory aspects are a bit complex, but you can ignore them and focus on the
issues in automating resource management. [PDF].

Black-box and Gray-box Strategies for Virtual Machine Migration. Clark
et.al. This is a follow up for live migration paper. The paper talks about
strategies for automating the migration of VMs to meet specific goals. USENIX
link.

The HOWTO on compiling CentOS kernel based on RHEL code base (currently 2.6.18) is overly complicated. Follow the following steps to build a custom Xen kernel, maybe with different kernel configuration.

Setting up Xen4 has been tricky, because of the move to the latest Linux
kernels with paravirt_ops. Compiling it on CentOS, which uses 2.6.18 kernel, and old toolset is a bit more challenging. The following are my notes for setting up Xen4 + pvops 2.6.32.x on CentOS 5.5 on BL460c blades.

Important note: I couldn’t get Xen 4.0.1 release work
on CentOS 5.5 on our blades. You have to get the latest Xen and pvops kernel
available from http://xenbits.xen.org/xen-4.0-testing.hg and
http://www.kernel.org/pub/scm/linux/kernel/git/jeremy/xen.git/ (4.0.2-rc1-pre
and 2.6.32.21 as of this writing). This may or may not be a problem for you.

This is the most critical step, and if you don’t configure the
kernel correctly, booting will fail badly. make kernels builds the
kernel with the default .config. To modify it, change to build-xxx directory and
run make menuconfig.

Troubleshooting

The biggest problem will be kernel not booting properly. There might be
various reasons.

Make sure to use the right set of configuration options for your hardware

Make sure to have all Xen options correctly compiled

out_of_memory kernel OOPS

CONFIG_XEN_MAX_DOMAIN_MEMORY=32 # change this to a lesser number

Add more memory to dom0 with dom0_mem=4096M

Network card not appearing – make sure to have the right network
module (bnx2x for BroadCom NetXtreme II card) inserted

Missing /dev/xen entries. Means that you haven’t loaded the
xen-evtchn and xen-gntdev. Run

modprobe xen-evtchn xen-gntdev

See step 9 above to update /etc/modprobe.conf

If you see the following in /var/log/messages

xenconsoled: Failed to contact xenstore (Connection refused)

Do step 2 to insert all the appropriate modules, and verify that you can
start xend

Installing VMs

First, verify that the initrd includes all the necessary Xen frontend
modules. Thisis not necessary for hvm VMs, but for paravirtualization it’s crucial
to have allthe right netfront and blkfront modules installed. A sample mkinitrd for domU
isbelow

References

http://wiki.xensource.com/xenwiki/XenParavirtOps. This is the most
authoritative reference for compiling the pvops kernel.

http://wiki.xensource.com/xenwiki/Xen4.0

http://bderzhavets.wordpress.com/2010/04/24/set-up-ubuntu-10-04-server-pv-domu-at-xen-4-0-dom0-pvops-2-6-32-10-kernel-dom0-on-top-of-ubuntu-10-04-server/
(this is Ubuntu specific, but has some useful details)

Conference rankings are a contentious topic, and it is often difficult to
directly compare two conferences, because each conference has unique flavor,
community and history. In general, it’s easy to identify top 1 or 2
conferences in a field, but it gets murky as you go down. There are a bunch of
webpages (see below), where you can look at the rankings. About the field of
computer systems, defining it is another blog post. For now, read the Section
3 of Eurosys.org white paper, and Liviu’s
presentation on what constitutes good systems research..

This post is my view on the systems conference rankings. Disclaimer: these
are just my personal opinions. Quality of conferences varies over time, and
these are not set in stone. In my opinion, all the conferences I mentioned here
are reputed and great conferences to publish in.

For general systems (operating systems, distributed systems)

These two conferences, which alternate every year are widely considered as
the top conferences in systems. SOSP has rich history, and is considered
slightly more prestigious. Both are highly selective and are very influential in
“real” systems.

Eurosys is still a new conference, but is quickly becoming a premier systems
conference. Though, it is hosted in Europe every year, published papers are from
all around the world, with majority of them coming from systems research groups
in the US.

For networked systems

Widely considered as the most difficult conference (in networking) to
publish in, SIGCOMM has rich history, and great papers every year. Earlier
SIGCOMM conferences have accepted more papers, but the acceptance rates have
dropped to 10-12% in the past few years.

Other systems-flavored conferences

For the combination of OS + architecture or OS + programming languages ASPLOS is
considered the top conference. It usually has multi-disciplinary flavor to
it.

Now, the fun part.

Is INFOCOM a
good networking conference for systems work? This is a tricky question, since
INFOCOM traditionally has accepted more theory/algorithms/analysis/simulation
oriented papers, with a bent on wireless systems. INFOCOM has low acceptance
rates in the range of 16-20%, but due to large number of tracks, the quality
sometimes is patchy. INFOCOM does accept systems papers.

Is NSDI better than Eurosys for networking + systems work? Another
interesting question to ponder during lunch breaks I think, if a project has more networking flavor
to it, one should consider submitting it to NSDI. On the other hand, if the
project is purely systems work, then Eurosys is a better venue.

What is the criterion for deciding the rankings of conferences? Should we
use acceptrance rate, or citation count or industry influence or a combination
of them?

Virtualization is a hot topic for research, and with the rise of cloud
computing, it has gained even more attention. Great number of research papers
are being published in many major conferences. The following is an attempt to
list some of the research work, and how they influenced real products or
features. Some of the products have been major open source projects, before
becoming purely commercial products. These are roughly in reverse-chronological
order.

Note: I will be updating this over time. Let me
know if you know of virtualization research papers that inspired or spawned real
products.