Re: Portable vkernel (emulator)

From:

Matthew Dillon <dillon@xxxxxxxxxxxxxxxxxxxx>

Date:

Thu, 10 Jul 2008 10:25:25 -0700 (PDT)

:It's not even finished yet, I don't think it's fair to hype it as a
:killer feature. There's a lot of proving and testing left before it is
:competitive with other modern filesystems. Right now it's still
:competing with UFSv1, which is how many decades old? Let's not get
:ahead of ourselves.
Well, it isn't that bad :-). All of HAMMER's major features are in
the tree now and neither UFS1 nor UFS2 can hold a candle to any of them.
But from a stability and use point of view I would agree that HAMMER
needs to age a bit to achieve the reputation that UFS has acquired
over the last 20 years. This means people have to start using it for
more then testing (once we release).
This is why I'm making a big push to get it running as smoothly as
possible for this release. A lot of people outside the DragonFly
project are going to be judging HAMMER based on its early use cases,
and the clock starts ticking with the first official release.
:Last time this was discussed, it was revealed that the vkernel
:implementation is so efficient specifically because it required
:extensions to the host DragonFly. So yes, it's a pure userland process
:in DragonFly, but it uses system calls that Linux doesn't have (and
:may not even have an equivalent to, since user-mode-linux is
:implemented very differently to vkernel). If I recall correctly these
:were extensions to mailbox signals and page protection and stuff like
:that. The closest you could do is add analogues of those system calls
:and semantics to your host of choice.
I think I stopped using the mailbox signals (they got replaced by the
co-thread I/O model), but the vkernel still needs the syscall support
for managing VM spaces and virtualized page tables. I'm not sure
what kind of APIs Linux has to support their UVM stuff.
:Until then, you can use KVM on Linux which has very close to hardware
:performance on modern CPUs. I use KVM to emulate FreeBSD on Linux and
:"I can't believe it's not native". Linux these days makes a fantastic
:virtual server, supporting every cross-platform virtualisation
:technology, most of them very efficiently. You can install Ubuntu
:Hardy in text mode, add on KVM and install DragonFly, all with 100%
:free software provided in the official repositories.
:
:In fact, just as we speak I'm using QEMU to install a big-endian
:Debian for testing C code's endian correctness. We're spoiled these
:days.
:...
:> Even if it might be super inefficient, it would be super cool to be able
:> to run a vkernel, which runs hammer, which runs samba, which serves as
:> fileserver for windoze ;-)
:> (or FreeBSD -> vkernel -> hammer -> nfs -> FreeBSD :)
:
:Yeah, you can do that with KVM. I'm not convinced that'd perform any
:better, or be any more reliable, than using native FreeBSD 7 with ZFS.
:
:--
:Dmitri Nikulin
The KVM stuff is pretty cool but the performance claims, particularly
by companies such as VMWare, are all hype. The bare fact of the matter
is that no matter what you do you still have to cross a protection
boundary to make a system call or do I/O or take a page fault.
Hardware-supported hypervisors don't help much in that regard, they
just aren't advanced enough yet. They will get there, for sure, but
it will be another couple of years at least. In many respects as cpu's
continue to evolve into more and more cores (or threads if you take
Sun's viewpoint), the overheads imposed by hypervisors and KVMs will
continue to drop. Strangly enough hyper-threading might be the best
solution for reducing KVM support overheads.
The issue, for a KVM, is that crossing this boundary takes about 2uS
verses the 150nS it takes on real hardware. Everything else will run
at full speed. Also, to run reasonably well on a KVM kernels often have
to be compiled to use far lower clock interrupt rates (an example of
this would be the custom linux kernels IBM runs on their main-frames).
For example, the scheduler interrupt might have to run at 50hz in a KVM
environment instead of 1000hz. This is because idle overheads are
magnified considerably. Multi-threading and interactive performance
of a kernel running under a KVM tends to suffer a lot due to the necessity
of reducing idle and clock interrupt overheads.
So the performance for something running under a KVM depends a lot on
what that something is doing. Cpu-bound programs which don't make many
system calls (such as gcc), or I/O-bound programs which would be
blocked on I/O much of the time anyway (such as sendmail), will
perform fairly well. System-call intensive programs, such as a web
server, will lose a lot in the translation.
A very large problem set is solved through the use of KVMs, even web
serving, as long as the load does not exceed available cpu power. As
machines have gotten faster the cpu bar has continued to move up and
more and more programs can be run in such an environment.
If you are pegging your cpu to the wall and need performance, though,
you don't want to use a KVM. If you need more hardware connectivity
then a simple network and disk interface, KVMs tends to be more trouble
then they are worth. A KVM emulating more complex and varied hardware,
such as audio and video hardware, very quickly runs into trouble from
a performance standpoint.
-Matt