The Penguin and the Dinosaur

One obvious question is, “what good does this do?” Let's
look at a few scenarios. Scott Courtney has addressed this issue
wonderfully in his essay at LinuxPlanet (see Resources), which
should be required reading if you're interested in L/390—it is a
great deal more in-depth than my article.

UNIX administration and development skills are much more
common and much cheaper than mainframe skills. Thus, it's going to
be easier to find people to work on Linux on your mainframe than it
would be to find OS/390, VSE or VM gurus.

What good is a mainframe? Mainframes traditionally had little
in the way of CPU power (no longer true—you get plenty of raw CPU
speed out of one), but have absolutely fantastic I/O capabilities.
One of the main needs in a big web server farm or a big e-commerce
server is I/O, of course.

Linux/390 presents an interesting migration path for
organizations which are seeking to de-emphasize their mainframes
but can't just decommission them, because it provides services
simply unavailable in other environments. I've personally seen this
scenario happen twice: once at Rice University and once at
Princeton. Plans to shut the dinosaur down were announced, then
retracted once it became clear that vital parts of the campus would
grind to a halt, since there was simply no viable alternative to
some of the mainframe services. Moving over to Linux for those
things it can do provides a smoother and
cheaper transition, without the need for additional hardware. Let's
face it: if your organization is going to be moving away from
OS/390 or VM, better they should move to Linux than to another OS.
Since Linux running Samba is already a good back-office substitute
for NT, you could provide Windows browsing services to your users
without ever needing PC hardware, let alone an NT license.

Another exciting possibility is that VM licenses tend to be
held by academic institutions. Imagine a third-year computer
science course on operating systems or networking. Now imagine each
student getting his very own Linux box with which to play. There's
full OS source code, a full development environment and isolation
from the production systems and other students. A fantastic course
could be developed around a study of Linux internals in such an
environment, and a medium-sized S/390 could support a class of 25
students, all recompiling the kernel at once, without breaking a
sweat.

The commercial version of this scenario is the commercial
web-hosting server. Traditionally, this means you get dedicated
access to a machine in a rack space somewhere, physically managed
by an ISP. We'll do the numbers a little later, but in short, if
you're doing this on a large scale, the price of a mainframe and a
VM license get dwarfed quickly by the price of a whole bunch of
fast, rack-mounted PCs. Your labor cost also drops radically, as
you don't have to physically set up yet another PC; you simply
create one more virtual machine on your VM box, and give it its own
copy of the installed system disks.

As an aside, it doesn't hurt to remember that the total cost
of network ownership typically breaks down to less than one-quarter
hardware, roughly one-third service and facilities, with the
remainder the necessary staff to support it (IDC, 1996).
Administration tasks are obviously greatly simplified when the
entire network of Linux machines is contained within a single
box.

It would also be quite possible to separate various services
onto various virtual machines. Sendmail would get its own (virtual)
Linux box, DNS another, Samba another. This would be good from both
a security standpoint (an exploit on one machine compromises only
one service) and a reliability standpoint. You can also split the
various pieces of a multi-tier application (e.g., web front end,
business rules processing engine in the middle and RDBMS on the
back end) among separate virtual machines, and run your database on
OS/390, if you prefer. The isolation would make both debugging and
development somewhat easier. I know this is
something we always laugh at the NT people for requiring, but there
are three advantages to the Linux-on-a-virtual-machine method of
service isolation:

Additional hardware cost is zero, as opposed to a
couple thousand dollars per machine.

Additional software cost is zero, as opposed to the
cost of an NT license per machine.

Actual resource utilization overhead is very low,
since VM's virtual machines consume almost no resources unless they
are actually running.

Price and Performance

Mainframes are expensive. There's no way around that fact.
They're less expensive than they used to be, but they're certainly
not $800 PCs. I'm not particularly au courant
with IBM's pricing structure, but let's take a million dollars as a
high-end price. A million dollars will buy you a
lot of mainframe; you'd get a terrific IBM
support contract with it and a VM license, as well as a backup
solution. Most mainframe shops run OS/390, often in tandem with VM,
but if your purpose is to run many Linux virtual machines, then
you'd want VM and would have no use for OS/390 unless you wanted to
do traditional mainframe computing too. I'm told a brand-new,
top-end system with several terabytes of DASD is closer to $600,000
than a million, and much cheaper secondhand. However, for the
purposes of argument, let's stick with a million as a nice round
figure.

What do you get for that price? A machine which will run—I'm
being extremely conservative here—1000 simultaneous Linux machines
without a problem. You're talking $1000 per Linux box: not too
different from the cost of a low-end rackmount Linux system, thus
the price right there is a wash.

Obviously, it's a lot cheaper to rent space for a single
S/390 and its associated disk arrays than it is to rent space for
1000 physical Linux boxes; even in one-unit packaging at 42
units/rack, you're still talking 24 racks. That's certainly more
than even a big S/390 is going to take. Networking gets easier too;
buy the 155MBps ATM options on your OSA cards, plug up to twelve
ATM interfaces into your machine (if you need more scalability,
options exist for you) and coordinate internal communication
between your virtual machines via a virtual LAN. An internal,
virtual LAN is much easier and cleaner to administer than a
physical topology of switches and runs of Cat5 and fiber.

You can cut back on resource usage, too. If some of your
machines will have identical file systems (it's fairly common, for
example, in the UNIX world, to mount /usr read-only and have
symlinks to what needs to be writable), then you can maintain a
single disk with the shared file system and mount it from each of
the virtual machines. It's just like sharing file systems over NFS,
only without the ugliness of NFS. This, incidentally, was what
David Boyes did when running his 41,400 machines: all shared /usr
and /bin; with a little more trickery, /lib could be shared too.
Furthermore, there's no need to specify much—or indeed, any—swap
for your Linux machines; VM knows all about memory management and
does it very effectively. Give each of your Linux machines a bunch
of virtual memory, and let the VM hypervisor worry about paging it
in and out.

Now let's look at reliability. We'll assume one of these
$1000 machines has a MTBF of 1000 days. That's probably on the high
side for a thousand-dollar machine, if you're pushing it fairly
hard. At $1000, you're not getting RAID, your disks are probably
IDE, and even redundant power supplies are unlikely. If you have a
thousand of these boxes in a room, the chance you will get through
a day with none of them failing is
1-(1/1000)<+>1000<+>, or about 37%. In other words, two
days out of three, you're going to have to replace
something.

Of course, if the mainframe fails, all
your machines fail. However, one of the things you're paying for
with your million dollars is rock-solid reliability: a System/390
is built with enough redundancy that if something fails, the rest
of the system stays up and can be hot-swapped. This isn't just
disks: on multiprocessor machines, you can replace a failed
processor without bringing down the system. I giggled when I saw
Microsoft trumpeting it had achieved 99.5% up time with NT. Thus,
under exceptionally good circumstances, properly administered and
maintained, NT is down, on average, a little less than an hour a
week. I was a VM systems programmer from 1992 to 1994; during that
time, we typically had under an hour of scheduled down time a
year. Unscheduled down
time was zero.

Backups cease to be an issue; because VM is managing all the
disks for the virtual machines, all their data is backed up with
the VM backup. The same with data integrity; for that kind of cash,
you get well-implemented hot-swappable RAID, in which the
complexity is never even visible to the Linux machines, because
they see their disk space just as devices presented to them by VM.
Basically, no matter how many virtual machines you have, you have
only one actual machine to protect, so the cost of doing so remains
constant, rather than scaling with the number of machines.

Furthermore, VM has an extremely efficient cache. Frequently
accessed disk blocks will be held in the cache and requests never
go to the drives at all. This is a huge win if you either share
disks among machines, or if you're running a server farm.

There's also a low end in the mainframe market. The P/390 is
a PCI card containing a chip with the S/390 instruction set on it.
It is sold in combination with another PCI card which provides a
channel interface (so you can drive your real S/390 peripherals), a
PC running OS/2 and some driver software; you run VM, VSE, OS/390
or Linux on the card. The prices for the PC Server System/390 are
under $10,000 now. There's also the slightly more expensive R/390,
which sits in an AIX box. Ten thousand dollars is well within the
budget of a smallish software development company. Of course, these
boxes won't support a thousand simultaneous machines, but they'd do
fifty fairly comfortably (they support 130 or so simultaneous CMS
users). In fact, I'm planning to use just such a system to play
with a virtual Beowulf cluster, among other things, and maybe
experiment with MOSIX, too.

There's also a lot of middle ground between these two ends.
In short, it's not much more expensive, from a purely hardware
point of view, to put N virtual Linux boxes on
a mainframe than it is to simply buy N boxes,
and it becomes a good deal cheaper as N
increases.

Trending Topics

Webinar: 8 Signs You’re Beyond Cron

Scheduling Crontabs With an Enterprise Scheduler
11am CDT, April 29th

Join Linux Journal and Pat Cameron, Director of Automation Technology at HelpSystems, as they discuss the eight primary advantages of moving beyond cron job scheduling. In this webinar, you’ll learn about integrating cron with an enterprise scheduler.