Porting Linux to the DEC Alpha: Infrastructure

In the first three parts, Jim describes his early work on Digital's “proof of concept” port of Linux to the Alpha, setting up an infrastructure.

Porting an operating system is not
trivial. Operating systems are large, complex, asynchronous
software systems whose behavior is not always deterministic. In
addition, there are numerous development tools, such as compilers,
debuggers, and libraries, that programmers generally take for
granted but which are not present at the start of the porting
project. The porting team must implement these tools and other
pieces of infrastructure before the porting work itself can begin.

This article is the first of three describing one such
porting effort by a small team of programmers at Digital Equipment
Corporation. Our goal was to port the Linux operating system to the
Digital Alpha family of microprocessors. These articles concentrate
on the initial proof-of-concept port that we did. Although much of
our early work has been superseded by Linus Torvalds' own
portability work for 1.2, our tale vividly illustrates the type and
scale of the tasks involved in an operating system port.

What Got Us into All This?

The article by Jon Hall on page 29 describes many of the
business-case justifications for our involvement in the Linux
porting effort. I will describe the actual events that led to my
starting work on the Linux port.

First, some background: I work for the Alpha Migration Tools
Group, which is an engineering development group within Digital
Semiconductor. We were initially chartered near the beginning of
the Alpha project to develop automated methods for migrating
Digital customers' legacy applications to Alpha-based systems. Our
first product was VEST, which translated VAX/VMS binary executables
into binaries that could be executed on OpenVMS Alpha. This was
soon followed by MX, which translated MIPS Ultrix executables into
executables that run on Alpha systems under Digital Unix. Since
then, our charter has expanded into other areas of “enabling
technology” (technology which enables users to move to Alpha). In
addition to producing translators and emulators, we have supplied
technology to third-party vendors, and we have participated in the
development of compilers and assemblers for Alpha.

Our involvement in Linux began at the end of 1993, when we
realized that there was no entry-level operating system for
Alpha-based systems. While OpenVMS, Digital Unix, and Windows NT
were all solid, powerful operating systems in their own right, they
were too resource-hungry to run on bare-bones system
configurations. In many cases, the smallest
usable configuration of a particular system
costs at least several thousand US dollars more than the smallest
possible configuration. We decided that to
compete on the low end with PC-clone systems, we needed to make the
lowest-priced system configurations usable. After investigating
various alternatives, we decided that Linux had the best
combination of price (free), performance (excellent), and support
(thousands of eager and competent hackers worldwide, with
third-party commercial support starting to appear as well).

Project Goals

When putting together the proposal to do the port, I set
forth the following goals for the Linux/Alpha project:

Price: Linux/Alpha would continue to be free
software. All code developed by Digital for Linux/Alpha would be
distributed free of charge according to the GNU General Public
License. In addition, all tools used to build Linux/Alpha would
also be free.

Resource stinginess: Linux/Alpha would be able to
run on base configurations of PC-class Alpha systems. My goal was
to be able to run in text mode in 8MB of memory and with X-Windows
in 16MB. In addition, a completely functional Linux/Alpha system
should be able to fit, with room to spare, on a 340MB hard
disk.

Performance: Linux/Alpha's performance should be
comparable to Digital Unix.

Compatibility: Linux/Alpha should be
source-code-compatible with existing Linux applications.

Schedule: We wanted to be able to show a working
port as quickly as possible.

Design Decisions

The above criteria drove several of the design decisions we
made regarding Linux/Alpha. To meet the schedule criterion, we
decided to “freeze” our initial code base at the Linux 1.0 level
and work from there, not incorporating later changes unless we
needed a bug fix. This would minimize perturbations to the code
stream (a necessity when you're reaching in and changing virtually
the whole universe), and would eliminate the schedule drain of
constantly catching up to the latest release. We reasoned that once
we got a working kernel, we could then make use of what we had
learned to catch up to the most current version.

The scheduling criterion also drove our decision to make our
initial port a 32-bit (as opposed to a 64-bit) implementation. The
major difference between the two involves the C programming model
used. Intel Linux uses a “32-bit” model where ints, longs, and
pointers are all 32 bits. Digital Unix uses a “64-bit” model
where ints are still 32 bits while longs and pointers are 64 bits.
At Digital, we have encountered a lot of C code that treats ints,
longs, and pointers interchangeably. Code like this might
fortuitously work in a 32-bit programming model, but it may produce
incorrect results in a 64-bit model. We decided to do a 32-bit
initial port so as to minimize the number of such problems. We felt
that limiting longs and pointers to 32 bits would not unduly hamper
any existing code and by the time new applications appeared which
would require larger datatypes, a 64-bit Linux implementation would
be available.

We also decided, in the interests of expediency, to use the
existing PALcode support for Digital Unix rather than write our
own. The Digital Unix PALcode was reasonably well-suited to other
Unix implementations, it was readily available, and it had already
been extremely well-tested. Using the Digital Unix PALcode in turn
required that we use the “SRM” console firmware. The SRM firmware
contained device drivers that could be used by Linux via callback
functions. While these console callback drivers were extremely slow
and had to be run with all interrupts turned off, they did allow us
to concentrate on other areas of the Linux port and defer the work
on device drivers.

Some design decisions were driven by differences in execution
environment between Intel and Alpha. On Intel, the kernel virtual
memory space is mapped one to one with system physical memory
space. Because of the potential collision with user virtual memory,
Intel Linux uses segment registers to keep the address spaces
separate. In kernel mode, the CS, DS, and SS segments point to
kernel virtual memory space, while the FS segment points to user
virtual memory space. This is why there are routines in the kernel
such as put_fs_byte(),
put_fs_word(), put_fs_long(),
etc; this is how data is transferred between kernel space and user
space on Intel Linux implementations.

Since Alpha does not have segmentation, we needed to use some
other mechanism to ensure that user and kernel address spaces did
not collide. One way would be to have only one address space mapped
at a time. This requires a translation buffer (sometimes called a
translation lookaside buffer, or TLB), a special cache on the CPU
used to considerably speed up virtual memory address lookups. But
this makes data transfer between user and kernel space cumbersome.
It can also exact a performance penalty; on systems that do not
implement address space identifiers, using the same virtual address
range for kernel space and user space requires that the entire
translation buffer be invalidated for that range for every
transition between user and kernel space. This could conceivably
cause multiple translation buffer misses across every system call,
timer tick, or device interrupt.

The other way to avoid address space collisions between user
and kernel is to partition the address space,
assigning specified address ranges to specified purposes. This is
the approach taken for the 32-bit Linux/Alpha port. It is simple,
it does not require wholesale translation buffer invalidation for
every entry to kernel mode, and it makes data transfer between user
and kernel an utterly trivial copy.

Designing the address space layout required attention to
certain other constraints. First, no address could be greater than
0x7fffffff, because of Alpha's treatment of 32-bit quantities in
64-bit registers. When one issues an LDL (Load Long) instruction,
the 32-bit quantity that is loaded is sign-extended into the 64-bit
register. Therefore, loading the address 0x81234560 into R0 would
result in R0 containing 0xffffffff81234560. Attempting to
dereference this pointer would result in a memory fault. There are
techniques for double-mapping such problematic addresses, but we
decided that we did not need the additional complications for a
proof-of-concept port. Therefore, we simply limited virtual
addresses to 31 bits.

The other consideration was that we needed an area which was
mapped one for one with system physical memory. We did not want to
simply use the low 256MB (for instance) because we wanted to be
able to place user programs in low addresses, so we chose an area
of high memory for this purpose and made the physical address equal
the virtual address minus a constant. This is referred to below as
the “mini-KSEG”.

Once all the constraints were considered, we ended up with a
system virtual memory layout as follows:

Finally, I had to decide how heavily I would modify the code
base to accomplish the port. I felt that I did not have the
latitude to make wholesale changes and rearrangements of the code
the way Linus did for the 1.1.x to 1.2.x transition. To do so would
cause my code to diverge further and further from the mainstream
code base, which would adversely affect its acceptance among the
Linux community.

I decided to keep the original Intel code 100% intact, so one
could conceivably still build an Intel kernel from my code base.
The Alpha code would be either additions to or replacements for the
Intel code base. Areas that needed to be changed would be set off
via conditional compilation. Sometimes this required me to swallow
my pride and devise a less clean Alpha-specific version of an
algorithm to correspond to a less clean Intel-specific version when
I really would rather have implemented a clean, generalized
algorithm that could accommodate both. Fortunately, Linus
implemented clean, generalized algorithms for all of us when he did
his portability work for Linux 1.1.x and Linux 1.2.x.

Trending Topics

Upcoming Webinar

Getting Started with DevOps - Including New Data on IT Performance from Puppet Labs 2015 State of DevOps Report

August 27, 2015
12:00 PM CDT

DevOps represents a profound change from the way most IT departments have traditionally worked: from siloed teams and high-anxiety releases to everyone collaborating on uneventful and more frequent releases of higher-quality code. It doesn't matter how large or small an organization is, or even whether it's historically slow moving or risk averse — there are ways to adopt DevOps sanely, and get measurable results in just weeks.