Linux Compatibility on BSD for the PPC Platform

This document deals with the main problems encountered when implementing
Linux binary compatibility for PowerPC-based NetBSD ports. It is
intended to document various parts of the emulation subsystem, and to
highlight some architecture-dependent issues that can arise in argument
passing, signal handling, and with the way some system calls work. I hope
it will help potential developers to do further work on the NetBSD
binary compatibility framework.

Most, if not all, of this paper is intended for technically oriented
readers. It is assumed that the reader has some understanding of the C
programming language and has a good understanding of how processes are
managed on a Unix system. Information about this, and much more, can be
found in Design and Implementation of the 4.4BSD Operating System, or
The Linux Kernel.

Setting up minimal emulation support

In this part, we will introduce Linux emulation and the way
it is implemented. Then we will describe the different steps required
in order to run statically linked Linux binaries on a NetBSD/PowerPC system.

What is Linux compatibility?

Some programs such as Netscape or Sun's JDK are not distributed with
source code, so it is not possible to port them to NetBSD. We have to make
do with a Linux binary, sometime a FreeBSD binary, but never a NetBSD
binary. Nevertheless, users want these kind of applications to run on
their NetBSD machines. To address this problem, Linux compatibility was
developed on NetBSD. This Linux emulation is available through the
COMPAT_LINUX kernel option on the NetBSD ports that support it (i386,
alpha, and m68k). The compatibility subsystem emulates Linux system
calls, and not the program itself. From the Linux program's point of
view, the NetBSD kernel just looks like the Linux kernel. The Linux
binary is thus able to run on NetBSD, at normal CPU speed. All its
system call are intercepted and mapped to native NetBSD system calls.
The overhead of Linux compatibility is hence very small.

How does it work: the global picture

A userland executable interacts in only two ways with the kernel. On one
hand, we have calls from the executable to the kernel, which are system
calls. On the other hand, we have the interaction from the kernel to the
executable, which is signal delivery.

In order to emulate Linux binaries, the NetBSD kernel must mimic Linux
kernel behaviour for system calls and signal delivery. Signal delivery
is the trickiest part of the job, and not all executables actually need
signals for normal operation, so we will keep signal handling for later.
On the other hand, system calls are mandatory. If you want your program
to do simple operations, such as reading a file or writing some text to
a terminal, you need to make system calls. You might build an
executable that does not make any system calls, but I am not sure that
running it will be actually of any interest. So let us talk about system
call emulation.

The main idea is to translate system calls. Each system call has a
system call number and some arguments. If you run a Linux binary on
NetBSD without writing any compatibility support in the NetBSD kernel, it
will not work, because the executable will use a system call number
that is incorrect for NetBSD. For instance, let us assume that our
program uses the nice() system call, which is syscall #43 on
Linux/PowerPC. If you run it as a Linux binary on NetBSD/PowerPC, it
will actually call fchflags(), which is the syscall #43 on
NetBSD/PowerPC. And even if the syscall is the same, the arguments will
probably not fit. For instance, Linux will use a 32-bit long where NetBSD
uses a 64-bit long long for the same argument, and this will cause the
program to fail.

The NetBSD kernel must therefore first match the Linux executable. That
is, it must recognise it as a Linux binary and not as a NetBSD binary.
Then, when the program makes a system call, the NetBSD kernel will
translate the Linux system call to a NetBSD system call. Of course,
executables matched as NetBSD binaries have their system calls
unchanged.