openSUSE@ARM/GSoC: Cross-compilation & speedup

This weeks topic was the integration of the cross-compilation mode into the build environment. But it’s more than just a cross-toolchain – it’s a speed-boost for our ARM build environment. As of today, the source is deployed in the repository Base:build:arm:cross. It’s not fully bootstrapped because of the current high load and the upcoming downtime – so watch out for changes there and in Base:build:arm.

But what are these “speedup’s” ? First, you’ve to know that in our build environment the ARM binaries are executed through an emulation-layer. This works on the cost of speed. The goal is now, to exchange some key parts in a transparent manner with native x86 binaries: no emulation, no slowdown. Sounds reasonable, but is it easily possible ?
I had to take care not to mix stuff too much because the environment would break. But now I’ve to say: WOW, this worked incredibly well 😉 .

The distinctive feature of our approach in comparison to usual cross-build environments is that we use the best of native environment emulation and the speed of cross-compilation. Because of this combination we don’t have to patch the individual packages to make them cross-compilation ready. This is a new way of cross-compiling suitable also for large number of packages. A detailed overview about the different crossbuild types can be found on this page.
Another feature to note is that the exchanged binaries (replacing ARM with x86 in the build environment) also don’t need heavy patching and there’s no need to compile them as static binaries. All of them are normal distribution packages.

A switch in the project enables/disables the new features. With the new changes in place, the speed could be vastly increased. Some figures:
* package rpm
* package glibc w/o locales

Build time in minutes

x86 native

armv5tel native

armv5tel cross

factor native

factor cross

rpm

8

107

17

13,38

2,13

glibc

33

505

63

15,3

1,91

Thats a drop from about x15 to x2 in comparison to the native x86 build-time !! See it yourself when the “crosscompiled” repo in Base:build:arm is up and running.

The simple reason for the speedup is that for your average 8-core or 16-core x86 build box, building an x86 package – even a very large one – can take only a few minutes. I can build a kernel for ARM using cross-compilation in less than 2 minutes, using -j10 parallel makes (it goes faster if I just use -j but I really do not like the way it impacts everyone else’s sessions 🙂

When you run a package build through QEMU you get a single emulated ARM processor running on a SINGLE PHYSICAL CPU core. The other 7-15 processors are sitting idle while this emulated box struggles to translate to x86 instructions and compile ARM code.

This is where the speedup comes from. Most of the problems with cross-compile builds are actually CAUSED by tools which are meant to take the problem away – GNU autoconf is a horrible mistake in design which allows developers to make cross-compilable packages but nobody is talented enough to script it correctly to make it work.