Saturday, March 16, 2013

Compile your own kernel – Tutorial

We’ll take you through the steps you need to compile your own
customised kernel for performance, specialised use and simply to learn
how the plumbing works

Back in the mid-1990s, recompiling the kernel was something of a
necessity, and it was also a good test that a user had mastered the
basics of administering Linux. These days, the stock kernel that comes
with most distros has much improved, removing the necessity of kernel
recompilation for basic use. However, there are cases where it’s well
worth becoming familiar with this area of tweaking your system. For one
thing, it’s a must if you want to access the latest and greatest kernel
improvements, hot off the press, so to speak. It’s also a good way of
understanding how the kernel and other fundamental parts of a Linux
system actually work. It can also be useful when troubleshooting: the
newest kernel might bug-fix the problem you’re having. On the other
hand, an older kernel might be the workaround that you need.
We’ll start you off with a simple example that begins with fetching
the source archive for the kernel that you are currently running,
proceeding through to configuration, compilation and installation.
Following this, we’ll go through some examples that are a bit more
specialist. Most of the examples are for Debian-derived distros, but
we’ve deliberately kept things as neutral as possible and added some
notes for how to handle things on Red Hat-based distros such as Fedora.Once the kernel is complete, you can add it to the system using the standard packaging tools

Step 02

Fetch the kernel (source archive)
If you need the latest kernel, use Git to fetch it (see later step),
but we are going to use the standard package tools in the first example.
Use ‘apt-get source linux-image-$(uname -r)’ to install the source for
the currently running kernel.

Step 03

Examine the source directory
You should now have a source directory in the current directory. Move
into it using the cd command. Note that there is an archived (tar.gz)
copy as well. In addition, there is a diff file that contains the
Ubuntu-specific additions to the standard kernel source tree.

Step 04

Generate a .config file
The (hidden) file ‘.config’, located in the source code directory,
tells the compiler what to build. The configuration file for each
installed kernel is stored in the /boot directory, but you can capture
the configuration of the current kernel (a good starting point) by
typing ‘make oldconfig’.

Step 05

Edit .config
Open up .config in a text editor. Note that there are thousands of
options, and this approach is best if you know exactly what settings you
would like to edit. It’s a good idea to search for
‘CONFIG_LOCALVERSION’ to add a small identifying string for your custom
kernel.

Step 06

Turn off debugging

One way to speed things up and produce smaller files is to turn off debugging.
It’s a specialist feature and mainly used by developers. You can use xconfig for this. Set ‘CONFIG_DEBUG_INFO:’ to ‘n’.

Step 07

Invoke xconfig
Type ‘make xconfig’ to launch the GUI config file editor. It’s a good
way to gain an overview, and it offers information for most of the
options. Run ‘sudo apt-get install libqt4-core libqt4-dev libqt4-gui’ if
it complains about not being able to find Qt.

Step 08

Prepare Debian scripts
Some required scripts lose their execution privileges due to how
apt-get works. Rectify this by typing ‘chmod -R u+x debian/ scripts/*’
and then ‘chmod u+x debian/rules’.

Step 09

Recreating the Source Tree
If you mess things up and want to start from scratch, delete the
source directory. Backup your .config file first, if needed. Then run
“tar xzf” on the source archive to unpack it. Move into the directory
and type “zcat ../[name of diff archive] | patch -p1” to add the Ubuntu
patches into the source tree.

Step 10

Compile the kernel
Tell the build environment how many cores you want to use with
‘export CONCURRENCY_LEVEL=[number of cores]’. Begin the build process
with ‘fakeroot make- kpkg –initrd –append-to-version=-luad kernel- image
kernel-headers’, which takes about an hour on a Core Duo 2.7GHz system.

Step 11

Install kernel
Warning: This is stage where you actually make some changes to your
machine. Using our method means that we can install our custom kernel
just like any other package. Type ‘sudo dpkg -i
linux-image[version].deb’ and then, ‘sudo dpkg -i
linux-headers-[version].deb’.

Step 12

Edit GRUB 2
Open up /etc/default/grub in a text editor. Edit the line
‘GRUB_HIDDEN_TIMEOUT=0’ to remove the zero. Now run ‘update-grub’. The
Grub Menu will be displayed at startup. The old, stock kernel can be
found under the ‘Previous Linux Versions’ entry. On reboot, type ‘uname
-a’ to check the kernel version.

Step 13

Fetch the kernel using Git (Ubuntu) This gives you the latest
version of the Ubuntu kernel source. Install Git: ‘sudo apt- get install
git-core’. Fetch the source with ‘git clone
git://kernel.ubuntu.com/ubuntu/ubuntu- oneiric.git’. Substitute
‘oneiric’ with the official
distribution name that you are using.

Step 14

Fetch kernel manually
Now that you’re familiar with compiling the kernel, consider grabbing
a newer kernel from kernel.org, although these kernels don’t have the
extra additions of Ubuntu. Unpack the kernel with ‘tar xjfv linux..tar.bz2’ and repeat the earlier steps.

Step 15

Building the kernel on Red Hat
Many of the techniques already applied can be used with Red
Hat-derived distributions such as Fedora. Begin by installing the
prerequisites: ‘su -c ‘yum install rpmdevtools yum-utils’ ’. Then type,
‘rpmdev-setuptree’, and then ‘yumdownloader –source kernel’ to fetch the
kernel. ‘su -c ‘yum- builddep kernel-[use tab completion].src.rpm’ ’
and then ‘rpm -Uvh kernel-[use tab completion]. src.rpm’. Move into the
SPECS directory with ‘cd ~/rpmbuild/SPECS’ and type ‘rpmbuild -bp
–target=$(uname -m) kernel.spec’. Now cd into the source directory
(something like, rpmbuild/
BUILD/kernel-3.6.fc17/linux-3.6.8-2.fc17.i686/)

Step 16

Begin the build (Red Hat)
Generate and edit the .config file as before. Open
~/rpmbuild/SPECS/kernel.spec and change the line ‘#% define buildid’
from ‘.local’ to something unique to identify your kernel. From within,
type ‘~/rpmbuild/SPECS/’ then ‘rpmbuild -bb –without debug debuginfo
–target=`uname -m` kernel.spec’ to build.

Step 17

Install ccache
ccache is a compiler cache that is particularly useful when compiling
the kernel, although you only start to see the full benefit on repeat
compilations. Enter ‘sudo apt-get install ccache’. Then type ‘export
CC=”ccache gcc” ’ and then ‘export CXX=”ccache g++” ’.

Step 18

Cross-compilation (Raspberry Pi)
You can use a powerful machine to ‘cross-compile’ a kernel for a
smaller one. We’ll use the Raspberry Pi as an example. You will need a
way of accessing files on the Pi storage device. Type ‘sudo apt-get
install gcc-arm-linux-
gnueabi make git-core ncurses-dev’ on the PC.

Step 19

Fetch the source

On the PC, create a directory and then move into it. Then type ‘git
clone https://github. com/raspberrypi/firmware’ followed by ‘git clone
https://github.com/raspberrypi/linux.git’. Now move into the ‘linux’
directory.

Step 20

Configuration
You can fetch the config from a running Raspberry Pi by typing ‘sudo
zcat /proc/config.gz > .config’ into a shared directory, and then
copying it to the ‘linux’ directory on the PC. Type ‘make ARCH=arm
CROSS_COMPILE=/usr/bin/arm-linux- gnueabi- oldconfig’.

Step 21

Cross-compile kernel
You can then edit the config with ‘make ARCH=arm
CROSS_COMPILE=/usr/bin/ arm-linux-gnueabi- xconfig’. Now run ‘make
ARCH=arm CROSS_COMPILE=/usr/bin/arm- linux-gnueabi- -k’ to compile.
Place the new kernel (linux/arch/arm/boot/Image /boot/ kernel.img) in
/boot/kernel.img.

Step 22

Tweak for MIDI performance 1
Here’s an example to tweak the kernel for improved MIDI latency for
an AV workstation. First, fetch a patch archive at an appropriate level
for your kernel from www.kernel.org/pub/linux/ kernel/projects/rt/.
Repeat the procedure for recreating the source tree, but apply the RT
patch
before the Ubuntu one.

Step 23

Tweak for MIDI performance 2
Follow the steps as for a normal installation, but first carry out
some tweaks in the .config file. Ensure that ‘CONFIG_HZ_1000’ is set to
‘y’ and that ‘CONFIG_APM’ is set to ‘n’.

Step 24

Work inside a virtualiser
If this is the first time you’ve built a kernel, you might like to
start by working inside a virtualiser (eg QEMU). This gives numerous
advantages such as being able to pause the build process. Try to devote
as much memory and as many cores as you possibly can.