Windows, Open source and beyond

Building embedded ARM systems with Crosstool-NG

This tutorial deals with creating a minimal embedded ARM system from scratch. The approach taken here requires no additional hardware beyond the development host and the source code of the required softwares. QEMU is used to run and debug the ARM system.

This tutorial details the following topics:

Building a toolchain for ARM using Crosstool-NG.

Building QEMU with ARM system emulation support.

Building an ARM Linux kernel.

Building and extending a tiny filesystem from scratch, based on BusyBox.

Debugging ARM applications inside QEMU.

I intentionally kept the default settings when applicable for simplicity and clarity reasons. Many things can be tweaked; you are encouraged to try alternate ways to become familiar with embedded systems.

Building Crosstool-NG

Crosstool-NG allows building cross-compiling toolchains for multiple target architectures. This versatile tool provides a menuconfig configuration interface, which makes it quite easy to use. Crosstool-NG comes with a set of ready-made configuration files for various typical setups and supports glibc, uclibc and eglibc.

Crosstool-NG comes with a set of ready-made configuration files for various typical setups called samples. They can be listed by running:

$ ./ct-ng list-samples

Use the ARM EABI cross compiler for use on linux. Configure and produce this toolchain by issuing:

$ ./ct-ng arm-unknown-linux-gnueabi

Reﬁne your conﬁguration by running the menuconﬁg interface:

$ ./ct-ng menuconfig

At this point it is up to you to select what options you want. You can explore the available options by traveling through the menus and looking at the help for some of the options.

I recommend the following minimal change:

In Path and misc options, set Number of parallel jobs to 2 times the number of CPU cores in your workstation. This causes a faster building of your toolchain.

Disable Fortran and Java programming languages, if you are not going to use them:
In C compiler options, uncheck Fortran and Java.

Build Crosstool-NG:

$ ./ct-ng build

The operation Retrieving needed toolchain components’ tarballs starts. The ./build.log file is useful to track issues. If you get a problem related to fetching a tarball then try downloading the same version from another location or mirror into ./.build/tarballs folder and rerun the ./ct-ng build command.

Compilation takes a long time, be patient.

For the subsequent steps in this tutorial, you need to add path of executables you just installed to your PATH environment variable by issuing the following command:

$ export PATH=$PATH:$HOME/x-tools/arm-unknown-linux-gnueabi/bin

If you want to make it session wide then add the previous command to ~/.bashrc or ~/.pam_environment.

Now, you can verify that your ARM toolchain works by typing:

$ arm-unknown-linux-gnueabi-gcc -v

Building an ARM Linux kernel

This ARM kernel, built with a predefined configuration for the versatile platform board, will be used in the QEMU virtual machine.

The last command brings up the kernel configuration menu, go to the Kernel Features section, check Use the ARM EABI to compile the kernel then exit (Make sure to save your changes).

Compile the kernel:

$ make ARCH=arm CROSS_COMPILE=arm-unknown-linux-gnueabi- all

Once it’s done building, the compressed kernel image is located at ./arch/arm/boot/zImage.

Building QEMU

QEMU is a high performance full system simulator supporting both emulation and virtualization. QEMU supports emulating several CPU architectures that are different from the host running the simulation. QEMU is used as the core device model by both Xen and KVM.

Building BusyBox

BusyBox combines tiny versions of many common UNIX utilities into a single small executable, including a shell and an init process. BusyBox is used extensively to build embedded systems of many types and is customizable so that only the needed utilities are built.

The minimal filesystem still lacks some shared libraries to run busybox, gdbserver and hello programs. These missing libraries can be enumerated using arm-unknown-linux-gnueabi-readelf (created when building the ARM toolchain) as following:

To add networking capabilities to the guest ARM system side and optionally use the same keyboard layout as your host machine, you need to create additional folders in the filesystem:

$ mkdir -p proc sys dev etc etc/init.d

Optionally, create a BusyBox-compatible keymap, in the filesystem, from the currently loaded keymap of your host:

$ sudo busybox dumpkmap > etc/host.kmap

The default initialization script that BusyBox searches for is /etc/init.d/rcS. Instead of using inittab, this is the preferred method to initialize an embedded system based on BusyBox. Create the rcS script as follows:

The redir option will redirect any TCP communication from port 65000 of host system to port 65000 of the guest ARM system. This arbitrarily unused port number is chosen for testing purposes.

The ARM system will boot in a QEMU window then displays the message Please press Enter to activate this console. Press enter and you should be presented with an interactive shell. Start the debugging server as follows:

$ gdbserver --multi localhost:65000

Open a new terminal window, and install the ddd debugger:

$ sudo apt-get install ddd

Edit a configuration file, for example exec-commands.dbg, using the following commands:

Thank you Kamel for this jobs. I have search a long time for a tutorial that summarize all these steps on Internet an no result. thak you to save my time.

but I have also a question:
when running arm-unknown-linux-gnueabi-readelf tool to determine all missing libraries, where you find exactly the name of the missed library? (only I have seen ld-linux.so.3 library !)

As described in this guide, one method to find out shared library dependencies of a executable is by running arm-unknown-linux-gnueabi-readelf as follows (pay attention to NEEDED and “Shared library” keywords):arm-unknown-linux-gnueabi-readelf your_executable -a |grep lib

Depending on your development environment you may end up copying a different set of shared libraries to your target. I also encourage you to read readelf documentation.