Standard Operating Procedures for Embedded Linux Systems

Follow these procedures for the smoothest path to great embedded Linux.

Ten Downsizing Methods

The organization of the downsizing issue is displayed in the bottom of
Figure 1. We divide the methods into two parts, because the software
platform of an embedded system typically consists of a kernel and root
filesystem. The first part is how to get a small kernel, and the second
part is how to downsize each component in the root filesystem, including
libraries and shells. The second part also discusses how to compress
the whole root filesystem. We describe all methods in detail below,
along with experimental results. After explaining all methods, we show
the effect of these methods on our laboratory embedded system, called
the Wall system. Table 1 presents the specification for the Wall system. The
system is a network security gateway that provides application-layer
content filters, such as antispam and antivirus.

Selecting an appropriate kernel is the first step in downsizing the
kernel. If you choose an inappropriate kernel, the system may
be not only large but also unable to use processor power effectively. For
example, a standard Linux kernel on a hardware platform without MMU
cannot work normally. Such a hardware platform requires a specific
MMU-less kernel, such as uClinux.
Most people use the standard Linux kernel and attempt to trim its size.

The next step is to include only the necessary modules in the standard Linux
kernel by a correct configuration. In fact, the default configuration
of a Linux kernel includes many unused modules, which causes you to have a
big kernel. Figure 3(a) shows the experimental results on the downsizing
effect of the correct configuration. In this case, a system supporting TCP/IP
has a kernel image that is only 59.84% of the size of a system supporting
all network protocols.

Figure 3. Effects of the Downsizing Methods on the Kernel

To downsize the kernel, the third step is to use the optimization
parameters when compiling the kernel. Using parameters -O1, -O2 or -O3
can improve performance, and using -Os can reduce size. However, optimizing for
both performance and size simultaneously is not possible. Therefore,
we generally select -O2 to achieve a balance between size and
performance. As shown in Figure 3(b), the -Os parameter reduces the
size of the kernel image by 22.82% as compared with -O3, but it causes
worse performance.

Besides including only the necessary modules and compiling the kernel with
the optimal parameters, to downsize the kernel further, you can decrease
the size of the static buffer and array allocated in the kernel, because the
kernel typically declares a large buffer and array for standard PCs. To
find out which buffer or array occupies large memory space, you can use the
command nm. This command can list the allocated size of each variable
in an object file. With that information, you can browse the corresponding
source code of the object file and alter the initial size of the buffer
or array. Another approach for shrinking the buffer size is to modify the
options in the menuconfig of the kernel to decrease the maximum number
of supported peripherals, as shown in Figure 3(c) and (d).

Methods for the Root Filesystem

As shown in Figure 1, we identify six methods for downsizing the root filesystem. First, you can adopt a tool called BusyBox, which provides a
fairly complete environment for any small or embedded system. BusyBox
combines tiny versions of many common UNIX utilities into a single small
executable file, and it is highly modular, allowing commands to be included
or excluded at compile time. The space used for BusyBox is 7.04% of that
of the original tool, as demonstrated in Figure 4.

Figure 4. Effects of Downsizing Methods on the Root Filesystem

Next, we introduce three methods for removing unused libraries or
downsizing required
libraries. First, you can use the command ldd to identify the required
shared libraries for each program, and then with this information, you can
remove the unused libraries. Notably, if a shared library is not used
by programs, you additionally should check whether it is used by other
shared libraries. Figure 4 shows that removing redundant libraries
reduces the root filesystem to 6.55% of its original size. Second,
you can replace the standard C library with a small C library, such as
uClibc, Newlib or diet libc. Such libraries remove the unused functions,
so their size is smaller than Glibc, as shown in Table 2. This table
presents the differences in functionality between the four libraries.
Third, you can use a library optimizer tool named Libopt to rebuild the
libraries that include the only necessary functions for the executable
programs and shared libraries found in the root filesystem. This tool
utilizes objdump and nm to gather information about library object files,
shared libraries and executable programs.

Table 2. Comparison between Different C Libraries

GNU C Library

uClibc

diet libc

Newlib

Size

Largest

Small

Smallest

Small

Compatibility

Good

Good

Bad

Normal

Speed

Fastest

Fast

Fast

Fast

Portability

Yes

Yes

Yes

Yes

MMU-less supporting

No

Yes

Yes

Yes

Licensing

LGPL

LGPL

LGPL

BSD, GPL

Setting

menuconfig

only make

./configure

Note

Standard C library

Needs cross-compiler toolchain

Often linked as static library

Managed by Red Hat

The fifth method for downsizing the root filesystem is to
remove unnecessary documents. You can eliminate some directories, such as
/home, /mnt, /opt, /root, /boot and /proc, if unused. You also can remove
the man, info, include and example directories to reduce the size when
additionally integrating a package into the root filesystem. In general,
an embedded system executes only specific programs, so users can operate
it easily without the help documents or examples in these directories.

The final method is to avoid uncompressing the whole root filesystem
into SDRAM. The root filesystem is compressed to save the stored
space, for example, Flash RAM. However, after the filesystem is uncompressed
into SDRAM, the Flash memory allocated for the filesystem is no longer
necessary. For instance, if the compressed size of the root filesystem
is 4MB and its compression rate is 50%, the system occupies 4MB of
Flash memory and 8MB of SDRAM. Therefore, the system wastes much memory
storage, because of the duplicate data. For this problem, you can use
CRAMFS. CRAMFS is a read-only filesystem, designed for simplicity and
space efficiency. You do not need to uncompress a CRAMFS image before
mounting it. A CRAMFS image is zlib-compressed, one page at a time to
enable random read access. The metadata is not compressed, but is
expressed in a terse representation that is more space-efficient than
in traditional filesystems, such as ext2 or FAT. However, due to the
read-only property of compressed files, random write access is hard to
implement for them. As shown in Figure 4, CRAMFS compresses the
filesystem to 12.77% of its original size.

Now that we've covered the six methods, let's move on to the effect of these methods
on the Wall Project, as shown in Figure 5. First, we used BusyBox
to substitute for the multiple utility programs used in the original
shell. Then, we compiled all the required packages with the parameters
--strip-unneeded and -O2. Next, we used the commands strip and objcopy
to remove the unnecessary contents of packages. Finally, we deleted
unnecessary directories, such as man, info and example. Figure 5(a)
illustrates the result of these processes. However, the size of Wall
was still 139MB. Hence, we had to view the contents of /usr indepth, as shown in Figure 5(b) and (c). In the Wall Project, removing
unneeded documents and files saved 20.6MB of space. About 15.9MB
of space then can be saved by eliminating unused libraries. However,
as you can see, Perl occupied much space in our system. Other methods may
exist to solve this problem, but it is sufficient to consider only what
we have done above.

We found that the optimization of package size is also useful
for downsizing when integrating a new package into the root filesystem. Actually, most programs and libraries are compiled
at optimizing level 2 by default (gcc options -g and -O2) and are compiled
for a specific CPU. On Intel platforms, software is compiled for i386
processors by default. To minimize the package size, you should
not adopt the -g option, which adds the debug info in the execution
files. Additionally, remember to use -strip and --strip-all to remove
all symbols. In more-advanced methods, we used the command readelf to check
for any redundant sections in the execution files, and we used objcopy to remove
those redundant sections. However, this approach may be not efficient
for small programs.

Figure 5. Downsizing Results on the Root Filesystem of the Wall Project