Debian On Flash

How to install and boot Linux Debian on Flash storage using the JFFS2 file system.

There are already projects on that matter, like RootSync. But these currently
seem to be in an early stage. So I tried out what is possible with existing
Linux distributions. Here are the results:

Introduction

Compact
Flash drives are, compared to a hard disk, extremely low power
devices. Of course, the price for a certain amount of storage is still
much more expensive on a Compact Flash drive than on a hard disk. But if
you don't need the capacity of a hard disk, you can save some energy (and
money[1]) by putting all data including the operating system on a
Compact Flash. The idea of the following project is to build a really low
power workstation or server which would boot a standard Linux off a
Compact Flash.

With an adapter, a Compact Flash can be used as a replacement for an
IDE disk. However, the Flash technology allows only a limited number of
write cycles for each memory cell, and most file systems have "hot spots"
which are written very frequently, which could render a flash drive
unusable within short time. Therefore a technique called wear levelling
is required, to distribute write access accross the medium. Rumors
say that this is done in the controller of the Compact Flash Cards, but I
have seen this only explicitely noted in the product description for
industrial grade Compact Flash cards. I am really not sure whether cheap
consumer grade Compact Flash Cards also perform wear levelling. The Compact Flash
Association does not mention wear levelling. There is only a
"defect managment and error correction", which, i.m.o. does not
prevent defects, but merely replace bad areas
after they have become faulty. When all spare sectors
are exhausted, the capability of the drive to replace bad blocks
eventually comes to an end, in which case the drive is defect despite of
"defect management and error correction"[2]. In order to use consumer grade cards safely, the wear
levelling task can be done in software. This is where a file system like
JFFS2 comes into play.

There is no obvious way to install Linux directly into a JFFS2
partition.

To overcome the first issue, a
block2mtd driver is available.

The second point is something to think about. Adding yet another
layer on top of an IDE emulation on top of a flash memory storage does not
increase performance. If performance is an issue (and
money is not), stop reading here. Buy an industrial
grade Compact Flash instead and proceed normally.

The third issue is what the remainder of this article is about. This
might also be useful for really embedded solutions where flash memory is
directly addressable by the CPU and does not have its own logic.

You can use Compact Flash drives off the shelf. We will leave
them formatted as FAT drives. The
Linux file system will be inside one or more container files. If you
like, you can attach the Compact Flash to any Windows box and copy
these files for backup.

We will have to customize the Linux boot procedure. This will
give you some insights of what is going on at a stage where most users
see only a progress bar.

My favourite Linux distribution is Debian, and the following
description applies to Debian Etch. However the general principle on
making Linux boot a JFFS2 drive can certainly be adapted for any other
Linux distribution.

Installing Debian

As mentioned above, there is no obvious way to install Linux
directly into a JFFS2 partition. So we don't.

Instead, we install Linux into a separate drive, which we attach
temporarly to the target. An USB disk is fine, even if the target does not
know how to boot USB disks. We won't boot that installation directly. So
just go ahead and install Linux onto that drive. I prefer the netinst CD, but this
requires a fast internet connection.

Note

Running the install CD with install
priority=medium gives you more choices. Especially, you
can choose a more recent kernel. However, you can update your kernel
later anyway.

Note

If your low-power system happens to be VIA C3 based, don't use a
i686 kernel. The C3 will not run i686 code.

Install the Compact Flash in the location where you will later boot
it. This might be an adapter which makes it an IDE drive. Compact Flash
drives come preformatted as FAT drives.
This is OK. Copy the kernel and the initial ram disk from your fresh Linux
installation to the root directory of the compact flash. Name them
vmlinuz and
initrd.img, respectively.

Preparing the flash drive for booting

We will boot the flash using Syslinux. If you can't boot your
newly installed Linux yet, you can use an already working Linux (a Knoppix system running from CD is
fine) to install Syslinux
on the Compact Flash. Create a configuration file
syslinux.cfg on the root of the Compact
Flash as follows:

default vmlinuz
append root=/dev/sda1 initrd=initrd.img

Replace /dev/sda1 with the path to
your (temporary) installation disk.

Install a master boot record on the flash. This can be done with the
command install-mbr. If you are using
Knoppix to prepare your flash
disk, this command will be already available. On a vanilla Debian, you
will probably have to install the package
mbr first.

You should now be able to boot your new Linux installation, using
the Compact Flash merely as an initial boot medium.

Note

If you update your kernel, don't forget to update the copies of
the kernel and the initial ram disk on the Compact Flash too.

We will later extend the boot configuration to allow to boot the
system we are now installing on the flash itself.

Create the Flash image(s)

Install the package mtd-tools. This
will provide mkfs.jffs2.

Create an image of a size that suits your Compact Flash (and the
expected final size of your Linux system). Use a command like the
following:

mkfs.jffs2 -d empty --eraseblock=65536 --pad=nnnnnnnnnn -o rootfs.img

Where empty is the name of an empty
directory. We will add files later. See below. The
eraseblock parameter should match the
actual technology used on the drive - which we usually don't know in case
of a Compact Flash drive. 64k might be a useful guess, but probably the
setting will not be optimal. You have been warned. The
pad parameter specifies the actual size
of the image. In bytes. For example a 800MB container (fitting nicely in a
1GB drive) has a size of 838860800. The size should be a multiple of the
eraseblock size (otherwise
mkfs.jffs2 will round it to an apropriate
value). The output parameter should point to your mounted Compact Flash
drive.

If you want to keep parts of the file name space like
/usr and
/var on separate containers, you are free
to to so. Just repeat the step above with other file names and sizes for
the additional containers. I.m.o this only complicates things, but it is
possible.

Mount the Flash image

Setup loop device(s)

losetup /dev/loop0 /media/hda1/rootfs.img

Where /media/hda1/rootfs.img is
the path where you mounted your Compact Flash drive.

Create the block2mtd device(s)

mknod /dev/mtdblock0 b 31 0

If your setup requires more than one image, repeat with increasing
index numbers (minor number)

Load the block2mtd driver

modprobe block2mtd block2mtd=/dev/loop0,65536

Note

The size parameter (65536) must match the
eraseblock parameter in mkfs.jffs2 above.

If your setup requires more than one image,
specifiy the remaining loop devices with additional
block2mtd=xxx parameters:

Unmount the Flash image

Unmount the image

umount /mnt

Unload block2mtd driver

rmmod block2mtd

Release loop device

losetup -d /dev/loop0

It is now safe to unmount the Compact Flash. But before we do, we
first

Create the Initial RAM Disk

When Linux first boots, there is just the kernel, a boot command
line, and an embryonal file system called the Initial RAM Disk. The latter
is normally created by the installation procedure, and updated by system
updates, but in general, we don't have to worry about much. Booting a JFFS
is different. We have to teach the Initial RAM Disk new tricks. Here is
how. The following requires the package
initramfs-tools. It usually comes with
the installation. If you don't have it, install it now.

Add required modules

Create a file in
/etc/initramfs-tools/hooks with the
following content. You can choose any file name, say
mtdsetup.

Make the script executable. It will be called during the boot
process just before the root file system is mounted.

Note

The MTD emulation stuff is made conditional with a boot
command line parameter mtd. This
makes the same RAM disk usable for both a flash disk and a
conventional boot. If you don't need this, you can leave that
out.

Note

If you want to put /usr and
/var into separate containers, just
add the required losetup calls and add more
block2mtd parameters to the
modprobe call as decribed above. The setup provides up to
eight loop and
mtdblock devices for that purpose.
The additional file systems will not be mounted until a later phase
in the boot process, but the
block2mtd setup must be done
here.

Post mount script

Create a file in
/etc/initramfs-tools/scripts/local-bottom
with the following content. You can choose any file name consisting of
alphanumeric characters. Don't put a dash (minus) in the
name![3]

Make the script executable. It will be called during the boot
process just after the root file system has been mounted, but before
it is made the root of the file system.

Note

The original mount point
(/flash) of the flash disk will
disappear when the embryonal root file system is replaced by the
real root. If we do nothing, the device will still be marked as
mounted, and the shutdown procedure will try to unmount it, but
cannot find a mount point. There are two possibilities to solve
this: either move the mount point to a path in the new root
(mount --move). This will cause the
flash disk to remain mounted and this will be visible. The
alternative used here is a lazy dismount. This will cause the device
to be dismounted when the last file access is gone.

The last commands create an empty dmesg file to keep the boot
procedure happy which tries to chown
it and complains if it is not there.

Create RAM Disk and setup Syslinux boot loader

We are almost done. We now recreate the initial RAM disk using the
command

update-initramfs -u

Copy the RAM disk image to the root of the flash disk. It should
be safe to replace the RAM disk image created above.

Note the mtd parameter at the
end of the kernel command line. It triggers the
jffs2 handling in the scripts.

Note

You will find messages like

block2mtd: Overrun end of disk in cache readahead

in the system logs. This is harmless.

System maintenance

After running the system for a while, you might encounter the
following problem:

Locale woes

After a system update, the defined locales might have disappeared.
The reason for this is that the program
locale-gen, which is supposed to
generate new or updated locales, uses file mapping to map the file
/usr/lib/locale/locale-archive directly
into memory. However file mapping does not work with a compressed file
system such as JFFS2[4], and generating the locales fails.

Solution: Generate the locales manually, after replacing the
directory /usr/lib/locale with a piece
of RAM disk as follows: