In this blog I will cover steps to create an x86 OpenWRT live USB on a Windows 7 PC. The steps are fairly simple. Just downloading files, move them around, and type a few commands. No Linux experience necessary. They can be easily adapted to Windows Vista or Windows XP if you have an older PC.

With a little more work, the technique discussed here can be adapted to create a bootable-x86-ISO/live-USB combination, for PCs that can’t boot from USB. That is something in my pipeline.

And of course Live USB can be created on a Linux system, but I don’t plan to write up the instructions. Leave a comment if you need help with that.

Finally, the theory of operation comes after the instructions. I think this live USB concept is a big improvement over my livecd idea (see Creating OpenWrt x86 Live CD with USB Support). I plan to keep up with newer OpenWRT releases.

Background

Occasionally I want to try different OpenWRT configurations while keeping the main installation intact. Although I can already do that with my custom live CDs , they are still cumbersome and slow when switching back and forth. An even better method is to boot from USB to a selection of firmware versions or configurations.

To create a bootable live USB using the steps discussed here, you’ll need

OpenWRT Files

Three components make up one OpenWRT configuration: a Linux kernel, an initial rootfs image, and an OpenWRT disk image. All three are stored as files on the USB drive, and must have the same firmware version. For example, to boot your PC to OpenWRT Backfire 10.03, you need the 10.03 kernel, 10.03 OpenWRT disk image, and 10.03 initramfs image.

When changes are made to an OpenWRT router, they are stored in the disk image. The linux kernel and rootfs image are not changed. Therefore some configurations can use the same Linux kernel and rootfs image, as long as they are of the same version. Also, the disk images are initially compressed (.gz format). They are decompressed when used for the first time.

For clarity I gather files of the same version in one folder, and use the version number as folder names.

Official OpenWRT website has the kernels and disk images for many different platforms and versions. The rootfs images are modified disk images that I created, and are hosted on SkyDrive. Here are the links to all files.

Insert the USB drive. For this discussion, I’ll assume the drive letter is “E:”.

Create two folders on “E:”: “10.03” and “10.03.1-rc5”.

Follow the links given above, and download version 10.03 kernel, disk image, and rootfs image. Move these files to “E:\10.03”.

Again, follow the links and download kernel, disk image, and rootfs image of version 10.03.1-rc5. Move these files to “E:\10.03.1-rc5”

Navigate to “E:10.03.1-rc5”. Make a copy of the disk image. Name the new copy “config-c-openwrt-x86-generic-combined-ext2.img.gz”.

Rename the original from “openwrt-x86-generic-combined-ext2.img.gz” to “config-b-openwrt-x86-generic-combined-ext2.img.gz”.

When you’re done, the USB drive and folders should look like these snapshots.

Making The USB Drive Bootable

We’ll use Syslinux as the bootloader.

Download Syslinux. Follow the link given above and choose the ZIP archive because it is easier to work with on a Windows PC.

Extract the content of the archive to a new folder. For this discussion, let’s assume that folder is “C:\syslinux-4.04”.

Start a command prompt as administrator. Press the Windows Logo key and type “command prompt” in the search box. Right click on “Command Prompt” under “Programs”. Select “Run as administrator.” Answer “Yes” in the UAC pop-up window. An administrator command prompt should now be open.

Go to the executable directory and run Syslinux. Again, assuming the USB drive’s letter is “E:”, type

64-bit systemcd c:\syslinux-4.04\win64
.\syslinux64.exe -m -a E:

32-bit systemcd c:\syslinux-4.04\win32
.\syslinux.exe -m -a E:

“-m” places Syslinux on the master boot record of “E:” drive. “-a” marks the partition active.

Label the flash drive. In the same command prompt, typelabel E: OPENWRTThis disk label (“OPENWRT” here) will identify the USB drive to Linux kernel. You can choose a different label, but remember to use it in the Syslinux configuration file discussed later. Also, remember that Windows uses only uppercase letters in the label but Linux is case sensitive.

Configuring Syslinux

The final step is to create a configuration file for Syslinux. This file is named “syslinux.cfg” and stored at the root level of the USB drive. That is, “E:\syslinux.cfg”.

In “syslinux.cfg” list all OpenWRT configurations. There are four lines for each configuration:

“label”: The label of a configuration. Entered at boot prompt to make a selection.

“kernel”: The path to a kernel file on the USB drive. Use “/” instead “\” and discard the drive letter. For example, the kernel file of of Config A isE:\10.03\openwrt-x86-vmlinuz
That makes the Syslinux entry/10.03/openwrt-x86-vmlinuz

“rootvol”: USB drive label. Add the prefix “LABEL=”, and remember to use all uppercase letters. For our USB drive, add the following text to the “append” line:rootvol=LABEL=OPENWRT
“rootvol” is my addition to OpenWRT.

“rootimage”: Path to disk image of a configuration. Discard drive letter and use “/”. For example, Config B uses the disk imageE:\10.03.1-rc5\config-b-openwrt-x86-generic-combined-ext2.img.gz
so add the following text to Config B’s “append” linerootimage=/10.03.1-rc5/config-b-openwrt-x86-generic-combined-ext2.img.gz
“rootimage” is also my addition.

“console”: Specify the linux console. If you have a video card and a monitor, use “console=tty0”. If your OpenWRT PC is headless, use “console=ttyS0,38400n8” to enable the serial port.

“reboot”: original from OpenWRT

These entries can create a simple user interface for selecting a configuration.

“say”: Display a message.

“default”: The default choice

“prompt”: 1 to turn on boot prompt.

“timeout”: time to wait for user’s selection, in 1/10 of a second. (100 = 10 seconds)

“ontimeout”: automatically selected choice on timeout.

The following configuration file will display the three choices discussed here. It will then wait 10 seconds for a user to enter “a”, “b”, or “c”. If a selection is made, Syslinux boots the chosen configuration. Otherwise, it boots Config A.

Live USB Ready

That’s it. Just eject the drive from Windows and you have a Live USB drive with two OpenWRT configurations. When booting from this drive, you’ll see a list of 3 choices and have 10 seconds to choose.

Always Shutting Down Gracefully

I noticed that the FAT/FAT32 file systems are quite susceptible to data corruption. During development I lost many files by pressing the reset button on my PC. Always shut down your system gracefully to avoid data loss.

Why Booting from USB Fails

From initial rootfs Kernel starts an initialization program, usually “/init”, to set up essential system components, including root file system.

If the necessary drivers are not built in the kernel, they are loaded from kernel modules.

These modules are included in the initial rootfs because it is custom built for each system (by “dracut” or “mkinitrd” usually).

Finally, the control is transferred to the “real” initialization program on the root file system.

The official x86 OpenWRT distribution differs from this flow. It skips the second step. The OpenWRT kernel itself initializes the root file system and transfer controls to it. This isn’t a problem when the root file system resides in an ext2 partition of a IDE or SATA hard disk. The ext2 and ATA drivers are already in the kernel. The USB drivers however are not, and this turns booting from USB into a Catch-22: The kernel can’t access the root file system without the USB modules, but the USB modules aren’t available because they are on the root file system.

Choosing an Alternative

One solution to this problem is to add the USB drivers and re-compile the kernel. It works but has two problems. It is opaque because changes in the kernel binary can’t be easily examined. It also requires the kernel on the official disk image to be replaced. That task is not simple for a novice.

Another solution is to use a combination of an initial rootfs image (initramfs.img) and Syslinux. This is a much better solution. The initial rootfs image can be easily examined. Setting up Syslinux is much simpler than replacing the kernel. Additionally, Syslinux works with FAT, which is the default file system on most USB drives, and it can be set up from a Windows PC, thus broadening the user base.

Setting Up USB Root

As mentioned earlier, a critical role of initramfs.img is to set up the root file system. In this case, the root file system is a image file on a USB drive. To set up the USB root, kernel runs “/init” of initramfs.img. “/init” is a shell script. It sets up the root file system in these steps:

Load all modules in initramfs.img. They include drives for USB and FAT file system. (See “Creating initramfs.img” on adding these modules.)

Using a parameter on kernel command line, identify the USB drive that holds the root file system. Mount that USB drive.

Again using a kernel command line parameter, find the root file system image file. Decompress the file if necessary.

Check the size of the image file. Pad it to ensure enough space is allocated for the root partition.

Associate the image file with a loop device, run fsck, and mount it. Now the root file system is online.

Add a step in OpenWRT’s pre-init flow to link “/dev/root” to the loop device.

Creating initramfs.img

“initramfs.img” is based on the official disk image, with the addition of USB and FAT modules and removal of others. I will describe just the general steps here because a detail discussion will be too long.

Write the official x86 disk image to the storage of a kernel virtual machine.

Start the KVM and configure its network interface so it can access the internet

Use opkg to add and remove modules. The final list of module is

base-files

kmod-nls-base

kmod-usb2

libuci

busybox

kmod-nls-cp437

libblkid

libuuid

e2fsprogs

kmod-nls-iso8859-1

libc

losetup

hotplug2

kmod-usb-core

libext2fs

opkg

kernel

kmod-usb-ohci

libgcc

udevtrigger

kmod-fs-vfat

kmod-usb-storage

libpthread

kmod-loop

kmod-usb-uhci

librt

Turn off the KVM.

Use “kpartx” to make the OpenWRT root partition available on the host system.

Use “scripts/gen_initramfs_list.sh” from a kernel build directory to create a file list. This list includes basic device nodes, “/init” script shown above, and the root file system of the OpenWRT KVM.

Edit the list to change absolute paths to relative. Also delete unnecessary files like “.svn” or “.git”.

Use “usr/gen_init_cpio” from a kernel build directory to generate the initial rootfs. Compress it with gzip.

Thank you. I’m glad you like it. RE: not latest firmware: I don’t quite understand your question since 10.03.1-rc5 is the latest released firmware. Are you asking why I don’t use the snapshot firmware? Being a snapshot, it is unstable and changes too quickly (daily I think). I need some stability, and I just can’t keep up with the changes.

I do have one problem, I installed just the 10.03 part on a 256 MB stick
I then istalled some small packages + samba + transmission all went well.
Later I wanted to install hdparm to set drive parameters and is telling me that it run out of disk space.
Being a novice I don’t know what to do to increase the size of the now 48MB root partition since there is plenty more space on the stick.

Hi, Luis, glad you find my write-up useful. Regarding the root partition size, do you have root privilege on a Fedora, Ubuntu, or some other desktop Linux? If you do, I’ll write up the instructions for you.

Hi macbruins , thanks for your quick response to my inquiry actually it was my fault, the /root partition was being filled by the misconfigured transmission package which was downloading into that partition.
Anyway I did found another problem, I can change the password on the command line, but whenever I go into the web interface it does not ask for password, I try to change it there and it returns the following error.

Luis, glad you solved your problem.
RE new problem: Is there a type in this line?/usr/lib/webif/apply.sh: line 353: /etc/init.4/httpd: not found”
Are you sure it says “init.4” and not “init.d“? I’m going to guess it’s supposed to be “init.d” because I’ve never seen “init.4” before. Now, missing /etc/init.d/httpd means you should re-install OpenWrt. /etc/init.d/httpd is a script that starts the web server for the OpenWrt GUI. It’s pre-installed in the official disk image. It wouldn’t be missing unless you deleted it or something pretty bad happened. If something bad happened, starting from scratch is safer than trying to repair the installation.

Hey, great write up… I did get the backfire cross compiler working earlier (x86 and ar7xx) but didn’t didn’t do much with it. Now I’d like to get back to that project, and this will be a nice jump start.

You mention “And of course Live USB can be created on a Linux system, but I don’t plan to write up the instructions. Leave a comment if you need help with that.”

Do you use syslinux to create a Live USB on Linux? I thought the ‘-m’ option only works on Windows… do you use a different boot loader for this, or how do you get the MBR updated? Thanks,

There are probably mistakes in your syslinux.cfg. kernel panic often happens when the kernel can’t access the disk image. Since you can boot 10.03 but not 10.03.1-rc6, probably there are errors in your 10.03.1-rc6 section. Check all lines carefully. If you used my example as the starting point, make sure you change all “10.03.1-rc5”.

check file names? Don’t know what else to say. Other possibilities exist but are not as likely. I want to avoid a debug session here. If you set up another location/site/blog, perhaps we can continue there.

I followed the instructions above. However, when I boot the PC with the USB stick, the OS starts loading and within a split second the boot process ends with a kernel panic and halts. Since the screen scrolls so fast, I cannot even see what the error is. Any ideas on what the issue could be? Is the initramfs only for specific hardware? Anybody else have this issue?

Sorry for the late reply. Does your PC have a serial port? If you use that as your console, the text will be captured in the terminal program’s scroll buffer. If you’d like to discuss your problem further, please send me email at “macbruins at gmail dot com”.

I wouldn’t do that. I tried mounting /dev/sda0 a few times, but often the USB disk or the root disk image became corrupted. I did not find the cause. If you need to access USB storage, either create two partitions on a disk or get two USB disks. Either way, you can modify /etc/config/fstab to mount /dev/sda1 automatically.

I created 2 partitions. right now it works with everything loaded from ramdisk, including usb storage driver… e.g. the root is in ramdisk and i can mount usb stick partition2 to /root so that i can run app from it. That is also not very handy, as I have to rebuild it for every minor change in config files. Any idea how can i install root filesystem to usb stick and mount it?