These instructions may or may not work for other Android phones. I'm not sure but I'm guessing that they will only have to be modified a little bit for other Android phones which have a USB cable, for instance, maybe /dev/block/mmcblk0p2 and /mnt/dev/block/\/vold/179\:1 have to be replaced by something else. If your phone isn't yet rooted, I don't know much about how to/if you can root it, but see here: http://forum.xda-developers.com/showthread.php?t=442480 (I think links that page may be G1 specific; but presumably there are other such guides for other devices; search Google for the word "root" plus the name of your phone).

Most of these instructions were written down after the fact, so they may not be correct or complete. I'd recommend reading through a given section before you follow the instructions in that section, to make sure you understand, and to make sure I didn't make a mistake and tell you to do something obviously wrong. Please tell me if you discover any errors or omissions in this document, so that I can fix it (my email is at ).

Some basic concepts and tools

In this document I will often say "sdcard" even though it's really a microSD card.

Various ways to transfer files between the phone and a PC

Here are four different ways:

adb

put the phone into USB mass storage mode

take out the microSD card and put it in a microSD card reader attached to the PC

scp

Details

adb: adb is a program for your PC which interfaces a PC to the phone over the USB cable which came with the phone. More on adb below.

USB mass storage: after you connect the USB cable to your PC and to the phone, a USB icon will pop up in the notification bar on the phone. To make your phone's sdcard available to your PC as a USB mass storage medium (just as if you had plugged in a USB stick -- your phone can essentially serve as a microsd card reader), on the phone, pull down the notification bar, click on the USB notification, then select "mount". While the phone is making the sdcard available to your computer, applications on the phone cannot access the sdcard. You may have to do something on your PC to actually mount the sdcard on that end (on my GNU/Linux system, I do, as root, something like "mkdir /mnt/usb; mount -t vfat /dev/sdc1 /mnt/usb". When you are done, unmount the sdcard from your computer. Then, pull down the notification bar, select the USB notification, and tell it to turn off USB mass storage. At this point, the sdcard is made available to apps on the phone again.

moving the microSD card between the computer and the phone: if you have another microSD card reader (besides the phone, I mean), then you can just take the microSD card out of the phone and put it in that reader, and then transfer files between it and your PC

scp: scp (which is one part of ssh) is a standard GNU/Linux program to transfer files over the internet (most Debian PCs run ssh servers). scp is not installed by default on your phone; however, after you get Debian running on the phone, you will be able to run scp on the phone to copy files between the phone and other computers which are running an ssh server -- and/or you can choose to run an ssh server on the phone, which will allow you to initiate an scp file copy from your PC.

System images

I'm somewhat unclear about this stuff, so some of this section may be incorrect. Please let me know if I got something wrong.

The Android OS is stored on flash media within the phone. There are a number of different flash partitions that store different aspects of the OS.

mtd1 stores the "recovery image", which is what is run when you boot the phone while holding down the Power key and the Home key.

mtd2 stores the "boot image", which I think is mounted on / (the root of the filesystem; filesystem type "rootfs"). This contains the init.rc file, which is a configuration file written in a special format, and which controls what is executed during boot.

The recovery and the boot images are read-only -- if you want to change either of them, you have to unpack the image file, edit what you want to change, repack it, and then flash the new image onto the flash media; see the section "Mounting your ext partition and getting Debian ready on boot (without CyanogenMod?)" below for an example.

mtd3 stores the system image, which I think is what is mounted on the /system subdirectory of the Android OS. The system partition (I mean, the thing which is mounted at /system) is normally mounted read-only, although you can remount it read/write and then directly make changes to it. You can use the "adb remount" command to accomplish this.

If adb says the device is not found, then it may be because you are running the adb daemon on your PC without root permissions (here I'm talking about root permissions with respect to your PC's operating system; not root permissions with respect to your phone). As root, do "adb kill-server" and then "adb shell". If you get into the shell, just exit. The adb daemon will stay running on your PC, but now with root permissions.

When you are in the adb shell, you can exit it either by using the command "exit" or by typing control-D (^D) at the beginning of a new line.

Repartitioning your sdcard to have an ext partition

This is necessary for either of two things:

installing Android apps to your sdcard, or

installing Debian

If you don't want to do either of those, then you don't need to make an ext partition. Note that you have to have root access in order to do either of those things. I think that Windows PCs will not be able to access the ext partition. These notes assume you have have a computer running GNU/Linux.

We are going to resize the initial, FAT partition, and add a new ext partition. The original FAT partition will still be available to Android and will still be used by Android apps for all of the same things it is used for now, so don't make it too small.

Attach the phone to the PC via the usb cable, and put the the phone into USB mass storage mode.

Making a backup

On the PC, as root:

# mount sdcard by (as root) creating a mountpoint;
# first make a mountpoint
mkdir /media/android-fat
# then mount the sdcard onto it
mount /dev/sdc1 /media/android-fat
## but yours might not be sdc1, it might be sdb1 or sdx1 for x=some other letter -- you can use fdisk -l
## on your PC (as root) to view attached disks, whether mounted or not
# then make a backup in case you accidentally delete everything on the sdcard in the subsequent steps
cp -a /media/android-fat ~/android-sdcard-fat-backup
# unmount the sdcard on the PC (but leave it in USB mass storage mode on the phone)
umount /media/android-fat

Repartitioning

On the PC, as root:

# install gparted
aptitude install gparted
# run gparted
gparted

Now, in gparted, repartition the SD card (make sure you don't repartition something else, such as your computer hard drive!!!!). Resize the first partition (it is either FAT16 or FAT32) and create a new partition that is either ext2 or ext3 (ext3 has journaling which won't get corrupted as easily (esp. since Android won't cleanly unmount your linux partition upon shutdown), but the journaling might wear out the flash faster (but some say it doesn't). i chose ext3). note that the ext partition must be the SECOND partition, not the first one (the first one should be the existing FAT partition).

I made the second partition about 750 MB on my 1 GB sdcard. Later I bought a class 6, 8 GB sdcard for under $30 on ebay (I recommend this), and I gave about 75% of the space on that to the ext partition. You may want to distribute space differently than me, though. Note that Android apps will still use the first, FAT partition for all of the same things they use it for now, so most people may want that one to have the majority of the space, unlike in my own setup.

Now, if you try to take your phone out of USB mass storage mode, the phone might complain that the SD card is blank or unreadable, and ask (mb via a notification?) if you want to format it. You should NOT reformat it this way (if you do, it'll delete everything on the card, but that shouldn't be a big problem, since you made a backup above; but it's a pain); instead, reboot the phone. Now it shouldn't complain anymore. At this point the phone should be able to use the resized FAT partition like normal (you may want to test this; i tested it by going into the camera gallery, which should show your saved pictures, if any).

Upgrading from ext3 to ext4

This is optional. DON'T do this unless you are going to run CyanogenMod? or something similar, because the standard Android kernel can't read ext4 (and perhaps the Linux on your PC can't, either -- 2.6.28 was the first Linux kernel with ext4 support; however, there are some data safety patches which aren't there until 2.6.30 (the patches are present, backported, in CyanogenMod?)).

If you want to upgrade your ext3 partition to an ext4 partition (I did), do this on your PC with the phone in USB mass storage mode (I actually did the following on my phone, in the Cyanogen recovery shell prompt (see below), but I think it'll be easier for you to do it on the computer, IF your computer supports ext4):

tune2fs -O sparse_super,filetype,dir_index,has_journal,extents,uninit_bg /dev/sdc2
# again, it might be some other sdXY for you -- be careful not to target the wrong drive!!
# the new options being enabled for ext3->ext4 are apparently extents,uninit_bg,dir_index, so you could just have those. there are some others but i didn't think they were useful (flex_bg, which can only be added at format time and which groups metadata together, extra_isize (nanosecond inode timestamps), dir_nlinks (removes 32000 subdirectory limit))
e2fsck -pDf /dev/sdc2
# -p: fix some things without asking, -D: optimize directory indices, -f: force check even if clean
# you'll get some errors being fixed but that's apparently normal. i got some checksum invalids.

If you want to do it on the phone itself, in the Cyanogen recovery shell prompt (see below) (this is what I did), then here's the commands I used:

tune2fs -O sparse_super,filetype,dir_index,has_journal,extents,uninit_bg /dev/block/mmcblk0p2
# the new options being enabled for ext3->ext4 are apparently extents,uninit_bg,dir_index, so you could just have those. there are some others but i didn't think they were useful (flex_bg, which can only be added at format time and which groups metadata together, extra_isize (nanosecond inode timestamps), dir_nlinks (removes 32000 subdirectory limit))
e2fsck -pDf /dev/block/mmcblk0p2 # -p: fix some things without asking, -D: optimize directory indices, -f: force check even if clean
# you'll get some errors being fixed but that's apparently normal. i got some checksum invalids.

Installing a non-standard version of Android

There are hackers (I don't mean people who break into computers, I mean people who modify computers) who make custom, supposedly improved versions of the Android operating system for your phone. I installed one called CyanogenMod? (I use version 3.6.8.1). It seemed to be slightly faster when I first installed it, although it's tough to be sure because at that time I had also just uninstalled a bunch of apps. It also has a PDF reader app, and it enables multitouch in the browser (although I use the Steel browser app, so that doesn't apply to me). It allows you to install Android apps to your sdcard (if you repartitioned your sdcard to have an ext4 partition), which means you have space for much more than 66 M or so of apps. It adds support for ext4 (with the data safety patches backported from 2.6.30).

it automatically mounts your ext partition, if you have one, upon each boot (to the /system/sd mountpoint)

it has a mechanism to run a shell script that you install upon each boot

Here's how to do it. First, you probably want to repartition your sdcard to add an ext* partition, following the instructions above. I think it should still work if you install this first and repartition second, but I'm not sure, so you may want to repartition first to be safe.

First, make a backup

You're supposed to make a "nandroid backup" (nandroid is a backup program available on the Cyanogen recovery image) before mucking with the system image, but I couldn't get that to work (probably because my tiny resized FAT16 partition couldn't contain the backup image). So here's a different method for making a backup of the contents of the internal flash memory of the phone (not the contents of the sdcard). I never actually used the backup I made this way, so I can't be sure that it's a complete backup...

Install Cyanogen's recovery image

After this (actually before, but you should do it after -- unless you are going to try to make a nandroid backup, in which case you must do it before because nandroid is installed as an option in the recovery image), I installed cyanogen's recovery image. Actually I think you can install CyanogenMod? using the standard recovery image, and never install cyanogen's recovery image, but I think it's nice to have cyanogen's recovery anyways -- it has a shell prompt option, and it has stuff like e2fsck available at that prompt.

I installed v. 1.4, see http://forum.xda-developers.com/showthread.php?p=3915123 (note: i think the standard build flashes /system/recovery.img upon each boot, so unless you replace that file with cyanogen's recovery image, it will only stay there for one boot. but if you're about to install the CyanogenMod? build, then it doesn't matter, just don't reboot yet).

then I rebooted the phone while holding down the Home key and the Power key until it went into Cyanogen's recovery. (at that point I told it to wipe -- but I don't think a wipe is necessary unless you are coming from Hero, but I felt like it -- note that a wipe deletes all of your apps and all of your data!! So most likely you do NOT want to wipe), and next I told it to flash /sdcard/update.zip. That's it.

Install Debian

Some basic concepts and tools for Debian

I assume you have also read the "Some basic concepts and tools" section above.

When I say that I installed Debian, I don't mean that I dual-boot the phone, and that on boot I choose either Android or Debian; neither do I mean that I use Debian software for my phone contacts, or to dial the phone. I boot into Android, and at all times, Android is running; I use the usual Android apps, and the usual Android GUI, to do "phone things" like dial the phone or lookup contacts. But in addition, I can go to a commandline and run any Debian program, for example emacs. It is possible to run graphical Debian programs -- although you have to run them within a special windowing environment. I also run servers and daemons, including cron and ssh.

Both Android and Debian are Linux distributions. When my phone boots, it runs using a Linux kernel. Whether I am running an Android or a Debian application, the same kernel is running; I am not dual-booting or using virtualization such as xen. Debian and Android are running side-by-side, at the same time, under the same kernel. When I run Debian's "top" program, I see all the Debian processes, as well as all the Android processes; similarly, from Android, all of Debian's and all of Android's processes are visible. From within Debian, I can issue a "kill" command to kill Android processes.

So, right now, your phone is running Android with a Linux kernel; "Android Linux". After this, you will also be able to run "Debian GNU/Linux". The difference is the userspace. I have a subdirectory in which I installed a normal Debian directory structure; inside that subdirectory are found subdirectories named bin, dev, etc, lib, sbin, usr, and others. When I run a Debian application that uses a shared library, it looks in this lib directory. But when I run an Android application, the Debian shared libraries are not used; instead, (I think) the Android shared libraries are used (although not at the app level; apps are written in a dialect of Java, and run within a special Java virtual machine called dalvik). Android's libc is not compatible with Debian's.

Before running any Debian program, I (run a script to) chroot into the Debian subdirectory.

Mountpoints

A mountpoint is a location in a filesystem tree to which some sort of storage medium is attached, or "mounted". For instance, a CD-ROM drive might be mounted at /mnt/cdrom, and a hard drive mount be mounted at /, and a USB flash drive might be mounted at /mnt/usb.

In the Android filesystem, the sdcard (which is typically a organized as a single FAT-formatted partition) is typically mounted at /sdcard. The "system" flash partition is typically mounted at /system. We have added a second partition to the sdcard, this one formatted as some sort of ext (ext2, ext3, or ext4), and we are going to mount it at /system/sd. The original, first partition of the sdcard will still be mounted at /sdcard.

To mount something in GNU/Linux, you use the "mount" command. Beforehand you create an empty folder to serve as the mountpoint. If you want to mount the same storage medium at two different mountpoints, first you mount it at one mountpoint, then you use mount with the "--bind" argument at the second mountpoint.

Chroot jails

A "chroot jail" is a subdirectory that you tell Linux to pretend is the root directory. In our example, we are going to tell Linux to pretend that the /system/sd directory is really the root directory (/). The command to do this is called "chroot" for "change root". This pretending is limited to one process, and any descendent processes spawned from it (and the processes spawned in turn from them, etc) (well, it's not necessarily "limited" because then you can chroot some other, unrelated process too -- I just mean that chrooted-ness is inherited by spawned processes). We say that the processes that are pretending are running in a "chroot jail" because they cannot access the outside world (e.g. the parts of the filesystem outside of the /system/sd directory).

Why would you want to do that? In our case, it's because there are certain standard locations in which Debian programs expect to find certain things; for instance, library files are in /lib. We don't want to mess with root directory on the Android filesystem, because that's a pain, and because that root directory is not stored on the sdcard, and because we don't want to accidentally mess up Android. So we're going to put all of the Debian stuff in the /system/sd directory (i.e. the root directory of the second, ext partition of the sdcard), and then run all of our Debian programs in a chroot jail that thinks that /system/sd is root.

Two commandlines, four ways to connect

So, after you install Debian, there will be two different commandlines in your phone:

The Android commandline; in the Android commandline, / refers to the Android filesystem root, and the shell is at /system/bin/sh

The Debian commandline; in the Debian commandline, / refers to the second partition of the sdcard (so, / in the Debian commandline is the same location as /system/sd in the Android commandline), and the shell is at /bin/bash (which is the location /system/sd/bin/bash in the Android commandline)

From the Android commandline, you can chroot into the Debian commandline, i.e. you can run the Debian commandline inside of the Android commandline. But starting from the Debian commandline, as far as I know you can't get to the Android commandline (Debian is imprisoned in a chroot JAIL, remember?). However, using mount --bind we will mount most of the Android filesystem at various mountpoints within the Debian directory, so from the Debian commandline you will still be able to read and write to most of the same files as you can from the Android commandline.

There will be five ways to connect to these commandlines:

On the PC, via "adb shell".

On the phone, via the Terminal Emulator app

On the phone, via the Connectbot app (not available until we install Debian and install and run ssh in Debian)

On the PC, via ssh (not available until we install Debian and install and run ssh in Debian)

On the phone, if you run a VNC server in Debian, you can connect via the AndroidVNC? app (I guess you could also connect from your PC to a VNC server running on the phone, but why would you want to do that?)

"adb shell" and "Terminal Emulator" will start out in the Android commandline (and from there you can chroot into the Debian commandline). Connectbot and ssh will start out (after logging in) in the Debian commandline. AndroidVNC? will start out in a graphical environment running within Debian.

(you don't need to install any of those unless you want them. less is a better replacement for the program "more". emacs is a text editor. openssh-server and openssh-client let you use ssh. mutt is an email client. offlineimap is a program that can synchronize email with another computer. unison is a general file synchronization program. python is my favorite programming language. awesome is an X window manager. screen is a terminal multiplexer. tightvncserver is a VNC server, which we will use in place of an X server on the phone. qiv is a picture slideshow program. Note: in this guide I have not yet written how to get offlineimap and mutt working yet. Currently, I can synchronize my email with my computer, and read it on my phone with mutt, but I cannot yet send it from my phone. After I get that working, perhaps I'll add configuration instructions to this document)

At this point df -h showed something like:

...
/dev/block/mmcblk0p2 703M 438M 230M 66% /
...

(that's total/used/free; 230M isn't much space, so the next week I got a class 6, 8 GB microSD card from ebay for under $30, and gave 75% of it to ext4, though, so now I have much more room -- i recommend buying one of these)

At this point, if you wish, you could use the instructions in the section "How to get into Debian manually", on the page tips-computer-android-g1-debian-cyanogenMod-misc.

(note: cat /dev/tty doesn't always work right if you just paste the stuff in at the same time that you pasted in cat /dev/tty, so check to make sure it came out the way you wanted)

To repack and replace the boot image:

cpt mtd2.img mtd2.img-old # save the file mtd2.img-old as a backup in a safe place on your computer
repack-bootimg.pl mtd2.img-kernel.gz mtd2.img-ramdisk mtd2-new.img
adb push mtd2-new.img /sdcard/mtd2-new.img
# now we install the new image as your actual system image
adb shell 'cat /dev/zero >> /dev/mtd/mtd2'
# you'lll get a "write: No space left on device", but this is OK
adb shell 'flash_image boot /sdcard/mtd2-new.img'

Now unmount stuff cleanly, and then reboot, just to check that you didn't totally break everything:

If it works, then you can go on to the next section (don't skip it; the next section tells you what to put into userinit.sh)

Mounting your ext partition and getting Debian ready on boot (with CyanogenMod; or, if you don't have CyanogenMod, then after the previous section)

Now we'll make our userinit.sh script. This script will run at boot and will mount a bunch of stuff.

Currently, on CyanogenMod? builds ("ROMs"), if you put a shell script in /system/sd/userinit.sh, it will run on boot (it is run by /bin/a2sd, which is run by init.rc on boot). In the previous section, we showed you how, without CyanogenMod?, you can make the phone run a script at /system/userinit.sh on boot.

So, if you are using CyanogenMod?, then this will be placed at /system/sd/userinit.sh. Otherwise, it will be placed at /system/userinit.sh.

and the output of the mount command should include mmcblk0p2 mounted on /system/sd and devpts mounted on /system/sd/dev/devpts.

NOTE:

if you are not using CyanogenMod?, then just before you mount /system/sd, i read in one place that you have to do this first:

mknod /dev/mmcblk0p2 b 179 2 1000

but it wasn't necessary for me, so I didn't include it above. But if things aren't working, you might want to try it.

Make a script to run Debian

Technically, it probably isn't right to say we will be "running" Debian. We never go through the Debian boot process. In this section we'll make a script called "d" that chroots into the Debian subdirectory, and runs the bash shell inside that Debian chroot jail.

Now try it out through adb shell; by giving the command "d", you should see:

adb shell
# d
localhost:/#

And now, on the phone, run the Terminal Emulator app. You get a prompt; enter "d"; if you're on CyanogenMod?, a window will popup labeled "SU Request". You can select "Allow" or "Always Allow". Then you're back in the terminal, in Debian (you can tell you're in Debian because of the "localhost:/#" prompt)

Doing some Debian initialization upon boot (including ssh server)

Now, the stuff in Debian's rc directories (/etc/rc*.d) aren't being run; Debian isn't being "booted", it's just being mounted after boot and chroot'd into. Rather than trying to run the stuff in some of those directories automatically (rc2.d has some "application level" stuff like cron, ssh server, exim4 server, and a syslogd), I prefer to put it into userinit.sh, since we are in a rather special situation, and don't want too much cruft in memory or taking time at boot. So I did:

Mounting the sdcard in Debian

For some reason I don't understand, apparently the --bind mount command for the sdcard in userinit.sh doesn't work -- maybe the sdcard isn't mounted yet at that point? Anyway, I solve this by putting the following at the end of my .bashrc:

. ~/.bashrc-local

And then creating a file at ~/.bashrc-local containing:

mount -t vfat /mnt/dev/block//vold/179\:1 /mnt/sdcard/

Public keys for sshing in from Connectbot (doesn't work yet)

Already, you can use Connectbot to login to your own phone (the address is root@localhost; I use Connectbot rather than Terminal Emulator because i think its keyboard shortcuts are better than Terminal Emulator and Better Terminal Emulator), but you have to enter the password, which is annoying. I don't know how to get rid of that annoyance -- here's what I tried, but it didn't work. Maybe you can tell me how it can be done.

On your phone, in Debian, do the following, and enter an empty password when prompted:

Now, within connectbot (the screen where you select a session, not inside a session), press menu and then "manage pubkeys","import", and then select id_dsa. Now, click on the id_dsa entry that has appeared in the key list, and it will turn from red (locked) to green (unlocked). Now, long click on it and select "load on start".

Now, on the phone, in Debian, do:

rm /mnt/sdcard/id_dsa

If you want to be able to login from your PC to your phone without a password (this part DOES work), generate an ssh key on your PC, and then append the contents of ~/.ssh/id_dsa.pub (from your PC) to the file ~/.ssh/authorized_keys (on your phone). And if you want to login from your phone to your PC without a password, append the contents of ~/.ssh/id_dsa.pub (from your phone) to the file ~/.ssh/authorized_keys (on your PC).

Connectbot root@localhost shortcut

Login to root@localhost using Connectbot. Press the Home key. Long click on the desktop. Click Shortcuts. Click Connectbot. Click root@localhost. Now you have a desktop shortcut to connect to root@localhost.

X-windows

As of this writing I don't think there is an X client for Android. But there is an app that is a VNC client. So what we'll do is run a VNC server on the Debian side, and we'll download the VNC client app, and connect it to localhost.

In Debian, on the phone:

aptitude install tightvncserver

You probably also want a window manager. I'm trying out "awesome", but I've never user it before so I don't know enough to recommend it yet. In Debian, on the phone:

export USER=root
vncserver -geometry 480x320
# (enter a password, but after that just enter "n" to decline to set a view only password)
vncserver -kill :1
nano ~root/.vnc/xstartup # or use your favorite text editor

and in the text editor, comment out the line for xsession: # #/etc/X11/xsession

and add two lines to the bottom: awesome& xterm

now save and quit it.

Now put the following in a script file (I put mine at "/usr/local/bin/x"):

I didn't setup the vncserver to run automatically on boot, because I don't want to spend scarce system resources (memory, CPU) on it. I don't expect I'll use it very often. But it's nice to know that every now and then, it's possible to run Debian visual applications on the phone.

If you use it alot, you should know that it create log files and pid files in the .vnc folder. You should maybe do something about that, since it causes needless flash writes. Maybe you could mount /tmp on a tmpfs (see below), link .vnc to /tmp/.vmc, and copy the files passwd and xstartup in upon each boot.