The system consists of:

Android phone — mounted on the car, captures video frames of the road
ahead using its built-in camera at ~15
fps. An app running on the phone
connects to a server running on a
laptop computer via wifi and streams
176x144 grayscale video frames across
the connection.

Computer — runs a little Java app called "Driver" which acts as both a
TCP server, receiving streamed image
frames from the phone and a user
interface allowing a human driver to
control the car with the cursor keys
or mouse. In record mode, the video
frames are saved to disk, labelled
with the current control input coming
from the human driver. The neural
network is trained using these
labelled frames in a separate
environment on the computer. Trained
parameters are saved out to files
which are in turn read by the Driver
app... which in auto mode can feed
incoming video frames directly to the
neural network and steer according to
its predictions, by sending
instructions over a serial interface
connected to an...

Arduino Uno — connected to the computer via USB and hacked to connect
to and simulate keypresses on the
car's radio controller PCB (as
described below).