Seeing the world through a retina

Original image from a CCD camera.

Same image seen by a retina (right eye, fixation
at center).

The retina is very different, in terms of scene digitization, from
a CCD camera which uniformly samples a visual scene. The retina has a
specialized central region (the fovea) which images the immediate
neighborhood of the point of fixation with a much higher resolution
than peripheral regions of the visual field. Around the fovea, the
sampling density (i.e. the actual resolution) of the retina falls-off
quickly as we move towards the periphery.

Another complication is the presence of a blind spot. At the blind
spot, all nerve fibers coming from the photoreception and processing
layers of the retina leave the eye to form the optic nerve (about one
million fibers for each eye). Because of the presence of this massive
bundle of nerve fibers, there are no photoreceptors at the location
where it leaves the eye. As a result, a small area of the visual
field, of about 5 degrees of visual angle in diameter and at 15
degrees of eccentricity from the fovea, on the temporal side of the
visual field, is not represented by the retina. Although you are not
aware of the presence of your blind spots, they are easy to find: with
one eye closed, fixate a point in front of you. Without moving your
open eye, move your finger at arm's length in front of you, from the
fixation point towards the periphery, in the horizontal plane. At an
angle of about 15 degrees, your finger will disappear. At smaller and
larger angles, you will be able to see your finger.

While we have three types of photoreceptors for color perception
(roughly sensitive for red, green and blue wavelengths), the central 1
degree of the visual field (the fovea) is almost entirely devoid of
blue-type photoreceptors.

Finally, we move our eyes on average 100,000 to 150,000 times every
day. The goal of these, usually very rapid ("saccadic", up to
velocities of about 800 degrees per second), eye movements is to
always move the fovea towards the objects of current interest, in
order to examine them with the region of the retina that has finest
detail.

We are not aware of most of the phenomena mentioned above. Here is
how the world would look if we were...

Simulation of how a video clip looks like seen by a retina. This is
our original demo which we developed with Prof. Christof Koch at
Caltech and which appeared in the May 2000 issue of Discover
Magazine.

These clips are another simulation of how the raw inputs to your
retina may look like when you inspect a video clip and make eye
movements, now using an eye-tracker to capture the eye position of a
human observer watching the video clip.

First, have a look at the first clip (top). This is a normal video
sequence filmed with a camcorder. We recorded eye position from a human
subject as he watched this video clip.

Now have a look at the second clip (middle). This is the same as
the first one, except that each frame has been shifted to the current
eye position of a human observer; thus this version shows you the
distribution of light intensity that hit the subject's retina as he
was making eye movements to explore the clip (assuming a black border
around the clip). Interestingly, while the motion of the scene induced
by eye movements is obvious and extremely annoying in the second clip,
subjects do not perceive the same annoyance when they execute their
own eye movements, although the visual input hitting their retinas is
essentially the same as what you can see here by maintaining your gaze
fixed at the center of the images.

The third clip (bottom) is the same as the second one, except for
a slightly wider field of view and grey background.