The introduction of a technological
breakthrough in digital imaging that has global significance is one of
the things photographers live for. But, for a product to be that significant,
it must be one of the basic building blocks that all camera designers
use. One such basic building block of a digital camera is the chip on
which the image is captured. The Foveon (www.foveon.com) X3 CMOS imaging
chip is both a technological breakthrough and a basic building block.
Is it important enough that photographers might end up seeing most digital
camera advertisements saying something like "X3 Inside"?

Backtracking To Understand
The Future
To understand the how, why, and, even, "who cares" about the
workings of the Foveon X3 chip means backtracking a bit to clearly understand
how a traditional CCD (or CMOS) chip sees, interprets, and records color.
For the sake of this discussion let's call the current crop of imaging
chips "mosaic" capture devices and imagine that their surface
is a three-colored checkerboard pattern of tiny green, red, and blue squares
called pixels. Running horizontally, the top row of pixels are green squares
alternating with red squares and the second row, just beneath it, is made
up of green squares alternating with blue squares. The pattern repeats
itself over the entire face of the chip€green and red, green and blue,
green and red, green and blue€etc. Each of the little colored squares
gets its color sensitivity from a little (again tiny) filter of that color
resting on top of the pixel.

This
illustration shows the difference between a current mosaic
chip and the new Foveon X3 CMOS chip. In both, notice the
pixel in the lower left corner. In the mosaic example the
pixel is just reacting to the green component of light while
with the multilevel technology of the X3 all three colors
are recorded.

Forgetting the difficulty of
manufacturing and correctly placing these gazillions (a technical term!)
of little filters for a moment, let's look at what happens when a ray
of light strikes one filtered pixel. Because of the little filters, the
green filter on top of the pixel only allows green light into that pixel,
a red filtered pixel only lets red light into it, and finally the blue
filtered pixel only receives blue light. Because of this, each pixel is
only reading 1/3 of white light that is falling upon it that matches its
filter; either the red, green, or blue component. Furthermore, considering
the layout of the red, green, and blue pixels, the final image is comprised
of 25 percent red pixels, 50 percent green pixels, and 25 percent blue
pixels.

Let us assume that we are concentrating
on a red filtered pixel for a moment. Imagine it is being struck by red
light and is therefore reacting by generating a tiny amount of electric
current. It is also surrounded by eight other pixels, four green and four
blue. When a digital camera processes this information it looks at all
the neighboring pixels surrounding our red example pixel and depending
on whether the surrounding pixels are being activated by the light it
can decide on the color for all nine pixels together as a group.

If, for example, in addition
to the red pixel being active all of the blue pixels are active but none
of the green ones are, the computer assumes that the red and blue pixels
taken together without any of the green pixels being active means the
area being examined is purple (red and blue together) and it registers
the area as purple when it maps out the image. If, on the other hand,
all eight pixels (four green, four blue) surrounding our red pixel are
all reacting to the light the computer will decide that the total area
in consideration is white.

This
illustration explains how film uses multiple layers to record
different colors of light. When you compare it to the X3
technology shown (far right) you'll see that the X3 chip
works in approximately the same manner.

In effect, the computer is
making an educated guess at color by looking at the color of one pixel
and comparing it to the surrounding ones. But, importantly, it is basing
that guess on only 1/3 of the light falling on any given pixel because
of the colored filters in front of each one. All of this computational
effort is called color interpolation (as opposed to resolution interpolation)
and a report from a major color print manufacturer's research department
estimated that there are 100 computations for each pixel as the processor
zeros in on a pixel's color value. While 100 computations per pixel seems
insignificant with today's speedy chips, a 3Mp camera must make approximately
300,000,000 (yes, that's three hundred million!) computations to interpolate
the final colors in a scene from the raw data the imaging chip captures.

Even considering the Herculean
computational effort required, everything works pretty well until you
get to the border between one color and another, or a large expanse of
primarily (but not only) one color, or, lastly, a finely detailed, multicolored
pattern. In these situations the monkey with the wrench rears its ugly
little head and you end up with color fringing or a moir pattern--two
of digital imaging's Achilles' heels. Furthermore, in an effort to make
the guessing of each color easier, the image collecting chip in most of
today's digital cameras have a layer of diffusion material made from glass
over them to soften the edges between colors and make the guessing job
easier. Some cameras add this blurring effect in software but the net
result is better color, less artifacts (false color pixels), and a softening
of the image. In effect, this means that the 6Mp chip (as an example only)
you paid for is giving less sharpness than it can optimally provide. To
solve these problems what was needed was a way to capture all the light€a
way to capture all three colors at every pixel location.

Notice
the magnified image of a single pixel in this illustration.
The X3 absorbs different colors of light at different depths.

Capturing All Three Colors
In 1997, Foveon, a Santa Clara, California based firm, was founded and
its goal was to become a world-class player in the digital imaging field.
Carver Mead, chairman of Foveon, and a heavy hitter in the Silicon Valley,
lined up backers with deep pockets (National Semiconductor and Synaptics),
put together a staff and went to work. Foveon's first system provided
a means to capture all three colors (with little or no guessing) using
three 2Kx2K chips positioned behind a beam splitting prism which broke
incoming light into the three colors and sent each component to a separate
chip. Because each color had its own channel the result was an amazingly
sharp image that is free from color artifacts.

This effort also resulted in
a camera that, while producing extremely high quality digital images,
was ungainly to use in the real world. But, the quality made possible
by recording each color individually was solidly demonstrated.

Getting Back To The Future
Foveon realized that silicon (used as both the supporting surface and
slab of an imaging chip) absorbs each color of light (red, green, blue)
at a different level within depth of the silicon slab. It seems blue light
is absorbed at .2m (2/10 of one millionth of a meter), green light is
absorbed at .8m (8/10 of one millionth of a meter), and, finally, red
light is absorbed at 3.2m (3.2 millionths of a meter).

The
current crop of mosaic chips guess at two of the three colors
for each pixel location while the new X3 chip captures all
three colors at every pixel location.

The Quantum Leap
Someone at Foveon had the brilliant thought of putting the sensors, which
until now rested on the surface of the silicon slab, at different levels
within the silicon to take advantage of silicon's light absorption qualities.
Next, the mosaic set of red, green, and blue filters were removed from
the surface of the silicon and each pixel could both see and record all
three colors. Foveon has designed the first chip that collected data from
detectors located at different levels within the silicon slab as opposed
to just resting them on the silicon's surface. The ramifications of this
breakthrough in digital imaging terms are enormous.

First off, it will represent
a huge jump from the price vs. quality standpoint. CMOS chips, which the
X3 is based on, are roughly 50-60 percent the cost of a similar sized
CCD chip. But wait€there's more than meets the eye here. Because each
pixel now captures all colors (instead of one out of three) the performance
of an X3 chip is about two times that of a mosaic chip. So, in real world
terms, you can be safe in expecting 6Mp performance from a 3Mp X3 chip.

From another, practical point
of view, a 3Mp chip will approximately double your storage capacity when
compared to storing 6Mp captures. This brings up an interesting point
when looking at the space needed for capture vs. storage as a TIFF file
when discussing the X3 chip.

This
is the sensor you might find in the next generation of digital
cameras. It is an important enough breakthrough that the
JCIA had to come up with new nomenclature to describe it!
Meet the 3.5Mp X3 Foveon chip.

With mosaic chips a 6Mp file
requires 18MB of storage space because each pixel of information captured
must be stored as either a red, green, or blue color. If a pixel is blue
it requires a "yes" in the blue 6MB storage file and a "no"
in both the red and green storage files. The Foveon X3 chip captures each
color separately so, for example only, a 3.5Mp chip captures a 10.5MB
file because each color is captured individually. Furthermore, because
the guessing of color on the computer's part has been eliminated, the
storage space required for a TIFF file is the same sized 10.5MB file.

This aspect of the capture
is so unique that the JCIA (Japan Camera Industry Association) came up
with a new nomenclature to describe the Foveon X3. According to this new
naming system the 3.5Mp, three color chip is called a "3.5Mp X3"
chip with the "X3" denoting that all three colors are captured
independently and the file size is three times what we've come to classify
the chip's resolution by.

Just Scratching The Surface
The first X3 chip captures a 10.5MB raw file and is approximately the
size of an APS frame of film. Foveon plans on producing other sized chips
for various applications (amateur to pro level) and they will have a web
site set up for the camera designers. From this site a manufacturer can
download the specs and requirements for a specific chip so they can design
a camera around it. At this reporting, there is already one camera ready
for production (from Sigma) using the Foveon X3. Reportedly, a few other
companies are already fast at work incorporating the new chip technology
into their camera designs.

But that is then and as of
now, using the 3.5Mp APS sized chip, Foveon feels that they can match
current 6Mp image quality with this sized chip. But, there are a lot of
other interesting possibilities attached to their multiple level sensor
technology. One of these is a readout mode called "Variable Pixel
Size" in which Foveon can link four or more pixels together into
one giant (relatively speaking), super pixel. While this will reduce the
resolution of the chip (in the four pixel linked example a 3Mp chip becomes
a .75Mp chip) the big advantage will be in less light required for exposure.
A four pixel super pixel needs two f/stops less light to record an image
when compared to four individual pixels. In certain situations, such as
digital video, low-light capability can be as important as resolution
and the Foveon technology will let photographers choose between qualities
that they deem most important.

According to Foveon there is
no practical limit as to how large a chip can be produced and, if you
add that to the user selectable super pixel's low-light capability, there
is the possibility of medium format chips that will have the ability to
work in available darkness. Because of the already strong interest in
this new technology, and if it lives up to Foveon's claims, digital imaging
has taken a quantum step forward. It just might be that your new digital
camera is going to say "X3" inside.