Adaptive Sensitivity is a visual information
processing feature that mimics the human eye's ability to compensate for
uneven illumination in high-contrast scenes, delivering an optimally contrasted
image without distortion or loss of quality.

How it works

Input to the cameras DSP are two simultaneous
mosaic format digital video signals. The two input signals are referred
to as "Long" and "Short".
The "Short" video input should contain data from the input image
as acquired by a CCD with a short exposure time. The bright regions should
register faithfully, while the dark regions may be close to or at cut-off,
and hence meaningless.
The "Long" video input should contain image data as acquired by
a CCD with a long exposure time. The bright regions may be at saturation,
but the darker regions should register faithfully.
The Adaptive Sensitivity algorithm for wide dynamic range video imaging
combines the meaningful information from both Short and Long
inputs, and generates a single output image in which the luminance range
is compressed to video standards (48 dB or 8 bits), without loss of image
detail or color fidelity.

More informtion

The sensor's output signal is separated to Long
and Short signals, pre- amplified, sampled by a correlated double sampler
and then digitized and fed into an iSP2000 DSP. The input image data from
each exposure is separated into the luminance ("luma") and chrominance
("chroma") information. The "Long" and "Short"
luma and chroma data are pre-processed and combined by a weighted summing
operation.
Internal look-up tables adjust the summing weights and control the picture
contrast and brightness.
Finally, the processed components are transformed into digital Y/Cr/Cb,
Y/I/Q, R/G/B, NTSC/YC and 601/4:2:2 formats (software selectable) or to
analog signals.
In the monochrome configuration, only the luminance channel and the Y
output are used.
In addition to its processing functions, the iSP2000 also provides centralized
digital control over camera functions.
It internally collects luminance, edge and color statistics for auto-exposure,
auto-focus and white-balance functions.
It is possible to assign different statistical weights to different regions
within the frame.
For example, an auto-exposure application which accumulates the average
luma of a given frame, might weigh the center of the picture at 1, the
bottom at ½ and the top at ¼.

Demo Images

The figures below provide an example of
Adaptive Sensitivity principles. The 'Long Exposure' and 'Short
Exposure' Figures show images of a scene requiring imaging with wide dynamic
range. Each image lacks the details which may be seen in its counterpart.
The central image, which is an image produced by the Adaptive Sensitivity
algorithm, clearly illustrates the benefits offered by both exposures.

Problem Statement

The (intraframe) dynamic range of a camera
is usually defined as the ratio of the brightest point of an image to the
darkest point of the same image. It is also called the maximum contrast
of that image. Unfortunately, the dynamic range of most electronic cameras
is severely limited. It is narrower than the dynamic range of most scenes,
and it is also more limited than that of photographic film.

Imaging applications often deal with situations
in which lighting conditions are far from optimal. In particular, these
may include objects positioned against strong back lighting, in which
case the objects details become too dark, since the camera adjusts itself
to the high average brightness. In some situations there will be many
spots with steep gradations of brightness, which are hard to handle by
standard cameras. Other situations depend on the dynamic behavior of the
camera: abrupt changes of illumination will cause profound transition
effects on the overall system.

All the above situations call for wide dynamic
range imaging, which is generally constrained by three factors: the sensor,
the signal processing circuits, and the display (or frame grabber).

Common CCD sensors can acquire a contrast of
roughly 1:1000 (60 dB) dynamic range of intensities. The darkest signal
is constrained by the thermal noise, or "dark current", of the sensor.
The brightest signal is limited by the total amount of charge that can
be accumulated in a single pixel. Usually CCD chips are built such that
this total maximum charge is about 1000 times the charge generated thermally.
This dynamic range can be substantially enhanced, in special applications
such as scientific or astronomical cameras for still imaging, by cooling
the sensor and by employing special readout circuits. However, such methods,
in addition to being very expensive, are inappropriate for real-time applications.

As described above, many applications usually
require an even wider dynamic range, such as 65-75 dB (1:1800 - 1:5600).
When imaging such a scene with a 60 dB imager, either details in the darker
areas get lost in the noise ("cut off"), or details in the brighter areas
are lost in saturation, or both. This is shown in the figure 1.

Figure 1: Response of a CCD

Charge readout circuits, analog amplifiers,
and A/D converters for real-time video typically limit the CCD signal
further to a dynamic range of 8 bits (48 dB). This range can be extended
to 10 bits, by employing appropriate analog processing and A/D converters;
however, this is impractical for most applications. Another alternative
type of circuitry employs non-linear transforms, such as logarithmic functions
or "knee" curves, to compress the 1:1000 CCD output signal down to an
8 bit signal. While depicting most of the image information, such methods
necessarily suppress details in the highlights.

The last limiting factor is the display (or
frame grabber). The dynamic range displayable by normal CRT monitors,
operating in a lighted room, is limited to about 1:100. An LCD screen
is even more limited. The 1:200 or so signal which is generated by the
video circuits is further reduced by the display. To optimize the display,
the user often needs to adjust the contrast and brightness control of
the monitor. A user who wants to display the image at its maximum contrast,
usually sacrifices some of the dynamic range.

Adaptive Sensitivity Solution

i Sight has developed proprietary non-linear
algorithms which reduce the dynamic range of the video signal without any
loss of details. i Sight has implemented the algorithms in a 650,000
transistors full custom VLSI chip. The chip takes a wide dynamic range video
input signal, and converts it to a 48 dB or less dynamic range signal, suitable
for display on any CRT monitor without any loss of details. All i Sight
cameras make use of this advanced video processor.

Ideally, the wide dynamic range input signal
would be represented by 12-16 bits per pixel. However, as explained above,
normal CCD chips cannot produce such wide dynamic range signals at video
rates. Hence, i Sight takes advantage of multiple exposures. The
same scene is imaged more than once (in most cases, two exposures are
sufficient). One exposure is made at a low level of sensitivity (e.g.
with an electronic shutter set at a short exposure time). That "short"
exposure contains highlight details, but most very dark image areas are
lost in the noise. A second exposure of the same image is taken at a relatively
high level of sensitivity (e.g. at a long exposure time). The "long" exposure
contains details of the darker parts of the image, but the brightest areas
may come out saturated, without any details. Subsequent to the acquisition,
the two images are combined in a special manner so as to produce a single,
very wide dynamic range image. See figure 2.

Figure 2: Wide Dynamic Range response of a CCD

That wide range image is further processed,
using i Sight's proprietary algorithm, to reduce the dynamic range
down to a useful level - normally 40 dB, which can be displayed on a CRT
monitor. A conceptual diagram of the i Sight camera is shown in the
figure 3.

Figure 3: A conceptual diagram of i Sight camera

United States Patent 5,247,366 - Color
wide dynamic range camera

Abstract:
The apparatus is a color wide dynamic range video camera which takes a
plurality of images at different exposure levels, applies neighborhood
processing to each of the images, and then combines the components into
a final image.

Adaptive Sensitivity and the advanced digital
video processor of i-sights cameras emulate the human eye's ability
to perceive detail in high contrast lighting environments, and provide
high detail video images even in the most extreme conditions. This unique
capability offers the best video imaging system available for such applications
as endoscopy, microscopy, machine vision, robotics, surveillance and many
other areas of industrial process control and research.