Bioinspired Camera Could Help Self-Driving Cars See Better

WASHINGTON--(BUSINESS WIRE)--Inspired by the visual system of the mantis shrimp—among the most
complex found in nature--researchers have created a new type of camera
that could greatly improve the ability of cars to spot hazards in
challenging imaging conditions.

The new camera accomplishes this feat by detecting a property of light
known as polarization and featuring a dynamic range about 10,000 times
higher than today’s commercial cameras. Dynamic range is a measure of
the brightest and darkest areas a camera can capture simultaneously.
With these, the camera can see better in driving conditions such as the
transition from a dark tunnel into bright sunlight or during hazy or
foggy conditions.

In Optica,
The Optical Society's journal for high impact research, the researchers
describe the new camera, which could be mass-produced for as little as
$10 apiece. The researchers say the new camera would enable cars to
detect hazards, other cars and people three times farther away than
color cameras used on cars today.

“In a recent crash involving a self-driving car, the car failed to
detect a semi-truck because its color and light intensity blended with
that of the sky in the background,” said research team leader Viktor
Gruev of the University
of Illinois at Urbana-Champaign, USA. “Our camera can solve this
problem because its high dynamic range makes it easier to detect objects
that are similar to the background and the polarization of a truck is
different than that of the sky.”

In addition to automotive applications, the researchers are exploring
using the cameras to detect cancerous cells, which exhibit a different
light polarization than normal tissue, and to improve ocean exploration.

“We are beginning to reach the limit of what traditional imaging sensors
can accomplish,” said Missael Garcia, first author of the paper. “Our
new bioinspired camera shows that nature has a lot of interesting
solutions that we can take advantage of for designing next-generation
sensors.”

Mimicking shrimp vision

Mantis shrimp, a grouping that includes hundreds of species worldwide,
have a logarithmic response to light intensity. This makes the shrimp
sensitive to a high range of light intensities, allowing them to
perceive very dark and very bright elements within a single scene.

To achieve a similarly high dynamic range for their new camera, the
researchers tweaked the way the camera’s photodiodes convert light into
an electrical current. Instead of operating the photodiodes in reverse
bias mode — which is traditionally used for imaging — the researchers
used forward bias mode. This changed the electrical current output from
being linearly proportional to the light input to having a logarithmic
response like the shrimp.

For the polarization sensitivity, the researchers mimicked the way that
the mantis shrimp integrates polarized light detection into its
photoreceptors by depositing nanomaterials directly onto the surface of
the imaging chip that contained the forward biased photodiodes. “These
nanomaterials essentially act as polarization filters at the pixel level
to detect polarization in the same way that the mantis shrimp sees
polarization,” said Gruev.

Although traditional imaging sensor fabrication processes can be used to
make the sensors, they are not optimized for making photodiodes that
operate in a forward bias. To compensate, the researchers developed
additional processing steps to clean up the images and to improve the
signal to noise ratio.

Taking the camera on the road

After testing the camera under different light intensities, colors and
polarization conditions in the lab, the researchers took the camera into
the field to see how well it operated in shadows as well as in bright
conditions. “We used the camera under different driving lighting
conditions such as tunnels or foggy conditions,” said Tyler Davis, a
member of the research team. “The camera handled these challenging
imaging conditions without any problems.”

The researchers are now working with a company that manufactures air
bags to see if the new camera’s high dynamic range and polarization
imaging capability can be used to better detect objects to either avert
a collision or deploy the air bag a few milliseconds earlier than is
currently possible.

Exploring the ocean

The researchers also received funding to use the new imaging system to
make small GoPro-like cameras that could be used to explore the ocean.
While GPS systems such as those in cell phones do not work under water,
the new camera’s polarization detection capability allows it to use the
polarization of sunlight in water to calculate location coordinates. In
addition, the camera’s high dynamic range could be used to record high
quality images under water.

“We are coming full circle by taking the camera, which was inspired by
mantis shrimp, to different tropical oceans to learn more about how this
shrimp behaves in its natural habitat,” said Gruev. “They live in
shallow waters and bury themselves under corals or in little burrow.
This creates a challenging high dynamic range imaging situation because
there's a lot of light in the water but dim conditions inside the holes.”

Optica is an open-access, online-only journal dedicated to the
rapid dissemination of high-impact peer-reviewed research across the
entire spectrum of optics and photonics. Published monthly by The
Optical Society (OSA), Optica provides a forum for pioneering
research to be swiftly accessed by the international community, whether
that research is theoretical or experimental, fundamental or applied. Optica
maintains a distinguished editorial board of more than 50 associate
editors from around the world and is overseen by Editor-in-Chief Alex
Gaeta, Columbia University, USA. For more information, visit Optica.

About The Optical Society

Founded in 1916, The Optical Society (OSA) is the leading professional
organization for scientists, engineers, students and business leaders
who fuel discoveries, shape real-life applications and accelerate
achievements in the science of light. Through world-renowned
publications, meetings and membership initiatives, OSA provides quality
research, inspired interactions and dedicated resources for its
extensive global network of optics and photonics experts. For more
information, visit osa.org.