Any sufficiently advanced technology is in distinguishable from magic.

Sony’s new DRAM-enhanced image sensor is a game changer for smartphone photography

As if the makers of standalone cameras don’t have enough problems holding off smartphones, Sony has just announced a sensor that has the potential to be yet another game changer for what’s possible in mobile device photography. Its new design adds a layer of DRAM between the pixel layer and the circuit layer. That allows it to capture and readout images several times faster than current designs.

Sony announced the new chip architecture at ISSCC this week, as the latest in its long line of innovations in sensor design. The addition of on-board DRAM will help solve at least three major issues with current smartphone cameras: rolling shutter artifacts, slow-motion videos, and multi-image noise reduction artifacts. We’ll look at each in turn.

A great band-aid for rolling shutter headaches

One problem with low-cost imagers, like those used in smartphones, is that they don’t have a mechanical shutter. Their images are read out one piece at a time while the sensor is still actively recording data (called a rolling shutter). That means that objects moving at high speed appear distorted, like the locomotive in this illustrative example from Sony:

While the new sensor still has a rolling shutter, its high speed readout greatly reduces artifacts

On the left, the typical 1/30s readout time smears the profile of the moving locomotive. On the right, while still technically a rolling shutter, the new chip’s 1/120s readout time greatly reduces the potential for distortion.

The new architecture adds a DRAM layer between the pixels and the circuitry

While the additional DRAM layer looks pretty simple in this block diagram, it changes the physics of the chip, and introduces additional opportunity for electrical noise. So far, the chip isn’t in production, so it isn’t possible to benchmark its real-world performance. But Sony claims it has been able to solve the new noise problems.

Support for super-slow-motion video and special effects

The DRAM also allows the sensor to capture 1000fps video at 1080p HD resolution. This makes for some very impressive super-slow-motion potential. By mixing high-frame-rate video with standard frame rate the chip also makes it possible to do some cool special effects right in a smartphone, as you can see in this Sony example video:

Expect record-setting image quality results from this sensor

Less obvious, but perhaps most importantly, the 30fps readout of full-resolution 19.3MP images is perfect for the sophisticated multi-image noise reduction that’s increasingly-common in high-end smartphones. For example, Google’s HDR+ silently combines between 3 and 9 frames to create a single higher-dynamic-range, lower-noise, final output image. That process can introduce artifacts if objects in the scene, or the camera, are moving. By having the frames captured and read out more quickly, those artifacts can be further reduced.