Imagination Reinvents Imaging Signal Processing

SAN FRANCISCO — Jumping on the IoT bandwagon, Imagination announced today its PowerVR Raptor Image Signal Processing (ISP) architecture for vision applications in both higher and lower performance needs, in a variety of markets from cameras and phones to automotive vision applications. The company says Raptor ISP architecture is optimized for SoC integration and could create a system-wide platform for vision because of its built-in flexible resource management.

Raptor provides "the basis for context-aware applications such as facial and gesture recognition, augmented reality, and more," says the press release.

“This is a new departure for us,” said Peter McGuinness, Imagination’s director of multimedia technology marketing, in a call with EE Times. The company combined high-performance camera SoC and ISP from its Nethra acquisition, with its existing IP creation and imaging expertise, to create Raptor. Imagination is known for its PowerVR SoCs and graphics IP, which use tile-based, deferred rendering, and shading architecture.

The result was Raptor, a configurable imaging pipeline that can crunch down on power-use but deliver higher-end HDTV quality and high-pixel-count photography. Using system partitioning, Raptor will send some tasks and image statistics to the CPU and others to the GPU, depending on the processing needed. The flexibility with pixel depth makes the architecture scalable and makes the power savings happen. Supporting up to 16-bit pixel depth, which is good for automotive and industrial applications, Raptor’s pipeline can also be scaled back to match pixel depth of a sensor. Reducing pixel depth saves power. “You only need 8-bit pixel depth for a camera pipeline,” said McGuinness.

The ISP also provides multi-sensor support and multi-camera connect, so, for instance, front and back of camera on phones can be processed through a single image processor rather than two.

Imagination created this architecture to ease imaging systems design and processing, not necessarily using Imaginations' PowerVR SoCs, either. “We do have all the blocks needed under one roof,” said McGuinness, "If customers use the full system they have more options than if using different vendors.” However, it’s not necessary, as the ISP can be applied to other vendors' SoCs.

Raptor ISP uses system partitioning to send data to different resources; different pipelines are also visible in this chart. (Source: Imagination)

I believe Ambric was an acquisition by Nethera as well and provided the hardware architecture used for the stream processing algorithms Nethera developed. I believe it allows video data to flow thru the system with portions of the algorithms executed at each step, simplifying and speeding overall processing.

Would be good to have someone else dig into the details however, my data is a few years old now.

It seems like image processing is going to be more and more on our minds moving forward. I keep seeing these amazing concepts that are only barely able to function with current tech. Can't wait to see what near future improvements bring to this area.

What I like most about announcements like this is that it shows how a new generation of bio-inspired vision insights are slowly working their way into commercial products.

It's easy to forget how astonishingly fast and energy efficient real-time object recognition and tracking is in biological sysems. But once innovators recognize what's possible and start to take the implications to heart, they enter a new competitive domain of building and selling vision-capable products that in time can achieve amazingly good price points. The old mathematically-inspired approach of treating "vision" strictly as a problem in how to process enormous 3D (2D x time) floating point matrices works after a fashion, but only by burning up an amazing number of CPU cycles and more than a little energy. Conversely, if you are simply reading this text right now, you are yourself an existence proof that more efficient approaches to parsing complex visual information are possible.