Why Today's Industrial Cameras Require Modern and Complex Electronics

Industrial cameras are getting more complex than ever due to the advances in electronics performance, increased sensor speed and an increase in resolution requirements for higher performance or emerging applications. The time when a camera was simply an image sensor with its output bits packed on a CameraLink bus is now long behind. Today's cameras exhibit high speed computer interfaces and complex on-board real time processing that require state-of-the-art electronic design for board level, FPGA, processor and memory as well as thermal and mechanical know-how.

Computer interface

With today's standards, the computer interfaces are based on Ethernet (1Gbps and 10Gbps), USB3, USB3.1, PCIe or other high speed and complex protocols and cabling. Such interfaces require careful board routing to maximize the integrity and timing matching of signals in the GHz range without compromising with ESD and EMC requirements.

Because of their more complex protocols, the camera's computer interface now requires some algorithms inside the camera to manage the bus protocol, its various commands and the packet based communication. It also has to manage the bandwidth and the burst based communication as well as latency and for many transport mediums, a not guaranteed packed delivery. These implementations require high speed memory, at least DDR2, and microcontrollers or DSPs either dedicated, like FX3 (Cypress) or i.MX (NXP), or embedded like Zynq (Xilinx) or Microblaze (Xilinx) and many others.

Several designs now use on-board GPU to assist with complex image processing and modern processors like i.MX have embedded GPU cores for 2D and 3D operations as well as vector based floating point coprocessors.

We also start to see some cameras with multiple processors or FPGAs and DDR4 class memories as well as very fast flash devices holding configuration sequences and calibration data.

Today's embedded designs also often make use of an operating system, typically a dedicated Linux distribution and therefore camera electronics engineers also need to master driver development as well as task management and file systems.

Embedded processing

There is more demand for embedded processing. We used to only do basic control of the sensor and basic corrections of the image, but now large kernel processing, multi-frame processing and advanced applications are requested. The processing also requires higher speed because of the increases in resolution and frame rate.

Image enhancement

A typical camera will "denoise" the image using reference images, linearity models and complex defect detection filters. The most common algorithm is the dark frame subtraction but the computation of that dark frame itself is already a complex piece of code and memory management.

There are advanced algorithms for image enhancement (not for viewing but to maximize the performance of image processing) that require the most powerful processors or very well thought FPGA architectures.

Image processing

There is now more demand for image processing inside the camera, either because the camera itself takes decisions or because the bandwidth to the computer is not large enough to transfer the raw unprocessed data.

As an example, the camera on the header's picture of this article performs 3D profiling out of 2D images at the maximum possible frame rate of the sensor and only transmits 3D profiles to the host application, which has a much lower data rate. Another hyperspectral camera perform classification on-board and only transmits the list and location of detected objects. Again another camera detects vehicles on a road and tracks them to provide traffic statistics and event warnings.

Thermal management

High speed image sensors, high speed processors or FPGAs, high speed drivers for computer interfaces as well as memories and other components tend to consume more power which turns into more heat.

As we all know, the performance and the lifetime of electronics, and especially image sensors, depends on temperature. Engineers now have to manage the heat produced by the devices, the uniformity of the heat (hot spots), the draining of the heat outside of the critical areas and, if needed, an active cooling of the devices with thermoelectric devices or fluids.

Algorithms and sensor control timings are not only developed for optimum performance but also for minimum heat.

Mechanical

The trend is to make small cameras. With smaller cameras, the robustness of the design and the assembly is a concern. Heat, vibration and shock resistance are mechanical topics that are addressed during design and validation. The quality of the assembly process of modern electronic components also has to be well thought and monitored.

Specialized teams

Camera designs now require specialized engineers in which each engineer focuses on one part of the camera design. Typically, a design team is made of one or more schematics and layout engineers, one or several FPGA or processor code engineers, one or several image processing engineers, usually at least one computer software engineer, a mechanical and thermal engineer and an image sensor specialist. These engineers also must master advanced design and simulation software.

Characterization and validation also require specialized engineers and equipment. In some projects, optical or physics engineers are also involved.

Aphesa develops custom cameras and custom electronics including FPGA code and embedded software. We also provide EMVA1288 test equipment and test services as well as consulting and training in machine vision and imaging technologies. Aphesa works in several markets including industrial, medical, oil&gas and security.