Previously we discussed some of the complexities and considerations associated with Software Engineering, particularly with medical devices. On area where this comes into play is integration of vision technologies.

Recently, a customer approached us to help upgrade to their endoscope system. Their first-generation product was successful, but lacked encryption and leveraged an SD card slot to store images. With increasing importance of security around HIPAA and DICOM standards, they wanted a 2nd generation device that included encrypted data storage of the images instead of the SD card, used existing endoscopes, and ran on a medically approved PC. In addition, the 2nd generation device needed to be an “all-in one” instrument.

The challenge was to construct a communication system that captured the images from the endoscope and transferred them to the PC in real-time while saving the encrypted video. The commitment of the project is the provide high picture quality. Fine tuning and experimental methods are required to achieve perfect results.

A camera interface module (CIM) was constructed to act as the interface point between the Endoscope and the PC. This piece of hardware consists of multiple CPU’s and an interface to the PC. In developing the CIM, latency of the images and real time streaming as they came from the endoscope had to be handled. To address this, the work was divided between the 2 CPU’s on the CIM. One of the CPUs was tasked to handle the communication between PC and camera interface board. The other one is employed to control low-level peripheral communication, perform some low-level calculations and set up the video IPs on the CIM related with video processing.

Within the CIM, intercommunications between the CPUs was needed. In particular, some low-level signaling and establishment of video processing IPs on the CIM according to the data coming in from PC are required to have a CPU to CPU message pipe. The message pipe was created on the built-in chip memory and is designed to run on non-cached memory to avoid data corruption along the CPUs due to the two-stage cache controller.

Because the CIM needed to communicate with the PC, and due to speed and bandwidth requirements, the video connection selected was UDP over a standard TCP/IP protocol. A real-time operating system called FreeRTOS was chosen to perform socket communications and other signal management functions.

With the CIM and connectivity resolved, the next step was to address processing speed on the PC running Windows 10. The endoscope is transmitting to the Windows PC images at a rate of 30 frames per second (fps). The Windows PC must then process the images fast enough to not fall behind the steady stream of images coming from the CIM. If the PC cannot process fast enough, the images shown to the physician during the procedure would be delayed or frames would be dropped.

There are two possible solution paths to address the 30fps requirement on the Windows PC. The first is to use a multi-threaded architecture and to allow Windows to deal with the resource and priorities associated with thread management. This requires a system powerful enough to handle all of the threads associated with communication, data transfer and image processing. The second approach is to use a single threaded architecture to handle the entire image pipe. Ideally, this single-threaded approach is masked to a single processor.

Resolution found that for this use-case the single-threaded approach allows for less expensive hardware and a smaller resource allocation to manage the burden associated with the image pipeline handling at the desired rate within the memory constraints. This approach reduces the cost of goods for the end product without compromising the quality of the product delivered.

While either solution works, the constraints of the project dictated a specific path. This holistic approach of analysis, attention to detail, and understanding of cost thresholds is often overlooked by outsourced product development companies.