Semantic coding of cockpit screen content

Recognition and coding of graphic elements in airplane screen videos

Aircraft screens contain graphical information such as altitude, speed, or lines or circles. If on the one hand this information is very important for the pilot, on the other hand it is “difficult” to code because it does not have the same characteristics as “natural” images, which leads to strong coding artifacts that may affect the readability of this information.

In this research activity we aim to extract graphical information such as text, lines and circles from an airplane screen video and code them separately from the “image” content. In the example of the figure above, we propose to code the text (13JAN2012 etc) and the image of the sea separately.

To this end, several problems must be solved:

Text detection and recognition of each character: a convolutional network is used for these tasks

Coding of the text as such and coding of the residual image

Taking into account the evolution of time

This project is at the heart of the thesis of our doctoral student Iulia Mitrica