Mission Driven Scene Understanding: Dynamic Environments

Report No.

ARL-TN-0764

Authors:

Arnold Tunick

Date/Pages:

June 2016; 26 pages

Abstract:

Knowledge of how time and space changing environmental conditions cause changes in the context of images is necessary for scene understanding. Such dynamic environmental conditions (e.g., changing illumination, precipitation, and vegetation) can modify saliency and context, obscure features, and degrade object recognition. Here, context means more than the typically referenced attributes, content, or composition of an outdoor scene. For Army applications, scene understanding needs to be viewed in the context of providing optimal value to the Army mission. Then, for example, helpful image cues that relate to mission activities may include time of day, current and future weather conditions, visibility, terrain, and scene location. In this report, we outline progress toward implementing our mission driven scene understanding approach to advance the value of Army autonomous intelligent systems. We describe the proof-of-principle installation, setup, and testing of a convolutional neural network (CNN) program developed in Python and all its required software dependencies. While we found that the CNN was able to determine the correct class labels for images taken from the training data set, the validation process did not appear to provide optimal results for images not previously seen. Thus, we recommend performing additional trials and analysis to better determine the feasibility of using the CNN to augment our approach.