Visual Search Target Inference for Pro-active User Support

Visual search is a perceptual task in which humans aim at identifying a search target object such as a traffic sign among other objects. Search target inference subsumes computational methods for predicting this target by tracking and analyzing overt behavioral cues of that person, e.g., the human gaze and fixated visual stimuli. In [Barz et al., 2020] we present a generic approach to inferring search targets in natural scenes by predicting the class of the surrounding image segment. Our method encodes visual search sequences as histograms of fixated segment classes determined by SegNet, a deep learning image segmentation model for natural scenes. We compare our sequence encoding and model training (SVM) to a recent baseline from the literature for predicting the target segment. Also, we use a new search target inference dataset.

Encoding Methods for Fixations of a Visual Search

Visual Search Target Inference Using Bag of Deep Visual Words

In our work [Stauden et al., 2018], we implement a new feature encoding, the Bag of Deep Visual Words, for search target inference using a pre-trained convolutional neural network (CNN). Our work is based on a recent approach from the literature that uses Bag of Visual Words, common in computer vision applications. We evaluate our method using a gold standard dataset. The results show that our new feature encoding outperforms the baseline from the literature, in particular, when excluding fixations on the target. We presented this work at the 41st German conference on Artificial Intelligence.

Search

Categories

Related Posts

In this present day and age of smart phones, ipads and other smart devices a digital camera has become an integral part of our daily experience. This combined with our desire to capture moments in Read more…

In the medical domain physicians are often forced to decide quickly whether a further examination is appropriate or which of several treatments is best suited for a patient.A first preliminary ruling can sometimes be made Read more…

In comparison to typing, handwriting stimulates different parts of the human brain and brings into play a very different cognitive process. Many of today’s tasks benefit from digitalization and digitization of handwriting input and with Read more…