Abstract

Detection of salient object sequences from video data is challenging
when the salient object changes between consecutive frames. In this
study, we addressed the salient object sequence rebuilding problem
with video segment analysis. We reformulated the problem as a binary
labeling problem, analyzed the potential salient object sequences
in the video using a clustering method, and separated the salient
object sequence from the background by applying an energy optimization
method. Our proposed approach determines whether temporal consecutive
pixels belong to the same salient object sequence. The conditional
random field is then learned to effectively integrate the salient
features and the sequence consecutive constraints. A dynamic programming
algorithm was developed to resolve the energy minimization problem
efficiently. Experimental results confirmed the ability of our approach
to address the salient object rebuilding problem in automatic visual
attention applications and video content analysis.

(Color online) An example in which multiple salient objects appear, while the sequence index is defined to distinguish between the different salient object sequences. Previous approaches [3,4]assume a single salient object sequence, and output one rectangle for all the salient objects.

(Color online) Examples used to compare the effectiveness of our approach with SSA. (a) Car sequence with cottages; (b) two different people appearing successively; (c) a person walks past a car; (d) a person walks in front of a sculpture.

(Color online) Salient object tracking with UAV vision system. The salient object is rebuilt well in frames ${\#}318,~{\#}338$. (a) From left to right: frame ${\#}216,~{\#}236,~{\#}256,~{\#}276$; (b) from left to right: frame ${\#}296,~{\#}308,~{\#}318,~{\#}338$.