Using dynamic time warping for online temporal fusion in multisensor systems

Access Status

Authors

Date

Type

Metadata

Abstract

Sensor fusion is concerned with gaining information from multiple sensors by fusing across raw data, features or decisions. Traditionally these fusion processes only concern fusion at specific points in time. However recently, there is a growing interest in inferring the behavioural aspects of environments or objects that are monitored by multisensor systems, rather than just their states at specific points in time. In order to infer environmental behaviours, it may be necessary to fuse data acquired from (i) geographically distributed sensors at specific points of time and (ii) specific sensors over a period of time. Fusing multisensor data over a period of time (also known as Temporal fusion) is a challenging task, since the data to be fused consists of complex sequences that are multi-dimensional, multimodal, interacting, and time-varying in nature. Additionally, performing temporal fusion efficiently in real-time is another challenge due to the large amounts of data to be fused. To address this issue, we propose a robust and efficient framework that uses dynamic time warping (DTW) as the core recognizer to perform online temporal fusion on either the raw data or the features. We evaluate the performance of the online temporal fusion system on two real world datasets: (1) accelerometer data acquired from performing two hand gestures, and (2) a benchmark dataset acquired from carrying a mobile device and performing the predefined user scenarios. Performance results of the DTW-based system are compared with those of a Hidden Markov Model (HMM) based system. The experimental results from both datasets demonstrate that the proposed system outperforms HMM based systems, and has the capability to perform online temporal fusion efficiently and accurately in real-time.

Collection

Related items

Fusion is a fundamental human process that occurs in some form at all levels of sense organs such as visual and sound information received from eyes and ears respectively, to the highest levels of decision making such as ...

Remote sensing provides rich sources of data for the monitoring of land surface dynamics. However, single-sensor systems are constrained from providing spatially high-resolution images with high revisit frequency due to ...

Data fusion is an important issue for object tracking in autonomous systems such as robotics and surveillance. In this paper, we present a multiple-object tracking system whose design is based on multiple Kalman filters ...