Multimodal Maps: An Agent-based Approach

Abstract

The SRI Multimedia Interfaces Group is currently building a series of
prototype map-based applications that accept handwritten, verbal and gestural
requests issued in a synergistic fashion.
These applications are distinguished by comparatively rich natural language
capabilities, access to existing data sources including the World Wide
Web, and a mobile handheld interface. To implement the described application,
a hierarchical distributed network of heterogeneous software agents was
augmented by appropriate functionality for developing synergistic multimodal
applications.

Features

Multimodal Input: A user interacts with the system through
handwritten, gestural, vocal, and direct manipulation commands.

Multimedial Output: Responses are provided by the system through
textual, graphical, audio and video media.

Agent Interaction: User requests are processed by a distributed
community of software agents, which automate tasks such as natural
language processing, interaction with planners or simulators, and
accessing data from remote datasources (Prolog DBs, X.500 DBs, WWW, etc.)

Demos

To see a demonstration (including video) of the multimodal map interface,
click here.

To see a picture representing the agents used in the multimodal map
demonstration, click here.