Hybrid Mixed Reality

This project explores the combination of Physical and Virtual Reality through the usage of Mixed Reality. Early explorations involved the usage of Spatial Augmented Reality in combination with Virtual Reality, two technologies with complementary characteristics that evolved separately in the past. Spatial Augmented Reality (SAR) augments the environment using projectors or screens, without the need of user instrumentation. By keeping a single unified frame of reference, it supports social interaction and natural perception of the space, but the augmentation is limited by physical constraints (e.g., it requires a surface to display information). Immersive head mounted displays on the other hand are not limited by the physical properties of the environment, yet they isolate the user from their environment.
By providing a unified frame of reference for both SAR and immersive displays, the users can select the visualization that is best suited for a given task. This enables both asymmetric collaboration between users, and back-and-forths for a single user.
These explorations were followed by the combination of additional modalities, in an incremental fashion. This way, one or more users can chose the desired modalities, and immerse themselves as much as the task requires. As a result, the virtual world can be framed in relationship with the physical one.

The combination of physical and virtual spaces could benefit multiple applications, particularly in the context of asymmetric collaboration. Complex industrial environments, such as the aerospace industry, already use Virtual Reality to ease the conception and iteration of ideas. Yet, the decisions involved are usually taken by experts discussing in physical meeting rooms. We explored the possibility of bridge these two spaces in order to ease their awareness and communication capabilities, by building on our previous work.

Currently, our efforts are focused on the understanding of the human capabilities to interact with these heterogeneous representations, with some very promising results.