Understanding palm-based imaginary interfaces

Imaginary Interfaces are screen-less ultra-mobile interfaces. Previously we showed that even though they offer no visual feedback they allow users to interact spatially, e.g., by pointing at a location on their non-dominant hand.

Figure 1: We adapted a non-visual audio interface that announced targets as users touch them, which allow users browse an unfamiliar imaginary interface.

The primary goal of this paper is to provide a deeper understanding of palm-based imaginary interfaces, i.e., whythey work. We perform our exploration using an interaction style inspired by interfaces for visually impaired users. We implemented a system that audibly announces target names as users scrub across their palm. Based on this interface, we conducted three studies. We found that (1) even though imaginary interfaces cannot display visual contents, users’ visual sense remains the main mechanism that allows users to control the interface, as they watch their hands interact. (2) When we remove the visual sense by blindfolding, the tactile cues of both hands feeling each other in part replace the lacking visual cues, keeping imaginary interfaces usable. (3) While we initially expected the cues sensed by the pointing finger to be most important, we found instead that it is the tactile cues sensed by the palm that allow users to orient themselves most effectively.

Figure 2: We performed three controlled studies with these interface conditions.

While these findings are primarily intended to deepen our understanding of Imaginary Interfaces, they also show that eyes-free interfaces located on skin outperform interfaces on physical devices. In particular, this suggests that palm-based imaginary interfaces may have benefits for visually impaired users, potentially outperforming the touchscreen-based devices they use today.

Youtube Video

Publication

Gustafson, S., Rabe, B. and Baudisch, P. Understanding Palm-Based Imaginary Interfaces: The Role of Visual and Tactile Cues when Browsing. In Proceedings of CHI 2013.