The ability to guide human motion through automatically generated feedback has significant potential for applications in areas, such as motor learning, human-computer interaction, telepresence, and augmented reality. This paper focuses on the design and development of such systems from a human cognition and perception perspective. We analyze the dimensions of the design space for motion guidance systems, spanned by technologies and human information processing, and identify opportunities for new feedback techniques. We present a novel motion guidance system, that was implemented based on these insights to enable feedback for position, direction and continuous velocities. It uses motion capture to track a user in space and guides using visual, vibrotactile and pneumatic actuation. Our system also introduces motion retargeting through time warping, motion dynamics and prediction, to allow more flexibility and adaptability to user performance.

Malleable and organic user interfaces have the potential to enable radically new forms of interactions and expressiveness through flexible, free-form and computationally controlled shapes and displays. This work, specifically focuses on particle jamming as a simple, effective method for flexible, shape-changing user interfaces where programmatic control of material stiffness enables haptic feedback, deformation, tunable affordances and control gain. We introduce a compact, low-power pneumatic jamming system suitable for mobile devices, and a new hydraulic-based technique with fast, silent actuation and optical shape sensing. We enable jamming structures to sense input and function as interaction devices through two contributed methods for high-resolution shape sensing using: 1) index-matched particles and fluids, and 2) capacitive and electric field sensing. We explore the design space of malleable and organic user interfaces enabled by jamming through four motivational prototypes that highlight jamming's potential in HCI, including applications for tabletops, tablets and for portable shape-changing mobile devices.

Mobile devices are currently difficult to customize for the usability needs of elderly users. The elderly are instead referred to specially designed "senior phones" or software add-ons. These tend to compromise in functionality as they attempt to solve many disabilities in a single solution. We present OldGen, a prototype framework where a novel concept enables accessibility features on generic mobile devices, by decoupling the software user interface from the phone's physical form factor. This opens up for better customization of the user interface, its functionality and behavior, and makes it possible to adapt it to the specific needs of each individual. OldGen makes the user interface portable, such that it could be moved between different phone hardware, regardless of model and brand. Preliminary observations and evaluations with elderly users indicate that this concept could address individual user interface related accessibility issues on general-purpose devices.

We present a number of rendering and interaction techniques that exploit the user's viewpoint for improved realism and immersion in 3D applications on handheld devices. Unlike 3D graphics on stationary screens, graphics on handheld devices are seldom regarded from a fixed perspective. This is particularly true for recent mobile platforms, where it is increasingly popular to use device orientation for interaction. We describe a set of techniques for improved perception of rendered 3D content. Viewpoint correct anamorphosis and stereoscopy are discussed along with ways to approximate the spatial relationship between the user and the device.

Mobile devices are now able to connect to a variety of sensors and provide personalized information to help people reflect on and improve their health. For example, pedometers, heart-rate sensors, glucometers, and other sensors can all provide real-time data to a variety of devices. Collecting and interacting with personal health or well-being data is a growing research area. This workshop will focus on the ways in which our mobile devices can aggregate and visualize these types of data and how these data streams can be presented to encourage interaction, increased awareness and positive behavior change.

Motion sensing is of fundamental importance for user interfaces and input devices. In applications, where optical sensing is preferred, traditional camera-based approaches can be prohibitive due to limited resolution, low frame rates and the required computational power for image processing. We introduce a novel set of motion-sensing configurations based on laser speckle sensing that are particularly suitable for human-computer interaction. The underlying principles allow these configurations to be fast, precise, extremely compact and low cost. We provide an overview and design guidelines for laser speckle sensing for user interaction and introduce four general speckle projector/sensor configurations. We describe a set of prototypes and applications that demonstrate the versatility of our laser speckle sensing techniques.

We introduce two families of techniques, rubbing and tapping, that use zooming to make possible precise interaction on passive touch screens, and describe examples of each. Rub-Pointing uses a diagonal rubbing gesture to integrate pointing and zooming in a single-handed technique. In contrast, Zoom-Tapping is a two-handed technique in which the dominant hand points, while the non-dominant hand taps to zoom, simulating multi-touch functionality on a single-touch display. Rub-Tapping is a hybrid technique that integrates rubbing with the dominant hand to point and zoom, and tapping with the non-dominant hand to confirm selection. We describe the results of a formal user study comparing these techniques with each other and with the well-known Take-Off and Zoom-Pointing selection techniques. Rub-Pointing and Zoom-Tapping had significantly fewer errors than Take-Off for small targets, and were significantly faster than Take-Off and Zoom-Pointing. We show how the techniques can be used for fluid interaction in an image viewer and in Google Maps.

Interactive surfaces and related tangible user interfaces often involve everyday objects that are identified, tracked, and augmented with digital information. Traditional approaches for recognizing these objects typically rely on complex pattern recognition techniques, or the addition of active electronics or fiducials that alter the visual qualities of those objects, making them less practical for real-world use. Radio Frequency Identification (RFID) technology provides an unobtrusive method of sensing the presence of and identifying tagged nearby objects but has no inherent means of determining the position of tagged objects. Computer vision, on the other hand, is an established approach to track objects with a camera. While shapes and movement on an interactive surface can be determined from classic image processing techniques, object recognition tends to be complex, computationally expensive and sensitive to environmental conditions. We present a set of techniques in which movement and shape information from the computer vision system is fused with RFID events that identify what objects are in the image. By synchronizing these two complementary sensing modalities, we can associate changes in the image with events in the RFID data, in order to recover position, shape and identification of the objects on the surface, while avoiding complex computer vision processes and exotic RFID solutions.

Augmented Reality (AR) systems that employ user-worn display and sensor technology can be problematic for certain applications as the technology might, for instance, be encumbering to the user or limit the deployment options of the system. Spatial AR systems instead use stationary displays that provide augmentation to an on-looking user. They could avoid issues with damage, breakage and wear, while enabling ubiquitous installations in unmanned environments, through protected display and sensing technology. Our contribution is an exploration of compatible interfaces for public AR environments. We investigate interactive technologies, such as touch, gesture and head tracking, which are specifically appropriate for spatial optical see-through displays. A prototype system for a digital museum display was implemented and evaluated. We present the feedback from domain experts, and the results from a qualitative user study of seven interfaces for public spatial optical see-through displays.

We describe an approach to 3D multimodal interaction in immersive augmented and virtual reality environments that accounts for the uncertain nature of the information sources. The resulting multimodal system fuses symbolic and statistical information from a set of 3D gesture, spoken language, and referential agents. The referential agents employ visible or invisible volumes that can be attached to 3D trackers in the environment, and which use a time-stamped history of the objects that intersect them to derive statistics for ranking potential referents. We discuss the means by which the system supports mutual disambiguation of these modalities and information sources, and show through a user study how mutual disambiguation accounts for over 45% of the successful 3D multimodal interpretations. An accompanying video demonstrates the system in action.