What's Missing?

A growing list of devices built with natural user interfaces (NUIs) accept input in the form of taps, swipes, and other ways of touching; hand and arm motions; body movement; and increasingly, natural language. Tablets and smartphones are the first in a growing array of devices that allow computers to recognize and interpret natural physical gestures as a means of control. These natural user interfaces allow users to engage in virtual activities with movements similar to what they would use in the real world, manipulating content intuitively. The idea of being able to have a completely natural interaction with your device is not new, but neither has its full potential been realized. What makes natural user interfaces especially interesting this year is the increasing high fidelity of systems that understand gestures, facial expressions, and their nuances, as well as the convergence of gesture-sensing technology with voice recognition. Users interact with their devices in an almost natural fashion, with gesture, expression, and voice communicating their intentions. The next wave of NUIs will likely be electrovibration, while involves the use of an electrostatic force to produce detailed tactile sensations that users can feel. Seen as the next evolution of touchscreen technology, it will allow users to not only provide touch-based input, but also tactile output via a wide variety of textures, topography, and other features as they interact with the screen.

INSTRUCTIONS: Enter your responses to the questions below. This is most easily done by moving your cursor to the end of the last item and pressing RETURN to create a new bullet point. Please include URLs whenever you can (full URLs will automatically be turned into hyperlinks; please type them out rather than using the linking tools in the toolbar).

Please "sign" your contributions by marking with the code of 4 tildes (~) in a row so that we can follow up with you if we need additional information or leads to examples- this produces a signature when the page is updated, like this: - LarryFeb 8, 2012

(1) How might this technology be relevant to the educational sector you know best?

The thing that interests me about natural user interfaces is the changing (changed?) nature of the interaction between user and device. And that the expectation from users that they will be able to swipe, pinch, or voice command screens and have them react. Remember the video that did the rounds in 2011 of the toddler who couldn't make a magazine 'respond' to her gestures? What are the challenges for us in creating in gallery interactives that are just as responsive to what we want to do as our devices at home? - ewallisFeb 28, 2016

One of the barriers to introducing new technologies in the museum space is the learning curve for users. To provide a frictionless experience, visitors shouldn't have to learn how to interface with the exhibit, the interface should be universal and intuitive. Sophisticated NUIs could be the bridge between users and all sorts of digital tech. If truly "natural," good interfaces would reduce the learning curve, and if universal, any learning that had to take place would be transferable to the whole universe of interactions. - elizabeth.merrittMar 1, 2016

(2) What themes are missing from the above description that you think are important?

I recommend we start saying "haptics" rather than "electrovibration" (which sounds like a marital aid). - elizabeth.merrittMar 1, 2016

add your response here

(3) What do you see as the potential impact of this technology on education and interpretation in museums?

It could help create frictionless experiences that integrate many kinds of digital interaction, by providing one, universal set of conventions on how to interact with various technologies.- elizabeth.merrittMar 1, 2016