Feb 27, 2009

In Part I of my "mini-series" about Tangible User Interfaces, I discussed the origins of TUI and provided some examples of Siftables. In this section, I've provided some links to information about Tangible User Interfaces and the abstracts of two articles pertaining to TUI's in educational settings.Zen Waves: A Digital (musical) Zen Garden

ReactableMore about the Reactable"The reactable hardware is based on a translucent, round multi-touch surface. A camera situated beneath the table, continuously analyzes the surface, tracking the player's finger tips and the nature, position and orientation of physical objects that are distributed on its surface. These objects represent the components of a classic modular synthesizer, the players interact by moving these objects, changing their distance, orientation and the relation to each other. These actions directly control the topological structure and parameters of the sound synthesizer. A projector, also from underneath the table, draws dynamic animations on its surface, providing a visual feedback of the state, the activity and the main characteristics of the sounds produced by the audio synthesizer."The Bubblegum Sequencer: Making Music with Candy

"Telestory is an educational, language learning application created by Seth Hunter. In this video, the child is looking at a television screen. He can control onscreen characters, events and objects with the siftables. For example, he has the dog and cat interact by placing the dog and cat siftables next to each other."TeleStory Project WebsiteHere is a video of how Siftables can be used as equation editors:

"Recently, the number of children having autism disorder increases rapidly all over the world. Computer-based training (CBT) has been applied to autism spectrum disorder treatment. Most CBT applications are based on the standard WIMP interface. However, recent study suggests that a Tangible User Interface (TUI) is easier to use for children with autism than the WIMP interface. In this paper, the efficiency of the TUI training system is considered, in comparison with a conventional method of training basic geometric shape classification. A CBT system with TUI was developed using standard computer equipment and a consumer video camera. The experiment was conducted to measure learning efficacy of the new system and the conventional training method. The results show that, under the same time constraint, children with autism who practiced with the new system were able to learn more shapes than those participating in the conventional method."

"External representations have been shown to play a key role in mediating cognition. Tangible environments offer the opportunity for novel representational formats and combinations, potentially increasing representational power for supporting learning. However, we currently know little about the specific learning benefits of tangible environments, and have no established framework within which to analyse the ways that external representations work in tangible environments to support learning. Taking external representation as the central focus, this paper proposes a framework for investigating the effect of tangible technologies on interaction and cognition. Key artefact-action-representation relationships are identified, and classified to form a structure for investigating the differential cognitive effects of these features. An example scenario from our current research is presented to illustrate how the framework can be used as a method for investigating the effectiveness of differential designs for supporting science learning"

"(pdf). According to this vision, "the goal of Tangible Bits is to bridge the gaps between both cyberspace and the physical environment, as well as the foreground and background of human activities." This article isis a must-read for anyone interested in "new" interactive technologies.

The pictures in the article of the metaDesk, transBoard, activeLENS, and ambientRoom, along with the references, are worth taking at least a glance of this seminal work.
Another must-read is Hiroshi Ishii's 2008 article, Tangible Bits: Beyond Pixels(pdf). In this article,Ishii provides a good overview of TUI concepts as well as the contributions of his lab to the field since the first paper was written.

Related tothe Tangible User Interface research is the work of theFluid Interfaces Group at MIT. The Fluid Interfaces Group was formerly known as the Ambient Intelligence Group, and many of the group's projects incorporate concepts related to TUI and ambient intelligence. According to the Fluid Interfaces website, the goal of thisresearch group is to "radically rethink the human-machine interactive experience. By designing interfaces that are more immersive, more intelligent, and more interactive we are changing the human-machine relationship and creating systems that are more responsive to people's needs and actions, and that become true "accessories" for expanding our minds."
The Siftables project is an example of how TUI and fluid interface (FI) interaction can be combined.
Siftables is the work of David Merrill and Pattie Maes, in collaboration with Jeevan Kalanithi, and was brought to popular attention through David Merrill's recent TED talk:David Merrill's TED Talk: Siftables - Making the digital physical -Grasp Information Physically "Siftables aims to enable people to interact with information and media in physical, natural ways that approach interactions with physical objects in our everyday lives. As an interaction platform, Siftables applies technology and methodology from wireless sensor networks to tangible user interfaces. Siftables are independent, compact devices with sensing, graphical display, and wireless communication capabilities. They can be physically manipulated as a group to interact with digital information and media. Siftables can be used to implement any number of gestural interaction languages and HCI applications....Siftables can sense their neighbors, allowing applications to utilize topological arrangement..No special sensing surface or cameras are needed."

"Imagine if all the little programs you had on your iphone were little separate chicklets in your pocket.You'd lose em.Your cat would eat em.You'd vacuum them up.They'd fall down in the sofa.They'd be all over the car floor.You'd throw them away by mistake..."

In my opinion, it is exciting to learn that perhaps some of this technology has the potential of becoming main-stream.

More from YDreams:"...Flapi, YDreams' in-house mascot, and other virtual characters interact in real-time with a little girl and other physical obstacles in a new seamless augmented playground environment."

Photo from YDreams Lab"YLabs’ main focus is on Reality Computing, which uses new technologies such as mobile computing, augmented reality and ubiquitous interactivity to bridge the distance between the user, information and the machine, in a physical, post-browser environment, where the real and the digital come together."

This is a photo of YDream's Architek's yMagic Books. Architek is used to create interactive digital content, including children's storybooks that are manipulated on a touch-screen.

This is a demonstration of Architek's yWalk, an immersive virtual playground that can be vertically projected onto soft mats and floors.

The Architek software provides information about user interaction. yWalk looks like it might be useful for occupational or physical therapists in their work with young children.

Feb 22, 2009

Rich White is an educational technologist for Greenbush County, Kansas, has been working with the 3D interactive virtual world, EduSim, for quite a while. He's taking EduSim to the next level.

"The concept is one of an enclosed virtual learning space - with surrounded projection of the virtual learning world the students are exploring - similar to the StarLab Concept (with a rectangular configuration). along the lines of a CAVE - however simpler, mobile, and relatively in-expensive by comparison."The project is at the beginning prototype stage.

Below is a demo of the virtual world as it is projected on two screens that are placed next to each other at a right angle, with the center of the virtual-world view positioned where the two screens meet:

This might be a great way of reaching students who have autism!

More about EduSim:

EduSim, for those of you haven't seen my previous posts on the topic, is a multi-user 3D interactive environment used in classrooms with interactive whiteboards:

Wikipedia entry:"Edusim is a Cave Automatic Virtual Environment based concept of lesson driven 3D virtual worlds on the classroom interactive whiteboard or classroom interactive surface. The Edusim concept is demonstrated by the Edusim free and open source multi-user 3DOpen Cobalt virtual world platform and authoring tool kit modified for the classroom interactive whiteboard or surface. The Edusim application is a modified edition of the open source Open Cobalt Project and relies heavily on the affordances of direct manipulation of 3D virtual learning models and Constructionist Learning Principles."History of Edusim:"The Edusim project began in September 2007 at the Greenbush Education Service Center in Southeast Kansas as an effort to bring an engaging 3D experience to the classroom interactive whiteboard. Pilot groups were established with 6th and 7th grade middle school students throughout Southeast Kansas to observe how students would be engaged through the software, and how the user interface would need to be augmented to account for the affordances of the whiteboard, and the usability of the students."

Here is a virtual world in Edusim in COBALT, showing how a drag and drop function is used for in-world VNC application sharing:

The Cobalt 3D metaverse browser has been modified for multi-touch interaction by some of the members of RENCI, a collaborative venture of Duke University and several other North Carolina universities. The video below is Dr. Xunlei Wu, demonstrating how gesture and touch is used to manipulate items and navigate through two Cobalt virtual worlds:

Some of the members of RENCI built a multi-touch table in addition to the collaborative multi-touch wall. For more information:

Jonathan Jarvis created a series of animated shorts as a project for his work as a graduate student in the Media Design Program at Art Center College of Design. He started exploring the concept of system diagrams and integrated them into motion interactions.

Jonathan worked on concept development, interface design & content development with a team for a multi-touch project, Interactive Oracles, for Acura:(Some of the information is cross-posted on the Economic Sounds and Sights blog.)

Feb 18, 2009

"The world's first multitouch, multiuser table for primary education - the SMART Table - is now available for purchase.Order the SMART Table"

"As a collaborative learning center, the SMART Table enables engaging and motivating small-group learning experiences. Up to eight students can use their fingers intuitively to sweep, slide and spin objects on the interactive screen. The SMART Table's ready-made activities help primary students gain and further their skills in areas like counting and reading."

"The SMART Table also makes an ideal complement to whole-class activities on the SMART Board™ interactive whiteboard. It helps reinforce concepts in a small-group setting and ensures students can participate in interactive and creative learning experiences."

Feb 15, 2009

The Interactive Displays Conference, sponsored by Intertech Pira, will highlight an interesting mix of existing and emerging interactive display technologies and applications. The conference will be held at the Hilton in San Jose, California, from Tuesday, April 21st through Thursday, April 23rd.

Pre-conference Seminar: Gesture Navigation in the World of Digital Contents, Enabled by a Single-Chip 3D Image SensorPresenter: Sakuya Morimoto, Senior Director, Business Development in Asia, CANESTA, JapanRelated:Hitachi at CES 2009: Use of Canesta's 3D sensor to control television and home systems using hand gestures.

"With the wave of a hand, with the shake of a hand, you can control volume, you can actually change the channels, watch your favorite program...the most exciting thing, I think, is that you can actually control your temperature and the lighting in the room, the environmental lighting. So..it is very unique technology that is out there.."

Another demonstration of Hitachi's gesture interaction using the Canesta's 3=D Depth camera:

How does Canesta's Electronic Perception Technology Work?"Canesta’s electronic perception technology forms 3-D, real time moving images in a single chip through patented methods which use light photons to “range” the image, similar to radar. The silicon sensor chip develops 3-D depth maps at a rate in excess of 30 frames per second, and then performs additional processing on these depth maps to resolve the images into application specific information that can easily be processed by embedded processor(s) in the end-use device or machine. Since Canesta’s software starts with a three-dimensional view of the world, provided immediately by the hardware, it has a substantial advantage over classical image processing software that struggles to construct three-dimensional representations using complex mathematics, and images from multiple cameras or points of view. This dramatic reduction in complexity makes it possible to embed the processing software directly into the chips themselves so they may be used in the most cost-conscious applications."

I will highlight some of the featured presentations in future blog posts:

Feb 11, 2009

It has been a while since I shared information about accessible games. If you are a parent of a child or teen with a disability, if you have a disability, or if you hope to keep on gaming through your golden years no matter what ails you, keeping an eye on innovations in this field is worth your time.

"GATE is actually a Wiki, which is a piece of server software that allows users to freely create and edit Web page content using any Web browser. It's a little like Wikipedia, but just concentrating on assistive technology. GATE is very simple to use, with a control panel enabling you to add content and more. More about Wikis . . .

This wiki has been created by AbilityNet, the UK's largest provider of advice and information on all aspects of Access to technology. The purpose of the wiki is to provide live and up to date information on all aspects of Assistive Technology."

"A really good place to start is their Switch Systems entry here. They are lacking their own accessible games section so hopefully someone (maybe me) will take up the gauntlet for that soon."

Mystic Mine Multi-player One Switch Game

"Just released this February - Mystic Mine is now available to buy on-line at Koonsolo for $19.95 (use www.xe.com for a currency conversion). It is massive fun as a multi-player game and highly recommended by OneSwitch.org.uk. If I had an all time top 10 list of one-switch games this would be a strong contender to make the list. Free demo version available here. Sweet." -posted by One-Switch Games on the Game Accessibility blog

"Pantech is one of three giant mobile phone manufacturers in Korea, among them Samsung and LG. Pantech sold over 10 million phones internationally in 2008, and under the "Sky" brand in the local market. Now, it's about to launch a blow-controlled mobile phone, the IM-S410K, which is also known as the Sky Wind." "Looks like it's got potential for some fun accessible games using sip/puff control." Link Via: Thomas Westin at IGDA GASIG Mailing List, via OneSwitch GamesStevie Wonder calls for accessible technology

"Stevie Wonder is calling for greater access in technology: "[technology] being more accessible is always a plus and I think really, for various companies ... making it exciting and accessible for people who can see, it would take very little to make it accessible to everyone. So I encourage all the manufacturers to do that.""When you can ... make it accessible and make it possible, you should just include that in the overall picture. ". Link via: Mike Taylor of Excitim.

Technology and Special Needs

Proceeds from ads on this blog will be put in a fund for the development of software and related technologies for people with special needs.

ABOUT THIS BLOG

This blog is my main on-line filing cabinet/archive, open to the world. Topics include touch, multi-touch, and gesture interaction, interactive displays, screens, surfaces, environments, and systems; collaborative and social technologies, serious (and not-so-serious) games, 3-D, virtual and augmented reality, innovative learning technologies, interactive TV, DOOH (Digital Out of Home), interactive information and data visualization, the creative use of interactive multimedia content across disciplines and art forms, and universal design/accessibility.

For readers who are interested in digging deeper into a topic, I often post video presentations, slides, links to publicly available scholarly articles, references, related news articles, blog posts, and websites, and references.

The views expressed here are my own do not represent the views of my employer.