AR SPOT: An Augmented-Reality Programming Environment for Children

AR SPOT is an augmented-reality authoring environment for children. An extension of MIT’s Scratch project, this environment allows children to create experiences that mix real and virtual elements. Children can display virtual objects on a real-world scene observed through a video camera, and they can control the virtual world through interactions between physical objects. This project aims to expand the range of creative experiences for young authors, by presenting AR technology in ways appropriate for this audience. In this process, we investigate how young children conceptualize augmented reality experiences, and shape the authoring environment according to this knowledge.

In the SpotDocumentation folder, you will find a file called “SPOT Cards”. This contains the tracker cards on the first page. Print these cards on a letter-size page.

Ensure that a video camera is connected to your computer.

In the SPOT folder, execute the “RunSpot.BAT” file.

For more information, see the “SPOT Documentation” file in the SpotDocumentation folder.

AR SPOT Details

The source code for Scratch was modified to include a camera feed, and novel functions were added to the library of programming blocks. The video feed is processed by an external DLL which detects and tracks special objects in the physical environment. Information about the physical objects is then accessible through the programming blocks.

Users can interact through two types of objects: cards and knobs. The system tracks the square marker patterns in order to detect position and orientation of the objects.

There are three ways in which actors can be rendered in the physical scene. One of these preserves the 2D dimensionality of the existing system, by having actors simply follow the position of the object on the screen (if physical objects are moved, the actors match the object 2D position on the screen; however, the actors do not change size or orientation if the objects are distanced or rotated). In two other cases, the actors become more 3D – they can lay or stand on the physical objects, and their appearance changes to follow the 3D location of the object.

New blocks were added to the programming library to make these interactions possible. Motion blocks were added to tell sprites to stick/unstick to the physical objects. Sensing blocks were added for detecting object properties such as distance, rotation, touch, etc. New events were also added which signal when objects become visible or touch.

Future Outlook
This environment should entice more children to create computer-based experiences, at the same time as it generates a lot of data about what mappings are intuitive between physical and virtual worlds. By leveraging the real world, mixed-reality technology can make it easier for children to explore the capabilities of virtual environments. This is because physically interacting with virtual environments may be more intuitive than through a typical keyboard/mouse interface. In crafting projects through this environment, Scratch programmers will generate mappings between tangible interactions and behaviors of programs; thus, a set of frequently-used natural mappings will emerge from the user community. Researchers may also create applications which make use of novel interaction techniques, and use the community to determine if and how children grasp these concepts. In this sense, we expect that the community will become a research partner for tangible interface research.

You can make a new card from an old card – for example, you can take the black barcode from the “blue” card, and put it on a picture of a dog. The software will detect the dog, because the software detects the barcode. However, the dog will be called “blue” in the Scratch program (eg: you will need to use the ‘stick to “blue” card’ block). This might not be a problem if you make the dog blue.. but unfortunately you cannot rename the card.

If you know how to program, you can take a look at the source code for SPOT, and rename the cards yourself. I believe it is fairly easy – let me know if you are interested.

Carlos Almeida

Marcio Monte

Hi Iulian,
Can we use different kind of pattern card or only the cards show in the sample? Can we use 3D models (.OBJ or .3DS)? The manual teach how to use all blocks?
Congratulations for your software.
Regards,
Marcio

Anthony

Hello Mr. Radu, your software looks great, but as im currently running android, I can’t get it, however I was hoping that since you seem to be on the front og the A.R. tech wave that you could point me to someone who could help me find a program that will allow me to perform vertual design physically, with holographic structure in Augmented Reality virtual space, I’ve been looking for one for some time now though, so it may not exist yet… but it is an idea at least of much potential.
Can anyone help me? Please, and Thanks to all.

atin

Hi Lulian,
Congrats on developing ArSpot! My son loves it! The whole AR part is so fascinating and has a great potential to make programming so much more interesting to learn for kids. I’m trying to explore further the possibility of using Scratch, ArSpot and Enchanting to teach programming to Elementary school kids.

I mainly develop on Linux (Centos) and was wondering if you could share the source, so that I could attempt to port it to Linux.