Publication Venue

Data Set Used

Key Phrases

Sketching is a powerful means of working out and communicating ideas. Sketch understanding involves a combination of visual, spatial, and conceptual knowledge and reasoning, which makes it both challenging to model and potentially illuminating for cognitive science. This paper describes CogSketch, an ongoing effort of the NSF-funded Spatial Intelligence and… (More)

In this paper, we describe CogSketch, an open-domain sketch understanding system built on the nuSketch architecture. CogSketch captures the multi-modal, unconstrained nature of sketching by focusing on reasoning over recognition. We describe this approach, as well as two application domains for CogSketch: cognitive modeling, and education.

Designers often use a series of sketches to explain how their design goes through different states or modes to achieve its intended function. Learning how to create such explanations turns out to be a difficult problem for engineering students. An automated " crash test dummy " to let students practice explanations would be desirable. This paper describes… (More)

This paper shows how a software toolkit can allow graphic designers to make camera-based interactive environments in a short period of time without experience in user interface design or machine vision. The Attention Meter, a vision-based input toolkit, gives users an analysis of faces found in a given image stream, including facial expression, body motion,… (More)

This paper shows how a software toolkit enables graphic designers to make camera-based interactive environments in a short period of time without requiring experience in user interface design or machine vision. The Attentive Interaction Design Toolkit, a vision-based input toolkit, gives users an analysis of faces found in a given image stream, including… (More)

Sketching is a powerful modality for thinking through, and communicating about, mechanical designs. Qualitative mechanics reasoning has been applied to sketched input before but not without succumbing to limitations of domain-based recognition or requiring complex annotation and additional explicit knowledge from the user. This paper presents solutions to… (More)

An important problem for engineering students is learning how to communicate. Based on collaborations with engineering design instructors, we are creating a system, Design Buddy, that is intended to help students learn how to explain their designs. The input is a sketched comic strip with graphical annotations that indicate forces and motions, plus a… (More)

An important problem in teaching engineering design is helping students learn to communicate via sketching. A good explanation of a design should include how the structure and behavior of the design enable it to achieve its intended purpose. Thus the ability to represent and reason about teleology becomes important for creating software coaches for… (More)

Spatial reasoning plays a critical role in STEM problem solving. Physics assessments, for example, are rich in diagrams and pictures, which help people understand concrete physical scenarios and abstract aspects of physical reasoning. In this paper we describe a system that analyzes sketched diagrams to solve qualitative physics problems from a popular… (More)

We have become accustomed to a digital desktop that is customizable and anticipates our needs. What if our physical work surfaces could interact with us in the same way? We have recently completed a working prototype of a work surface that automatically adjusts to individual users and anticipates their needs. The work surface is a large sink and counter… (More)