Abstract

We present a new task investigating how people interact with and
make judgments about towers of blocks. In Experiment~1, participants in the lab
solved problems in which they had to re-configure three blocks from an initial
to a final configuration. We recorded whether they used one hand or two hands to
do so. In Experiment~2, we asked participants online to judge whether they think
the person in the lab used one or two hands. We found a close correspondence
between participants' actions in the lab, and the mental simulations of
participants online. To explain participants' actions and mental simulations, we
develop a model that plans over a symbolic representation of the situation,
executes the plan using a geometric solver, and checks the plan's feasibility by
taking into account the physical constraints of the scene. Our model explains
participants' actions and judgments to a high degree of quantitative
accuracy.