Using an eyetracker and simultaneously recording the streaming x,y coordinates of the subject's computer mouse movements, he gets an online measure of some of the probabilistic representations (or “tentative interpretations”) that get computed in real-time as the subject attempts to integrate various sources of visual and/or linguistic information.

One finding from this study is that spoken word recognition and syntactic parsing are immediately influenced by relevant visual context — thus compromising strict modular theories of language processing. This, and related, work is described in his book, The Continuity of Mind (2007).