Abstract

We present a new method for automatically providing feedback for
introductory programming problems. In order to use this method,
we need a reference implementation of the assignment, and an error
model consisting of potential corrections to errors that students
might make. Using this information, the system automatically derives
minimal corrections to student’s incorrect solutions, providing
them with a measure of exactly how incorrect a given solution was,
as well as feedback about what they did wrong.

We introduce a simple language for describing error models
in terms of correction rules, and formally define a rule-directed
translation strategy that reduces the problem of finding minimal
corrections in an incorrect program to the problem of synthesizing
a correct program from a sketch. We have evaluated our system on
thousands of real student attempts obtained from the Introduction to
Programming course at MIT (6.00) and MITx (6.00x). Our results
show that relatively simple error models can correct on average
64% of all incorrect submissions in our benchmark set.