The vast majority of video games that claim to teach mathematics do not actually do that. Rather, what they do is provide a means for students to practice what they have already been taught. For the most part, the focus is on basic computational skills.

A good example is the first-person shooter Timez Attack. Mastery of the multiplication bonds (times tables in parent-speak) is an extremely useful thing to achieve, and the sooner the better. All it requires is sufficient repetition, and I know of no better way to achieve that than with an entertaining video game.

Such games are the low hanging fruit for the math ed video game designer, and like most low hanging fruit, it has pretty well all been picked, leaving game designers coming into the math ed space having to look elsewhere for a useful application of their talents. The good news is, since repetitive practice of basic computational skills is a tiny part of learning mathematics — albeit an important part, in my view (some educators disagree) — most of the fruit in the math ed orchard is still waiting to be picked. The bad news is, that fruit is a lot higher up, and thus more difficult to reach.

The difficulty hits you as soon as you decide to go for more than mastery (ideally to fluency) of already taught basic computational skills. Are you going to approach mathematics as a collection of procedures or as a way of thinking? These are not completely separate classifications; indeed, the latter is in many ways a broader conception than the former. But they do tend to cash out in very different forms of pedagogy. (Spoiler: instruction versus guided-discovery.)

This distinction is to a great extent relatively recent. Until the nineteenth century, mathematicians viewed the discipline as a collection of procedures for solving various kinds of problems. Originally, the problems studied arose in the world. Then, in due course, the focus widened to include more abstract problems arising within mathematics itself. Proficiency in math meant being able to carry out calculations or manipulate symbolic expressions to solve problems.

By and large, high school mathematics is still very much based on that earlier tradition, so few people outside the professional mathematical community are aware that in the middle of the 19th century, a revolution took place.

Working in the revolution’s epicenter, the small university town of Göttingen in Germany, the mathematicians Lejeune Dirichlet, Richard Dedekind, and Bernhard Riemann pioneered a new, broader conception of mathematics, where the primary focus was not performing a calculation or computing an answer, but formulating and understanding abstract concepts and relationships. This was a shift in emphasis from doing to understanding.

For the Göttingen revolutionaries, mathematics was about “Thinking in concepts” (Denken in Begriffen). Mathematical objects were no longer thought of as given primarily by formulas, but rather as carriers of conceptual properties. Proving was no longer a matter of transforming terms in accordance with rules, but a process of logical deduction from concepts.

Of course, during the course of this conceptual thinking, mathematicians still made use of procedures. What changed was the primary emphasis. The reason for the change? An increase in complexity, in science, technology, business, society, and, derivatively, within mathematics itself. In a simple world, a few well-practiced procedures can generally get you by. But when things get more complex, you need understanding in order to select from a variety of different procedures, to fix old procedures that no longer work, and to develop new ones.

I give this somewhat lengthy detour through recent mathematical history not because it has a direct bearing on how we teach K-12 mathematics. The one attempt to modify K-12 education to take account of the 19th century shift in mathematics as practiced by the professional mathematicians, the “New Math” movement of the 1960s, was so badly bungled that even a professional mathematician, Tom Lehrer, satirized it. (It was also hardly “new math” at the time, being already a century old.) Rather, I am stressing the distinction between math-as-procedures and math-as-thinking because it is now extremely relevant to the way we educate our next generation of citizens. The complexity of 21st Century life is such that ordinary citizens now need to upgrade their mathematical knowledge and abilities the same way the professional mathematicians did in the mid nineteenth century. The changes in society, and in particular technology and the way we do business, that were made possible by the newer, richer, and more powerful mathematics that developed in the 20th Century, now affect us all in the 21st.

I discussed the growing importance of “mathematical thinking” in a “Devlin’s Angle” column for the MAA back in 2010, and summarized those arguments in a more recent article in the Huffington Post. My purpose here is not to argue for any one approach to the design of video games to help students learn math. Heavens, the medium is so new, and there are so few games of any real educational merit, there is scope for a wide variety of approaches. Give me any video game that plays well and helps students learn math and I’ll applaud, whatever the pedagogy.

What distresses me is that the medium offers so much promise for good mathematics learning, it is a waste of time, effort, and money to focus on the lowest level — repetitive practice of the basic, procedural, computational skills. We’ve done that. Let’s move on.

Step 1 for the math ed video game designer today is, to recap, deciding whether to develop a game to help students master mathematics procedures or to develop powerful mathematical thinking capacities. As readers of my book will already know, I favor the latter, in large part because mastery of mathematical thinking capacity carries mastery of procedures along with it, just as the person who sets out to build a house will have to develop skills in bricklaying, carpentry, plumbing, and so on, along the way. But as I said a moment ago, the challenge we face in K-12 mathematics education is so great, and video games offer such potential, hitherto largely untapped, I’ll settle for any approach that works.

It’s your call which view of mathematics you take, pre-1850 or post. Both have strong track records. But you do have to make that call, as it will affect every design choice you make from then on. Engineers who set out to build a bicycle and then act as if they are building a car tend not to succeed, even though both are transportation devices. I’ll try to make this blog series helpful whichever way you make the call.

In designing a video game to help students learn mathematics, it’s important not to over-estimate the capabilities of the medium. Breathless articles about the imminent arrival of HAL-like artificial intelligence notwithstanding, the day is not yet here (and in my view won’t be for a long time, if indeed at all) when we can take the human teacher out of the loop. If you really want to develop video games that contribute in a significant way to mathematics education, you should view them as supplementary educational materials or tools to be used by teachers.

A major problem with video games, or more generally any mechanized educational delivery system, is that the system has no way of knowing what the player, or student, is learning. That a player who moves up a level in a video game has learned something is clear. Video games are all about learning. But all you can reliably conclude from a player’s leveling up is that she or he has leveled up. It could have been happenstance.

Okay, but what if the player keeps leveling up? Surely that is not just chance? Possibly not; indeed, given good game design, probably not. The player must have learned something. But what? It might be precisely what that game activity was designed to teach. But it could be something quite different.

This is not a problem unique to video games, or to educational technology in general. It’s a fundamental problem about teaching and learning.

Take a look at the following video from the well-known educational consultant Marilyn Burns:

If you are like me the first time I saw this video, when you heard Cena’s answer in the class you concluded that she understood place value representation. She certainly gave the right answer. Moreover, to those of us who do understand place-value, her verbally articulated reasoning indicated she had conceptual understanding. But she had nothing of the kind, as the subsequent interview made clear.

And therein lies the problem. The human brain is a remarkable pattern-recognizing device. It will even discern a pattern – usually many patterns – in a random display of dots on a screen, where by definition there is no pattern. But is it the pattern the brain recognizes “right” pattern? Cena clearly recognized a pattern, and it yielded the “right answer.” But was it place-value? Perhaps some aspect, but we have no way of knowing

Of course, video games are highly interactive and ongoing. Surely, with video game learning a false understanding will eventually become apparent. Eventually the player will demonstrate that something has gone wrong. Right?

Unfortunately not, as was discovered in 1973, albeit not in the context of a video game but something with similar features. In what rapidly became one of the most famous and heavily studied papers in the mathematics education research literature, Stanley Erlwanger exposed the crippling limitations of what at the time was thought to be a major step forward in mathematics education: Individually Prescribed Instruction (IPI).

Though not a video game, nor indeed delivered by any technology beyond printed sheets of paper, IPI was very similar to an educational video game, in that it presented students with a series of mathematical problems that were selected and delivered at a rate thought to be ideally suited to the individual student, leading the student forward in the same way good level design does in a video game. Without a doubt, if IPI has problems, so too will video games. And as Erlwanger’s paper “Benny’s Conception of Rules and answers in IPI Mathematics” showed, IPI had problems. Big problems. How big? It didn’t work.

The subject of Erlwanger’s study was a twelve-year-old boy called Benny, chosen because he was doing particularly well on the program, moving rapidly from level to level, scoring highly at each stage. As Erlwanger states in his paper, Benny’s teacher, who was administering the program for Benny, felt sure that his pupil could not have progressed so far without having a good understanding of previous work.

Erlwanger’s research methodology was essentially the same as the approach Marilyn Burns used. He interviewed Benny to see what the boy understood. And when he did, a large can of worms spilled out. Though he got high scores on all the question sheets, Benny had almost no understanding of any mathematics, and a totally warped view of what mathematics is, to boot.

Being bright, Benny had quickly worked out a strategy for tacking the IPI question sheets. His strategy was based in part on pattern recognition, and in part on developing a theory about how the game was constructed – yes, he viewed it as a game! And he did what any smart kid would do, he figured out how to game the game.

What the designers of the IPI program had intended was that gaming the game required mastering the mathematics. Unfortunately, there is no way to prevent people, particularly smart ones, from coming up with alternative systems.

In Benny’s case, this involved developing a complete set of rules for adding, subtracting, multiplying and dividing fractions. Though his rules were symbolic manipulation procedures that made no sense mathematically, they enabled him to move through the sheets faster than everyone else in his cohort group, scoring 80% or better at each stage.

Whenever his rules yielded wrong answers, he simply adapted them to fit the new information he had acquired.

When asked by Erwanger, Benny was able to provide consistent, coherent explanations of his methods and why they worked. He was also very confident in his performance, and would stick to his explanations and would not alter his answers when pressured.

I won’t spend time here going through the details. You can read it all in Erlwanger’s paper, which is available here. Anyone who is about to embark on designing a video game for mathematics education should read that paper thoroughly. You need to know what you are up against. (The same dangers arise with gamification, and for the same reason.)

What I will do is say briefly what the fundamental issue is. The designer of the video game (just like the developers of the IPI worksheets) starts with an understanding of the mathematics to be learned, and creates a system to deliver it. The player, or student, does not yet know that mathematics, so they approach the system as what they see: a video game in our case or a series of quizzes in Benny’s. In both cases, the rewards come not from mastery of the underlying mathematics, but from successful completion of the challenge qua challenge. Indeed, with many educational video games, that’s the whole point: turn mathematics learning into a game!

In Benny’s case, not only was he successful in “playing the game”, in the process he developed an entire conception of mathematics as consisting of pointless questions that have a range of possible correct answers, one of which the test maker (in our case, read game developer) had decided, according to some secret but arbitrary set of rules, to declare as the “correct” one. Benny saw his task as to figure out the arbitrary rules the test-maker was using.

Only when you understand the nature of mathematics does Benny’s strategy seem crazy. Without such understanding, his approach is perfectly sensible. He does not know about math, but he already knows a lot about people and about playing games of different kinds. And when this particular game keeps telling him he is doing well, and making progress, he has no reason to change his basic assumptions.

Anyone who sets out to develop a math ed video game needs to have a strategy to avoid falling into the Benny Trap. Personally, I know of no way to do that with any hope of success other than conducting Marilyn Burns type player interviews throughout the development cycle. Fortunately, game developers are already used to doing lots of player testing. Mostly, they are checking for playability and engagement. With an educational video game, they need to augment those tests with interviews to see what is being learned.

That should at least ensure that the game will stand a chance of achieving the educational goal you want. The next issue to address is the circumstances under which the game will be played, and in particular the role of the (human) teacher.