Tuesday, July 11, 2017

New evidence that ‘gamification’ does NOT work

Gamification is touted as new and a game changer. Full of hyperbolic claims about its efficacy, it's not short of hyperbolic claims about increasing learning. Well it's not so new, games have
been used in learning forever, from the very earleist days of computer based learning, but that’s often the way with fads, people think
they’re doing ground-breaking work, when it’s been around for eons.

At last we have a study that actually tests ‘gamification’
and its effect on mental performance, using cognitive tests and brain scans. The
Journal of Neuroscience has just published an excellent study, in a respected, peer reviewed Journal, with the
unambiguous title, ‘No Effect of
Commercial Cognitive Training on Neural Activity During Decision-Making’ by
Kable et al.

Gamfication has no effect on learning

The researchers looked for change behaviour in 128 young adults, using pre- and post testing, before and after 10 weeks of training on gamified brain training products (Lumosity), commercial computer
games and normal practice. Specifically they looked for improvements in in memory, decision-making,
sustained attention or ability to switch between mental tasks. They found no improvements. “We found no evidence for relative benefits of cognitive
training with respect to changes in decision-making behaviour or brain
response, or for cognitive task performance.”

What is clever about the study is that three groups
were tested:

1. Gamified quizzes (Lumosity)

2. Simple computer games

3. Simple practice

All three groups, were found to have the ‘same’ level of
improvement in tasks, so learning did take place but the significant word here
is ‘same’, showing that brain games and gamification had no special effect. Note that the Lumosity product is gamification (not a learning game), as it has gamification elements, such as Lumosity scores, speed scores and so on, and is compared with the other two groups, one which is 'game-based' learning, controlled against a third non-gamified, non-game practice only group. One of the problems here is the overlap between gamification and game-based learning. They are not entirely mutually exclusive, as most gamification techniques have pedagogic implications and are not just motivational elements.
The important point here, is the point made by the 69 scientists who orginally criticised the Luminosity product and claims, that any activity by the brain can improve performance but that does not give gamification an advantage. In fact, the cognitive effort needed to master and play the 'game' components may take more overall effort that other, simpler methods of learning.

Lumosity have form

Lumosity are no strangers to false claims, based on dodgy neuroscience, and were fined $2m in 2015 for claiming that evidence of neuroplasticity, supported their claims on brain training. There is perhaps no other term in neuroscience that is more overused or misunderstood than 'neuroplasticity' as it is usually quoted as an excuse for going back to the old behaviourist 'blank slate' model of cogntion and learning. Luminosity, and many others, were making outrageous claims about halting dementia and Alzheimer’s disease. Sixty seven senior psychologists and neuroscientists blasted their claim and the Federal Trade Commission swung into action. The myth was literally busted.

Pavlovian gamification

I have argued for some time that the claims of gamification are exaggerated and this study is the first I’ve seen that really
put this to the test, with a strong methodology in a respected peer reviewed
journal. This is not to say that some of aspects of gaming are not useful, for example its motivational effect, just that much of what passes for gamification is Pavlovian nonsense, backed up with spurious claims. I do think that gamification can be useful, as there are DOs and DON'Ts, but that it is often counterproductive.

Conclusion

The problem here is that, in the case of Lumosity, tens of millions are being 'duped' into buying a subscription product that has no real extra effiacy over other methods. Similarly, in the e-learning market, people may be being duped into thinking that gamified product is intrinsically superior to other forms of online learning - when it is not. You may be paying premium price for a non-premium product that has no extra performance efficacy.

That is not the way the word is used by suppliers. It is not simply the addition of league tables or badges. The word is used in a much wider sense and meaning is use. In any case, your definition of 'gamification' is exactly what Lumosity do with Lunosity 'point scores', 'unlock features', 'scores for speed' and so on - these are simply gamification add-ons to their content.

This is not necessarily a negative. Given that the improvement outcome was the same then other questions need to be asked to see if gamified learning was more enjoyable, more engaging, quicker and so on.

As you say, games are not new in education or training and existed before computers entered the picture. There are still some non-computer games in existence I believe. My experience suggests that "gamified" stuff can be useful and effective as part of a varied learning programme. Touting them as a superior alternative to other techniques for teaching and learning is just silly.