Sunday, January 07, 2018

Astonishing fake in education and training - the graph that never was

I have seen this in presentations by the CEO of a large
online learning company, Vice-Chancellor of a University, Deloitte’s Bersin,
and in innumerable keynotes and talks over many years. It’s a sure sign that
the speaker has no real background in learning theory and is basically winging
it. Still a staple in education and training, especially in 'train the trainer' and teaching courses,
a quick glance is enough to be suspicious.

Dales’s cone

The whole
mess has its origins in a book by Edgar Dale way back in 1946. There he listed
things from the most abstract to the most concrete: Still pictures, Visual
symbols, Verbal symbols, Radio recordings, Motion pictures, Exhibits, Field
trips, Demonstrations, Dramatic participation, Contrived experiences,
Purposeful experience and Direct. In the second edition (1954) he added
Dramatised experiences through Television and in the third edition, and heavily
influenced by Bruner (1966), he added enactive,iconicsymbolic.

But let’s
not blame Dale. He admitted that it is NOT based on any research, only a simple
intuitive model and he did NOT include any numbers. It was, in fact, simply a
gradated model to show the concreteness of different audio-visual media. Dale
warned against taking all of this too seriously, as a ranked or hierarchical
order. Which is exactly what everyone did. He actually listed the
misconceptions in his 1969 third edition p128-134. So the first act of fakery was to
take a simple model, ignore its original purpose, and the authors warnings, and
use it for other ends.

Add fake numbers

First up,
why would anyone with a modicum of sense believe a graph with such rounded
numbers? Any study that produces a series of results bang on units of ten would
seem highly suspicious to someone with the most basic knowledge of statistics.
The answer, of course, is that people are gullible, especially to messages that
appeal to their intuitive beliefs, no matter how wrong. The graph almost
induces confirmation bias. In any case, these numbers are senseless unless you have a
definition of what you mean by learning and the nature of the content. Of
course, there was no measurement – the numbers were made up.

Add Fake Author

At this point the graph has quite simply been sexed up by
adding a seemingly genuine citation from an academic and Journal. This is a
real paper, about self-generated explanations, but has nothing to with the fake
histogram. The lead author of the cited study, Dr. Chi of the University of
Pittsburgh, a leading expert on ‘expertise’, when contacted
by Will Thalheimer, who uncovered the
deception, said, "I don't
recognize this graph at all. So the citation is definitely wrong; since it's
not my graph." Serious looking histograms can look scientific,
especially when supported by bogus academic credentials.

Add new categories

The fourth bit of fakery was to add ‘teaching others’ to
the end, topping it up to, you guessed it – 90%. You can see what’s happening here
– flatter teachers and teaching, and they’ll believe anything. They also added
the ‘Lecture’ category on at the front – and curiously CD-ROM! In fact, the histogram has
appeared in many different forms, simply altered to suit the presenter's point
in a book or course. This is from Josh Bersin’s book on Blended Learning. It is
easy to see how the meme gets transmitted when consultants tout it around in
published books. Bersin was bought by Deloittes. What happens here is that
Dales original concept is turned from a description of media into the
prescription of methods.

The Colored Pyramid

The third bit of fakery, was to sex it up with colour and
shape, going back to Dale’s pyramid but with the fake numbers and new categories added.
It is a cunning switch, to make it look like that other caricature of human nature,
Maslow’s hierarchy of needs. It suffers from the same simplistic idiocy that
Maslow’s pyramid does – that complex and very different things lie in a linear
sequence one after the other. It is essentially a series of category mistakes,
as it takes very different things and assumes they all have the same output –
learning. In fact, learning is a complex thing, not a single output. A good
lecture may be highly motivating, there are semantic tasks that are well suited
to reading and reflection, discussion groups may be useless when struggling
with deep and complex semantic problems and so on. Of course, the coloured
pyramid makes it look more vivid and real, all too easy to slot into one of
those awful 'train the trainer' or 'teacher training' courses that love simplistic bromides.

Conclusion

What’s damning is that this image and variations of the data
have been circulating in thousands of PowerPoints, articles and books
since the 60s.Investigations
of these graphs by Kinnamon (2002) found dozens of references to
these numbers in reports and promotional material. Michael Molenda (2003),
did a similar job. Their investigations found that the
percentages have even been modified to suit the presenter’s needs. This is a
sorry tale of how a simple model published with lots of original caveats can morph
into a meme that actually lies about the author, the numbers, adds categories
and is uncritically adopted by educators and trainers.

PS

Much of
this comes from the wonderful Will Thalhemer’s original work. I wanted to give
it an extra and more structured spin on the development of the fakery.