Donald Clark Plan B

What is Plan B? Not Plan A!

Saturday, September 19, 2015

10 uncomfortable truths about the con that is University rankings

Universities claim to be above the lowly business of
commerce but still willingly contribute to the petty hit parades in the
University league table season. It demeans the sector. They search for whatever
scraps they can find by selecting data from one ranking table or another. Why?
For money. They love to claim they are above the competitive, capitalist, corporate game but they are by far the worst when it comes to the dog-eat-dog, institutional competition that are the rankings.
Nozick wrote an incisive essay called "Why do Intellectuals hate Capitalism'. It explains the causality behind the duplicitous thinking of academic distaste for commerce but the utter love of commerce when it suits their institutions. Worst of all, for the people that pay, whether its taxpayers,
parents, national or international students, the University Rankings are largely a con.

1. Bait and switch

The sector loves to take the high moral ground on keeping
managerialism out of education, then use the slimiest form of managerial
marketing, ranking tables, to promote their wares. Aimed firmly at parents and
students, they bait and switch. The hook is baited with data on research and
facilities, then the message switched to make it look like the teaching experience
you’ll pay for, when in fact, the rankings are about measures that have little to
do with teaching. That is a classic 'bait and switch' con.

2. Teaching ignored

They may SAY they take teaching into account but they don’t.
They often claim to have ‘measures’
on teaching, but actually draw their data from proxies, such as staff qualifications and
research activity and use nothing but indirect measures to measure teaching.The Times rankings are a case in
point. They claim that their ranking scores include teaching. In fact, only 30%
is based on teaching but they use NO direct metrics. The proxies include
student/staff ratios (which is skewed by how much research is done) and, even
more absurdly, the ratio of PhDs to BAs. It is therefore, a self-fulfilling
table, where the elite Universities are bound to rise to the top. There is
little direct measurement of face-to face time, lecture attendance or student
satisfaction. In some cases it’s laughable, as Malcolm Gladwell pointed out,
with Faculty salary, levels of degree in Faculty and proportion of faculty who
are full time, being taken as proxies for quality of teaching. It’s like having
a Premier League table based on the performance of the backroom staff and not
the real games and players.

3. False precision

Up one place in the rankings – yippee! Down two places –
time to worry. Yet the idea that these rankings are in any way precise is
silly. They’re a mish-mash of misleading data, under vague (even misleading)
categories and often watered with a heavy dose of opinion (expert panels drwn from top Universities). In any case, they’re
always changing the criteria for ranking, so year-on-year comparisons are useless. This shows itself in the huge disparities between the different ranking
systems. The LSE is 3rd in The Sunday Times rankings but 328th in
the US News &
World Report Rankings, 71st in the QS Rankings
and 34th in the THE Rankings).
Other universities like Manchester
and KCL
do badly in British rankings but well in international tables. This gives ample
room for cherry picking but is poof enough that the way the rankings are
calculated is seriously flawed. If the rankings were research they'd be rejected by even the lowliest of Journals.

4. Apples and oranges

They don’t compare like with like. In Edinburgh, where I
come from, we have four Universities; Napier, Heriot-Watt, Edinburgh and Queen
Margaret. You couldn’t get four more diverse institutions in terms of what they
teach and their history. In 2012 Edinburgh were in top five for research but
came stone-cold last in the teaching survey. That same year, Heriot Watt came
top in Scotland and 4th in UK on Student experience but way, way
down in the rankings. In that same year, more than a third of the Russell Group
Universities found themselves in the bottom 40 of 125 institutions (2012) on
teaching. These comparisons are truly odious.

5. Skews spending

What is sad, even morally wrong, is they they really do
influence strategy and spending. Ranking status is often stated explicitly in
their goals. In effect, as teaching doesn’t really get measured, except through
false proxies, it leads to spending on everything but good teaching – physical
facilities, research and so on. This direct causal effect on behaviour also
leads to overspending, as it’s a runaway train, where everyone tries to outdo
everyone else. There is no incentive to save money and become more efficient,
only to spend more. Weirdly, there’s rarely any accounting for students costs
in calculating the rankings. Shouldn’t a University that costs a lot less get
ranked higher than one that does not? It would appear that prejudice trumps
economics. This is a topsy-turvy world, where being more expensive is an
intrinsic good.

6. Gaming the system

It’s not just spending that’s skewed by rankings, they also skew
behaviour and priorities. Universities are far from being free from the rat
race, they just have some very smart rats. In practice, this means that they
are good at gaming the system. What are the criteria and weightings for ranking?
OK, those are this year’s targets. More facilities, let’s get them built.

7. Self-fulfilling
prophecy

The more you spend, the higher your ranking. So the rich get
richer, the poor get poorer. The separation, in terms of research grants
between the handful at the top and the rest is huge. Naturally, this leads to a
separation of the so-called cream from the so-called milk. In that sense it’s a
deterministic system, where the top remain at the top and the rest scrabble
around for the scraps.

8. Agendas

What’s more, the different tables often have uncomfortable
relationships with newspapers. And let’s not imagine that, given the nature of
newspaper ownership in this country, they don’t have agendas. The Complete
University Guide has had relationships with The Telegraph, Times and
Independent. They keep falling out. The Sunday Times has its Good University
Guide. The Guardian has yet another. These tables sell newspapers to middle class parents, that’s the
real driver.

10. Old boys club

Reputation scores feature in lots of the rankings. You go
out and ask people what they think; academics, publishers, employers etc. Of
course, given that most of the people asked are from the highly ranked
Universities, there’s an obviousskew in
the data. That's shameful, qualitative nonsense.

10. Status anxiety

What is their real effect on parents and students? Nothing
but an irrational race. They induce ridiculous amounts of status anxiety.
Parents and kids are being encouraged to play a game which is already gamed and
get stressed over data that encourages distasteful behaviour.

Conclusion

I haven’t even begun to tackle the issue of cheating, being
economical with the truth or fiddling around with the submissions. There are
examples of straight up cheating, and as there’s no real quality control, it’s likely
to be far more common than reported. In truth, no one really knows what the ideal criteria for ranking should be, as it’s a set of competing ideological choices
– accessibility, teaching, research, graduation rates? And with what
weightings? That’s why the different rankings have these huge disparities. We
need, like Reed University in the US, to refuse to hand in the assessments. If
the game is being gamed, don’t play the game.

Monday, September 14, 2015

10 essential rules on the use of ANIMATION in online learning

Animation, don’t we just love it – The Simpsons, Family Guy,
Southpark, The Lion King, Wallace and Grommet. Unfortunately in many online
learning programmes, I’ve learned to loathe it, not because it’s bad but
because it’s often inappropriate and sometimes unnecessary.

By animation, I mean graphics that move. This ranges from
full character animation, to crude figure animation, lip-synched but static
faces, objects that move, lines that are drawn upon the screen, builds and
animated transitions.

1. Be wary of
extraneous animation

In a now famous experiment, where one group who only saw a
relevant three minute animation of a lighting strike was compared to a group
who viewed the same animation of a lightning strike with six additional, ten
second animation clips, of bending trees, lightning striking into the trees, an
ambulance arriving and a victim being stretched off with a crowd of onlookers.
The first group outperformed the second by coming up with 30% more solutions,
Mayer, Heiser and Lonn (2001). You see a lot of this, as animators go into
movie mode and extend simple, meaningful, explanatory, instructional animation
into complex, extraneous, non-explanatory moving wallpaper.

2. Rinse the
animation

In a car brake animation, Mayer found that stripping out
some of the complex concepts and getting these over, before the animation was
shown, resulted in an increased efficacy of the animation. The lesson here, is
to examine the animation script and rinse out things that you think could be
better explained using text and static graphics. This also makes budgetary sense.

3. Avoid animating
text

Text is meant to be read not admired for its dance moves.
Whenever I see a tile or text flip, rotate or bend without good reason, I’m
annoyed. Maybe, just maybe, if it’s meaningful. I once animated the word
University to make it turn upside down, as I was talking about the ‘Flipped University’.
That’s fair.

4. Do not use text
and animation together

‘Text and
animation’ which both use the visual channel, cause cognitive dissonance and
often confuse rather than achieve learning. Animation, like video, should use
audio narration, rather than accompanying text, Moreno & Mayer (1999).

5. Do not separate
animations from explanatory audio

In an experiment where the workings of a bicycle tyre were
explained, to a group with an integrated animation and audio and another group
with separate ( one after the other) animation and audio, when tested
for transfer, the first group generated 50% more solutions. This large effect
was repeated over eight different experiments, Mayer & Anderson (1991)
(1992), Mayer & Sims (1994), Mayer et al (1999).

6. Animate for flow

For processes, procedures and ordered steps,
animation can be used to build and explain the flow. In particular, animated invisible
processes, such as fluid (blood flow, hydraulics etc.) and air-flow can be used
to good effect. Flow charts, mindmaps and other complex diagrams can benefit
from meaningful and useful animation. The 2D and 3D benefits of animation can do what text and audio can never do, show things they can never show.

7. Graphics and
diagrams

Simple wipes that show lines on a line graph appear as if
being drawn from left to right, or histogram bars being wiped on, can be
explanatory and therefore useful. Indeed, animation can be used to bring data to life, with just a few simple moves.

8. Avoid animation in
navigation

Animated icons, menus that have things rotating, spinning
logos – you name it, I’ve seen it. There are things that just need to be
stopped, superfluous distractions that damage learning. Don’t turn your
learning programme into a fruit machine – unless it’s meant to be a fruit
machine. Just don’t do it.

9. Lipsynched agents

The jury is out on animated tutors and agents but crudely animated
agents or crudely animated lipsynch is a no, no.

10. Don’t get seduced

There is a bottom line here – don’t use animation until you
have exhausted all other possibilities. It’s expensive, can be distracting and
is difficult to change and update. Don’t get seduced by animation, move on.

Conclusion

Once again, rules are never absolute. There are times, in
children’s learning, for people with specific learning difficulties and so on,
where these rules should be broken. But there’s nothing like animation for
producing cognitive overload which inhibits learning. Note also that I’m not
referring to animated graphics as used in high level games and virtual reality.