Saturday, May 18, 2013

MOOC on Human-Computer Interaction: videos have 7 fails in HCI

Enjoying Stanford’s Coursera MOOC on Human-Computer
Interaction but (oh the irony) the screen design and pedagogy of the many
videos, is awful. Don’t get me wrong, it’s easy to use, has good content and I’m
getting what I thought I’d get - a reasonable course. It is very video heavy, which is OK; let’s face
it most HE courses are lecture heavy - at least they’re not an hour long, I can
watch these when I want, repeat them and, in Coursera fashion, I get a bit of
formative assessment during the videos, something students rarely get in real
institutions. But it could have been so much better.

1. Small screen, low retention.

Now I don’t mean to be picky but having a tiny talking head (see above),
literally less than 4.5%the size of the screen on the bottom
right, is a BAD idea. Nass & Reeves (ironically from Stanford) did a great
experiment with video at different screen sizes, showing that
the smaller the screen size the lower the retention. Read the research guys.

2. Too much talking head.

It’s dull, the delivery is often poor and poorly edited
(i.e. padded out). It’s like watching a very long news item read on one news
story, time after time. Even worse is the fact that it’s a medium shot, showing
the whole upper body. Go on a video course guys, you need to see the white of
their eyes. Use close ups. To be fair there’s more images, graphs and screens
with audio only as the course progresses. Think about ‘attention’ guys.

3. Cognitive dissonance

The too much talking head error is compounded by presenting
text headings and blocks of text in a huge font over the rest of the screen.Mayer & Clark’s research showed that you don’t show text and video at same
time, as you have to hop back and forth from visual (reading - semantic) to audio channels. Even
worse, Scott the presenter reads the text but it’s not the text that appears
on the screen. Also, the framing of the video, with text cut in half behind the presenter is cognitive noise we can do without. Watch some TV guys.

4. Paucity of images

What’s odd is the fact that when schemas or techniques or
procedures are being described there’s often no images shown. This is like PowerPoint
without any pictures, just big headings. In many places the point, event or
procedure would have been better served by cutting away to what was being
described or relevant images. It gets better in some places. Very strange.

5. Presentation style

As it’s often a little dull, I found my attention tended to
drift. I can read faster than the presenter speaks, and when in the first video
he started looking down and reading points one by one, the video producer in me
rebelled. I get impatient with slow, amateurish delivery, which is why I like
the edX and Coursera x1.5 speed feature. In fact, so much of this sounds like
the reading of written material that it could have been text. Read something on
relevant media mix guys. I like Scott Klemmer, but he’s no presenter, and after a
while his excessive hand gestures and delivery style start to grate. This can be a problem for the single academic courses - it's like watching a ten week news programme with only one presenter.

6. Poor editing

The in-video questions are poorly edited in, so you often
get a snippet of a sentence from the next sequence. Small point but it makes
the production seem a little amateurish. Edit it properly guys. Again Nass & Reeves showed that these unexpected and awkward pauses and edits lower
retention.

7. Poor question design

In-video questions are made progressively easier and meaningless.
This is learning design at its worst. The
same question is posed, with the same options, up to four times. So when you’ve
answered 1 out of 4, the next question is 1 out of 3, the next 1 out of 2 and
the last a meaningless 1/1. Even worse, is the cardinal sin of two options
actually being correct but only one accepted. All of this is bizarre and lazy.
Read something on test items guys.

Not all Coursera MOOCs are so poor on video. The University
of Edinburgh MOOC on E-learning and Digital Cultures (one of 6 MOOCs attracting
306,000 starters), which ran at the start of the year, didn’t do talking heads,
relying on curated video. This caused some consternation with students who
expected lectures. This course had much more of a looser structure with
discussion, Google Hangouts and social media, none of it moderated by academics. Interestingly, I
liked this Edinburgh course less, as I thought it was weak on focus, depth and content.
There’s a balance to be struck here and much to do on improving the pedagogy
and design of MOOCs. I don't buy the cMOOC/xMOOC thing - it's a simplistic dualism. There a whole variety of pedagogies that lie between straight instruction and social constructivism (I like neither in their pure form).

Conclusion

What
Coursera should have done is do what this course recommends, apply the
usability test strategies that Krug, Norman and Nielsen recommend. Get in the
experts and do ‘voiced trials'. I have spent nearly 30 years designing and
producing online learning and would never have got a client to pay for these
courses. To be fair, compared to the benchmark of dull one hour long lectures,
it’s an improvement and it’s a start. This is a constructive critique of the
videos and I must remind you that the course is rich in assignments and practical work. Let’s hope they
get some HCI professionals in to make it a little more usable and ‘learner
and learning’ friendly. It’s not as if people haven’t done this before.

5 Comments:

You might want to give The Media Equation's chapter on screen size another read. The section on 'Big Screens and Education' (p. 200-201) clearly states that "there are some good arguments against big pictures. One is that too much arousal can be counter-productive in learning. It is quite possible for emotional experiences to distract."

Mather (2007) reviews the two opposing hypotheses that arousal enhances or impairs binding in memory and presents a framework that facilitates predictions of whether an arousing object will enhance or impair memory-binding. Mather (2007, pp. 44-45) clarifies that "arousal associated with an object elicits focused attention that enhances within-object binding, making it easier to later remember which features (such as color and location) were associated with that object. But that focused attention does not benefit binding of the arousing object with other objects or with contextual features and sometimes even impairs it."Given that learning requires the formation of connections between new objects and existing ones, this raises a serious issues with simplistic claims about the association between attention and learning.

I was going to say that it appears those who lecture on HCI don't heed their own words. When I was lecturing in that discipline and referred my learners to Neilsen's site for research it was the opposite of what he talked about. Two column layout, awful font choices, navigation could be disorienting.....I've just checked this out again and at last he seems to be practising what he preaches a bit more. However he still goes for the two column layout. Anyone who has done anything about accessibility should know this is a no no for screen readers.Its a case of physician heal thyself. We know the rules of HCI so we can ignore them!

GingerboxCouldn't agree more. Neilsen was famous for not eating his own dogfood. However, in a course on HCI, where usability affectes learning and retention it is astonishing that they break so many rules. The problem is that academics rarely have sufficient knowledge of the psychology of learning, especially online learning. There is a culture of letting the academics do the learning design. They are different disciplines. This is a general problem for MOOCs. I do think that Udacity are ahead of Coursera and edX on this, but all need to improve.

Rene - the study, I think, was clear "Larger screens mean more arousal, stronger memories, and more positive evaluations of the content they displayed". The point you refer to comes up in the general discussion at the end of the paper and concerns distractive arousal, such as high action images of roller coasters that detract from learning, say about gravity. This is a point that Mayer tested and confirmed. Note that Nass & Reeves are NOT saying that arousal is BAD. In fact they correlate arousal with retention. In the case of the Coursera course, there is no fast action distraction. The 4.5% of the screen video is small and does not arouse in the first place - confirming their study.The Mayer research confirms this. What the Coursera course does on screen is present the SAME competing objects on the SAME screen in two different modes - text/audio. Their research confirmed that this impairs learning. We are not talking about roller-coasters here - but about the presentation of knowledge aimed at semantic memory. I think, in the end you, I, Nass, Reeves and Mayer all agree - media rich does not mean mind rich, but media poor often means mind poor.