Data about the Metadata MOOC, Part 1: Student Activity

My Metadata MOOC is over! I clicked the button to issue Statements of Accomplishment and Verified Certificates earlier today. (If you were a student in the MOOC and you’re reading this, it will take about a week for these to actually be issued, during which Coursera will complete their review and work their magic on their side.) I’m very pleased with how it went, but I’ll make that the topic of another post. Right now, I want to present some statistics about the course.

Coursera provides an embarrassment of riches to the instructor, where data about the course is concerned. We have stats about student activity, stats about views & downloads of the videos, stats about in-video quizzes and homeworks submitted, stats about participation on the discussion forums. We collected some, but by no means all, of this data, while the course was ongoing. And we posted the data we collected to the discussion forums, because we thought that the students might like to see the State of the Course in more or less real-time. (Where “real-time” is a word which here means “weekly.” Note: weekly video content was always launched on Monday mornings, and we collected & posted this data on the following Friday.) That was well received on the forums, and I’ll definitely be doing that again, for the second offering of the course. I recommend it to all MOOC instructors. Taking an online course can feel isolating. The discussion forums were very active, which certainly combatted the feeling of isolation. But I figured, any mechanism to help convey that this course is ongoing right now, live, as we speak, would help. And I’ll be honest, I really enjoyed watching these stats over time. But then, I’m a geek.

Anyway, this is the first of what will probably be several posts, where I report on the Metadata MOOC. First, let’s look at student activity:

I know that hard data on MOOCs is fairly thin on the ground. So for all you data wonks out there, here’s the raw data:

Week

Total Registered Students

Total Active Students

Active Students Last Week

Active Last Week / Total Active

1

27623

10476

10470

99.9

2

27764

12789

8236

64.4

3

27400

13511

6552

48.5

4

26935

13781

5019

36.4

5

26591

13862

4199

30.3

6

26289

13963

3933

28.2

7

26050

14035

3436

24.5

8

25867

14130

3334

23.6

The Coursera help documentation defines these categories as follows:

Total Registered Students: “The number of (unique) students currently registered for this session, excluding those who have unregistered.” Some wag compared registering for a MOOC to clicking the Like button on Facebook. Total Registered Students is, to me, not a very meaningful statistic, since there’s no penalty for an individual to register for a MOOC and then never participate. If this stat means anything at all, it’s as a measure of interest.

Total Active Students: “The number of (unique) students that have logged in at least once to the session site.” This is, for my money, a far more meaningful statistic: the number of individuals who actually did anything in the course.

Active Students Last Week: “The number of (unique) students who logged on to the session site in the last week.” So that’s a rolling week, counting backwards from whenever the stats were generated.

It is interesting, but not surprising, that Total Registered Students decreases over the span of the course. I hypothesize that these un-registrants fall into 2 categories:

Individuals who registered and then didn’t participate in the course, who got annoyed that they kept getting auto-generated emails about the course when we posted announcements, and so un-registered to shut those up.

Individuals who participated in the course up to a point, realized that the course wasn’t as interesting to them as they thought it would be, or life intervened and they had to stop participating, or whatever, and dropped out.

It’s also interesting, but not surprising, that Total Active Students plateaued by around week 3. This makes sense when you consider that the homeworks in the course had deadlines: they were due about 2 weeks after the content for that unit launched. So if you started participating in the course during week 3, you’d already missed the deadline for the homework for unit 1. It would therefore be impossible for you to earn a Statement of Accomplishment, which required completion of all graded assignments.

This is something that I need to reconsider for the next offering of the course: Whether homeworks should have deadlines, or should be available until the end of the course. The former model forces students to form a huge cohort, proceeding through the course more or less in parallel. That seemed to work well this time around, and, I’d hypothesize, kept the discussion forums active. The latter model would allow students to do the course more at their own pace, and could enable more students to complete the course. I got a few annoyed emails & comments in the discussion forums about homework deadlines. But by and large, students seemed to be ok with the deadlines. I’m tempted to go with the latter model on the second round, just as an experiment. But that’s not a pedagogically sound reason. Anyone have thoughts on this, one way or the other?

The stat Active Last Week / Total Active is of my own invention. What percentage of all students in the course have participated recently? It’s a simple, but I think reasonably effective, measure of current activity in the course. I have to admit, I’m pretty pleased that I ended with about a quarter of all students still actively participating. I’ve had class sessions in the classroom that weren’t that active.

Stay tuned, folks, over the next few days, for more reporting about the Metadata MOOC. I’m just trying to keep these posts short enough that you’ll actually read them, plus give myself and the team at UNC some time to do some analysis.

12 Responses to Data about the Metadata MOOC, Part 1: Student Activity

One of the blessings of a MOOC is that you can do it at your own pace … if you choose to do this then you don’t benefit from the cohort that’s pressing ahead at a quicker pace, but you benefit from what they’ve left behind (discussions, etc.). I don’t see a reason not to offer both options: select one on your way in, understand the implications of the option you’ve chosen, and then get started. Until MOOCs start awarding real credentials (college credit?) I’m not sure that the pace matters much.

I didn’t think the discussions were all that great, but I think that has more to do with the really diverse crowd which was attracted to the course.

I find the deadlines for the homework useful as reminders. During the week I was not able to follow the course in parallel with my work, so I followed the course and did the homework in the weekends. Without the deadlines I would probably not have completed the course. I found the homework (too) easy. I was always able to get a 100% score after 2-3 iterations. You have 10 in total … A colleague of mine accomplished the homework each time without actually having seen the videos which is a real shame and not the aim of the MOOC …

I did exactly as you did … well, I also did something else: I ran through the homework once at first (and I only did this near the end) to determine what I already knew and what I needed to watch for … then I watched the videos and did the embedded quizzes (which mostly worked for me–occasionally there’d be a question which I couldn’t sort out) … then I did the homework “for real.”

Willem, have you considered that many course participants may come to the class with a high level of subject matter knowledge and may not need to access the lecture videos? They may just be surveying the course to see if their knowledge is still cutting edge, or in sync with the market. Others may be having their first exposure to the subject during this MOOC and may benefit from being able to practice and submit the homework multiple times. It is hard to be all things to all people, but that is what these courses strive to do. Like most aspects of education, the lesson that is hardest for students to learn is that they can only get back what they are willing to put in. Respectfully, Scott