Not All Online Students Are the Same: A Summary of Stanford’s MOOC User Study

At every class meetup I’ve attended with MOOC users, students ask each other what other courses they have taken, how the current course compares to those and whether they completed them. They often do, but many students also just audit or sample classes and then drop them very quickly. Now, new research from Stanford shows what those individual conversations suggest — that not all MOOC students are the same.

A significant percentage of students engage in the course for social reasons and do not plan to do all the work required to finish. Some are drawn by the fame of the professor or are intrigued by the course topic. They might want to find out what all the fuss is about. Others know a thing or two about the topic and want to clean up some blind spots they have or dig into only a part of the course.

Most reports about MOOC users have focused on the percentage of students completing a course compared to how many signed up at the beginning. But the LyticsLab@Stanford project followed students lesson-by-lesson and provides a new way for you to think about your experience.

So, if you’re one of the 95% of people enrolled in MOOCs who have up till now been lumped together as “dropouts,” take heart and make sure you document your experience. Course designers want to know how large online classes are being used even by those who aren’t earning a certificate at they end, and now they have some ways to identify you in the mix.

Rene Kizelcec, Chris Piech and Emily Schneider, the Stanford faculty who wrote “Deconstructing Disengagement: Analyzing Learner Subpopulations in Massive Open Online Courses,” were able to identify four significant clusters of students who exhibited the same traits across different courses for high school, college and graduate levels. They recognized that students were of all ages and signed up for a variety of reason from all over the globe. Their research takes into account interactions with the lectures, discussions and quizzes for each lesson for nearly one hundred thousand students.

On track for a while

Regardless of the reasons students take a course, they follow a few identifiable trajectories. Only a small percentage of students stay “on track” with each lesson to complete a course. Staying on track requires a student to digest both the video lecture materials and suggested readings while performing the quizzes and other assignments on time. Many students who complete a course fall behind at times or miss assignments along the way.

Either way, completing the course merits respect if not credit. Certificates of completion are worn as badges of honor and have led directly to jobs for some students. Given that most current MOOCs are given by rock star professors from well-known universities, these top level professors are motivated to not “water the class down” at all. So, if you have completed a MOOC, consider yourself congratulated.

Apart from the “on track” students, the research identified when students were “behind” in their lessons. These students did their weekly assignments but might finish a lesson after the due date. Yet another group of students only watched videos and didn’t do the assignments for a lesson. These were tagged as “auditors”.

Finally, many students didn’t engage in a week’s lessons at all, and were called “out”. Using these tags, the researchers were able to predict how many would take the final exam within an accuracy of 9%. The graphic above illustrates some of the “prototypical” trajectories that students followed, but Kizelcec et al. identified over 20,000 different trajectories through a single course! As the graphic below shows, a student could “audit” the first lessons, fall “behind” for one and then “track” a few, drop “out” for one and then “audit” the next before taking the final.

Auditing, Completing, Disengaging and Sampling

In general, the different trajectories experienced by all students fell into four clusters.

“Auditing” learners watched lectures throughout the course, but attempted very few assessments.

“Completing” learners attempted most of the assessments offered in the course.

“Disengaging” learners attempted assessments at the beginning of the course but then sometimes only watched lectures or disappeared entirely from the course.

“Sampling” learners briefly explored the course by watching a few videos.

By tracking and clustering the engagement of students in three courses taught at different levels, the researchers were able to identify four patterns or trajectories that provide a basis for understanding how MOOCs are being used.

As Brian Whitmer from Instructure argued recently, the MOOC experiment needs to be expanded, and the LyticsLab at Stanford report suggests some intriguing possibilities. For course developers, the findings point to areas where MOOCs can be improved. Courses can be designed to identify trajectories early on and to customize course features based on how a student is doing lesson by lesson.

Not surprisingly, “completing” students are most satisfied with their course experience. They also interact more in forum discussions. New features might move “disengaging” students to another track or offer tutoring to “auditors”. Experiments can be designed to test whether activity with peers within a class motivates students to dig in more deeply.

The researchers leave unanswered a question that might make the biggest difference to anyone who is taking a large online class. What if the class itself can be considered a resource as opposed to a course? Will we be seeing mobile apps and news streams from our classmates, coming soon?

Why is this research important? For most students, it changes nothing about the work they do in a given course. But, it also sheds light on their behavior and provides evidence that many people don’t finish a course but still interact with it in several ways. It’s not just complete or go home!

John Duhring has been s a founding team member at nine startups, including Supermac Software and Bitmenu. During his career he has also applied technology to learning at large companies such as Prentice-Hall, Apple and AOL. Follow him on Twitter @duhring.