Does the future belong to MOOCs? Massive Open Online Courses, or MOOCs for short, have been hailed as the next wave in secondary education, poised to replace brick-and-mortar colleges with their expensive infrastructure and sky-high tuition. In this episode of Rationally Speaking, Julia and Massimo discuss how to measure MOOCs' effectiveness, separating the hype from the genuine promise.

Reader Comments (12)

Regarding higher education as signaling - wouldn't this work for MOOCs as well, albeit in a different way? Given the low completion rate and even lower pass rate, surely passing (as opposed to enrolling in) a reputable MOOC course (e.g. the excellent and hard Stanford Crypto course on Coursera) would signal something

thank you for pointing to several serious issues around MOOCs, it was an interesting episode again, as always.

As an enthusiastic MOOC participant however, I feel urged to add some thoughts.

You compare MOOCs with brick and mortar colleges, and that is a possible starting point, of course, but it is only part of the story.

To me, MOOCs are much more about life long learning than about education for pubescent children. This changes the view on several points.- I have a full time job, and thus no chance to join a physical college. Not even a part time one. Only because I can view the videos, and do the homework, at any time I want, it is possible for me to join.- I have only partly control over my time, as the (job) workload may explode for reasons I cannot influence. A committment to something beside job and family is something I cannot easily give.

This means that for me, high quality MOOCs are one of the best developments of the last years.

Julias suggestion to increase engagement (put money in, and only get it back when you finish) would possibly make the statistics better. But among other reasons, because people like me, who know to have only some chance to find the time, may not even start. The best writeup I've seen about this is by Robert Wright (from the side of the teacher) this April [1]

I find it funny that Massimo doesn't believe in the first O of MOOC. Massimo, you of all people? A person who is among the makers of one of the best podcasts on this planet, for free. I giggled a bit when hearing you say this.

One factual addition. Massimo assumed that MOOCs miss the writing. This is only partly correct. First, many MOOCs, e.g. Peter Singer's MOOC btw., ask for written essays. These are peer assessed. I have been told that this is an interesting approximation to "real" assessments, but didn't read the article, because paywall [2]. MOOCs that are more mathematical, or around computer science, have other possibilities, as tasks can have results that are checkable automatically (here Coursera was a forerunner, as Andrew Ng, one of the founders, did this already in 2011 in his Machine Learning class, with such success that he decided to found).

An interesting point may be that the motivation for MOOCs are mainly intrinsic, and about knowing material the lecture is about. I assume this, because most of the other motivations Julia mentioned at the start, don't apply in a MOOC. So, even if Massimo is right, that MOOCs support those who are already motivated, and don't (he thinks) need much support - here the support goes to motivated people who until know only had the motivation, but not the possibility.

Well, I'm just one data point. Nevertheless, to me, MOOCs are a massive enrichment of my learning possibilities.

[1] Robert Wright: "The Death of MOOCs Has Been Greatly Exaggerated" http://www.slate.com/articles/technology/future_tense/2014/04/mooc_completion_rates_don_t_matter.html (April 2014)

I want to echo what Falko said. I am in a similar situation - full time job and using MOOC courses for lifelong learning opportunities.

I disagree with a lot of what was said on this podcast (so much so that I had to stop listening halfway through because of the obvious anti-MOOC bias someone brought to the table). One thing in particular was the view that MOOCs are only good for the human capital improvement. I think there is a networking opportunity with MOOCs because the instructors highly encourage participation in forums and providing feedback on assignments (at least with respect to my experience with Coursera). While this is not, of course, the same as fact-to-face interaction, you do have the opportunity to share ideas with others. I've personally learned a lot about being a better writer just by interacting with others on exercises.

I also disagree with the idea that only through traditional schooling can one acquire the skills to become a better citizen in a democracy. Most of the seven courses that I have taken were taken because I want to be a more well-rounded person. I took an applied statistics course so I can better understand science news; I took a songwriting course to become more creative; and I took a Con Law course with Akhil Reed Amar to get a different interpretation of U.S. constitutional history. None of these have a direct relationship to my actual job as a software architect, but they all have made me a person (I hope at least).

Anyway, what was said in this episode did not reflect my own personal experience.

Whatever the platform, MOOC, online, blended, lecture-based, teachers need to look at the assignments they are giving students. The tasks assigned to students should not just involve them in the content of the particular subject of the course. They should require the student to exercise those discrete abilities that teachers want the students to retain long after they have graduated. Reading a lot, writing a lot = good. But activities can be developed that reinforce (for want of a better word) virtues. And it is possible for collaboration to take place across disciplines so that teachers in science, philosophy, and -- gasp -- the humanities could mutually reinforce the activities undertaken in each subject area. Drs. R. Paul and L. Elder will be known followers of critical thinking and skepticism. They have some useful practical suggestions at http://www.criticalthinking.org. I find their 8-item breakdown of critical thinking elements a little unwieldy. In the context of interdepartmental meetings or curriculum planning anything more than 5 items becomes difficult to keep in focus (The Foundation's 8 elements measured across 10 standards = 80 items to keep track of: you try addressing them all at a 90 minute staff meeting!). And it would be awful if their taxonomy were implemented as some bureaucratic "Core Teaching Principles and Practices Doctrine" and imposed on a campus or a state-wide university or college system. But I would be surprised if greater the commonality of standards across different subject areas did not result in greater performance in the areas evaluated by the Academically Adrift research. And computer databases, keeping track of individual students' performance in the basic elements and standards of thinking, might offer a way to provide meaningful feedback, remediation, and enrichment tailored to each member of a course with large or massive enrolment. Students appreciate quality instruction and any department or departments could do worse than try to address their students' needs in a collegial fashion.

As a graduate student at Yale, I feel obligated to point out that Massimo was wrong. Yale pays significantly more than 3300$ per course as a TA. The standard Tf2 position pays about 5k per course, and that it is with a work load of about 10hrs per week. The range actually goes all the way up to 9800$ per course. 3x as much as what Massimo says in the podcast.

I am just another sample of 1, who has had extensive experience with small classes at elite universities and in two graduate disciplines and my more limited experience of MOOCs diverges widely from some of the assumptions advanced in the podcast.

1. Small class size is better? Check out EDX, Thinking 101X MOOC, https://courses.edx.org, for a well researched rebuttal on this, not to mention the questionable assumption that small classes exist on an undergraduate level at any but the most expensive schools & advanced classes.

2. I would agree that much of the completion data is contaminated by shoppers. I have completed 4 out of 8 MOOCs so far and two of these 'failed to completes' were because I was uninterested in the material after sampling them. In a third, I determined I did not have the prerequisites after the third class. And the fourth, I failed to complete because I overextended myself and that may have been addressed by Julia's pre-pay suggestion.

3. The lecture format and talking head model may be dominant among MOOCs, however the production quality of the above Thinking course and Dan Ariely's & Duke University's A Beginner's Guide to Irrational Behavior on https://www.coursera.org are phenomenal, high quality video experiences on a par at least with a Nova or Frontline broadcast.

4. Two proven pedagogical tools are frequent testing and application or reformulations of the material to establish new relationships and ladder it with existing knowledge. My experience of the frequent testing on both the lecture and reading materials was that it really helped me learn the material, learn what I had not learned yet and practice. The tests, all be it multiple choice, are available throughout the course. This made, for me, a wonderful tool to anticipate and review the lectures and reading, before, between and after the scheduled releases.

5. Extensive writing opportunities exist in both written assignments and their peer evaluation. The assignments have been demanding: designing experiments, extensive research and journaling assignments. Most were evaluated. These evaluations were often through multiple reviews with very clear rubrics and multiple reviewers and opportunities to perform multiple reviews. All of the above were very helpful methods to really apply and integrate the course material. I would venture that the peer reviews received and opportunity to review the work of other students compares quite well to both the dogged review of teaching assistants in huge classes and the better than average professor's review of the essay submitted by the 23rd out of 24 students in the rare small class, college experience that they have taught for the past ten years.

6. Discussion groups are another arena for reorganizing and reapplying the course content that I felt really helped me understand the material and opened new connections that had not occurred to me earlier. After the frequent testing, I would rate the discussion groups as the most effective pedagogical tool. Massimo's concerns for discussion groups going off topic or being skewed by some individuals is real. My probably biased experience was that it did not seem much of a problem. I am not sure whether this was a result of skillful moderators or cooperative students, but the sincerity and helpfulness of the students and moderators outweighed the distracting rants 50 to 1. I understand that even small classroom students are sometimes subject to such behavior.

I would like to second or third many of Falko's comments as well.

I want to thank you for all the effort you have put in to the production of this podcast. You always make it sound fun and enjoyable, although I suspect it often feels more like work in the preparatory phase. Hopefully your satisfaction with a fine product, your peers' recognition and participation in promoting your field, the thanks of a grateful nation and my sincere praise will encourage you to keep exerting your effort and sharing your expertise. I think you do a real service by making this material accessible to a hopefully massive online audience in such an open way.

One of my main learnings in this podcast was about maintaining a skeptical view, "Where's the evidence?" rather than sinking into the cynical slam of "This is all bull!" If "The test of a first-rate intelligence is the ability to hold two opposed ideas in mind at the same time and still retain the ability to function." F. Scott Fitzgerald, I found out that I do not have one.

One ad hominem point I will allow myself, since it probably applies to the endowment effect and confirmation biases of my own opinions as well as some of the views expressed in this podcast, are the words of Upton Sinclair, "It is difficult to get a man to understand something when his salary depends upon his not understanding it."

Mort

PS Dan Ariely shares Massimo's concern on the Open future of MOOCs. In one of his office hour sessions (3rd or 4th/ or search the term) of his behavioral economics course he waxes rather pessimistically about the business model and its sustainability.

Bill, above, inadvertently demonstrates a problem with MOOCs. He said:

(so much so that I had to stop listening halfway through because of the obvious anti-MOOC bias someone brought to the table).

Bill ran into something that he disagreed with and stopped listening. You can't do this in a real course. Sometimes students have to be intellectually dragged into having an insight, which is hard work for the teacher and painful for the student. You'll never experience any intellectual growth if you stop listening the moment something makes you uncomfortable.

I'm a senior undergraduate at a small state school, and I must say that my feelings toward my school and toward my culture in general have become more and more bitter each time I think about my mounting debts. I'm a math student (plus I'm very introverted and quite annoyed to be in class usually), so from my admittedly biased perspective, it seems that if our university system has gotten so out of control that I and my fellow millennials are essentially creating an economic bubble with our debt, then people who want these less economically clamored-for degrees (in Literature for example) who need papers graded will just have to pay that pound of flesh to go to a brick and mortar school perhaps, and if MOOCs become viable then more computer science, math, etc students could now have the double benefit of less tuition plus more job security. This might incentivize our students to major more in these programs such that our student makeup is more like that of China or Japan with lots of engineers and such. I love literature, but it's a cold world out there, and reading a bunch of Chaucer doesn't keep the heat on.