Good morning.
Friday I asked - does anyone know of any randomized evaluations of Khan Academy?

So far, the answer seems to be: no. However, I did learn a lot...

1. Via Sharon at Boston Prep (thanks!), this study will be out in December 2012

SRI International's Center for Technology in Learning (CTL), with funding from the Bill & Melinda Gates Foundation, is studying the adoption of Khan Academy in a wide variety of primary and secondary schools and classrooms in Northern California during the 2011–12 school year. Twenty-one public, charter, and independent schools are participating in the study. CTL's research encompasses

*Case studies to understand how Khan Academy is being used to support math instruction in different school settings
*Quasi-experimental designs to measure the potential impacts on student learning
*An analysis of the costs of adoption

2. Okay, but what do we know now? Cormac reminded me that our friend Brian Greenberg of Envision (former Match principal intern) had written a non-experimental study of using Khan in their charter school summer academy.

It's excellent. If you have any interest in this topic, I encourage you to read the whole paper. I'd share the best nuggets, but I think the file is somehow encrypted to make pasting difficult, or I'm just dumb. Since I can't think of a good reason why they'd do the former, let's bet on the latter.

3. It's not a study, but a friend at a West Coast CMO described a widespread implementation of technology, which included kids working with Khan. Her summary:

...The kids really like using their laptops (though keeping them on track is sometimes a challenge).

That's exactly what I figure a randomized experiment would account for -- the "net" of the advantages vs. the disadvantages.

4. This is a Slate article about Khan. I link to it because the comments are valuable. If you skip past the whining and technophilia, you get some nuance. Like this from teacher "Dag":

Is the idea of the reverse lecture good. Absolutely and with my AP Calculus students I take advantage of it time from time (I make my own video's for my students and the students have said they appreciate them).

But realize, these are my AP students we are talking about, students who would thrive in pretty much any environment. These students are the ones who will actively seek help when they need to whether its through internet resources to coming in at lunch or after school to get additional help from me.

Unfortunately, the positive work ethic displayed in my Calc class is not prevalent in the lower classes.

I also teach Algebra 2 students, a class with a mixture of motivated to not motivated students. They are good students but if I tell them to go watch a certain video lecture so that tomorrow we can spend the day working on problems about 1/3 to 1/2 might do so. It's not that they are stupid or lazy they just think that they don't need to learn on their own, that they will get the concepts without help. The problem is of course, that what ends up happening is that the next day they will either call me over or try having a friend help them (teach them all the concepts that they should have been exposed to in the video), or simply don't understand the concept.

And these are my Algebra 2 students. In an Algebra 1 class you probably would get next to no students who would bother to watch the video lecturers.

Please don't get me wrong, I would love for the reverse lecture model to work. I am at my best as a teacher, when I am going around helping students individually. I can pinpoint where their issues are and fix them. If I knew for certain that my students were getting a decent introduction into the concepts outside of the classroom I would not hesitate to adopt this style. But it is simply not the case.

...The AP Calculus students that I taught at this previous school were so vastly different than those I currently have. Their precalculus teacher was a sub, they were never taught trigonometry which is vital for Calculus. For many english was not their first language and while they had obviously found a successful way of learning the concepts that does not change the fact that they were at a disadvantage to my current students.

Would having access to a source like Khan's academy help students like these, probably (though many of the student did not have access to a computer) but it would not be a "fix" for the situation they are in by any stretch of the imagination.

Another teacher, Kate, writes:

My students like using Khan and they like going at their own pace. I don't use Khan exclusively, but it is useful for allowing my top students to move forward . It allows me to free up more of my time to spend on the students who really need the help, which is most of them.

I have found since using Khan (last year and this year) that my students check out of algebra because they don't have a strong foundations in arithmetic. I had my students start at 1+1 exercises this year and move forward. This has made a world of difference and we are further ahead, as a class, than ever before. I have found this is the motivating factor. It seems once my students are able to master the basics, it makes algebra easier and they will actually do the work.

5. And this one, from Slashdot....a range of comments about Khan. This one by "Bruce" was interesting:

Everyone hates to have their business made into a commodity, that's simple economics. Once it happens, you have to compete on cost alone and be hyper-efficient to make a buck. You can only stay above that if you have a clear and provable advantage over the commodity version, and such things are difficult to maintain as the quality of the commodity version improves.

Look at this like Wikipedia. There are obvious quality problems, but Wikipedia keeps improving and getting larger, and if you're Microsoft Encarta, there's just no market for you any longer (thus, the first MS product actually killed by Open Source).

The guild apprenticeship system really hated book-learning. Copyists really hated printing. Both of these were previous means to commoditize education. This is just more of the same.

There will be tremendous economic repercussions from the further commoditization of education.

I've been involved with Match Education for about 12 years — for seven years as a board member and as CEO since 2011. Before joining Match, I started and ran the Newark Charter School Fund and taught education stuff at Harvard Business School (odd but true). Way back, I was a dot-com entrepreneur. My first job in education, at 23, was as an assistant principal in a catholic school in Harlem.

We do four things here. We run a public K12 charter school in Boston (Match Charter School). We run a graduate school of education that prepares rookie teachers for work in high-need schools (Sposato GSE). We run an alternative college and jobs program for low-income students (Match Beyond). And we share our ideas and practices with the world (Match Export).

Assorted personal facts: I moved to New Jersey from Denmark when I was nine (the Danish part explains my weird name). Upon arrival, I learned English by watching television. I have three brothers. My wife and I have three daughters. The first thing on my mind when I wake up every day is espresso - I really like it. I also watch a lot of soccer on tv. I think it's the greatest sport in the world and a force for world peace.