Thoughts on Education Reform

it makes sense for students to watch video of the clearest calculus teacher or the most lucid analyst of the Revolutionary War rather than having thousands of separate efforts.

True enough, but I think one should try to go beyond a vision in which technology just gets bolted on to the traditional system.

I have been looking at aleks.com, a math tutoring site. It has interesting strengths, along with frustrating weaknesses. I think that its greatest strength is that it offers very clear definitions of mastery of a topic, and it assesses students against those standards. It is very systematic about making sure you know something before you move on to something else. It can substitute for a teacher giving exercises. It gives feedback that is more personal to the student--what you need to work on, as opposed to what the median student out of a class of 25 needs to work on. It costs very little.

Against this, there are a number of weaknesses. One is that working with the software adds a layer of overhead to effort. For example, when I accidentally type in 1.08 instead of 1.80, I am punished for getting the wrong answer by being told I now have to work three more problems correctly in order to move on to the next concept.

In fact, the assessment process is in some ways worse than multiple choice. As with multiple choice, you get graded solely on results, not on process. But unlike multiple choice, you have an opportunity to make all sorts of irrelevant errors, such as accidentally inserting a typo in your answer.

Maybe on net ALEKS is a great step forward. But in the end, it feels to me somewhat like bolting technology onto an older model of learning rather than trying to imagine something really revolutionary. My ideal would be AI software that watches you do a problem, nudges you when you're going off track, and knows the difference between a careless mistake and a fundamental lack of understanding.

Back to Larry Summers, I think that education reform is not going to come from the college Presidents of the world. I think it is going to come from the bottom up, driven by the people who want to learn and by people who have innovative ideas for assisting them.

I think that at some point the best educated people will be self-educated. People like Ben Casnocha, who left college because I presume he felt it was slowing down his learning. Ben does not sit around at home, by any means. Every time I check out what he is doing, he seems to be in a different country.

I think that long before policy makers have figured out how to get everyone into college, college will have become obsolete.

rid yourself of the notion that your current opinions on curriculum, teacher evaluation, technology, or anything else will be the foundation for dramatic gains in student achievement...let me suggest another identity--one whose charge is to return power, in a thoughtful manner, back to parents and educators. Let's call these types of superintendents Relinquishers. With great diligence, these superintendents attempt to transfer power away from a centralized bureaucracy.

Comments and Sharing

I agree with your assessment of flaws in aleks, as well as with the fact that it has great qualities.

One fact obscured by newly applied tech is that the analytical feedback/skill mastery part is the easy step. It was simply a matter of waiting until enough people were ready to do it. It's being done in many ways be a lot of people now, and this analytical piece will be entirely ubiquitous sooner than most people realize.

Then everyone will master rote skills quicker. Then what?

Then we're ready for creative and big picture learning -- a new paradigm of curriculum creation. That's the real new overhead. But it will trade off with so many of current educational costs and costs of the large publishing houses.

People who are experts instantly know how to identify big picture concepts within their fields, and understand why these ideas are so important. We'll know technology is being used right when these "specialists' ideas" become so common that we're less focused on what they are and more focused on teaching students how to constantly recognize, identify, and absorb them in a fraction of the time it took us.

And they'll enjoy the process more. Everybody will.

Solow gave us a good definition of technology, and it may still be that pedagogy trumps analytical tools, but as decentralized as the analytical tools make education, the fundamentals will be taught by a few master teachers. Khan is just one of the first. We'll soon see more.

I agree whole heartedly that education reform will happen from the bottom up, because people will be much more willing to learn the things they need to know to make money in a digital economy than a centralized school will be able to unseat incumbents within the system to adapt to changing times.

However, I've always thought you over-estimate the software element to the education problem.

I think people are developing a clearer picture of what they need to do to create a future for themselves, and it's becoming clearer that traditional higher education is not a workable solution for many. However, for many there's still the problem of person-to-person contact.

For the majority of people, there's no substitute for being in the same room as other people interested in doing the same thing. Without it your interest wanes, and comprehension fails, and it's this aspect (along with credentialization) that traditional schooling undoubtedly has a huge leg up on the open source education movement (for now).

Open source education will take off when self-organizing groups can regularly emerge to provide structure and social support to someone interested in learning something.

This structure need not look anything like a traditional school. A library room, coffee shop, or someone's den would suffice, but some sort of physical organization has to be available to average Joe's for the type of change you describe to occur. (And I think it will.....eventually)

Larry Summers is channeling Frederick Winslow Taylor here: "[I]t makes sense for students to watch video of the clearest calculus teacher or the most lucid analyst of the Revolutionary War rather than having thousands of separate efforts."

Taylorism, as it is derisively referred to today, was a dismal failure. Taylor's factory reforms started with the idea of standardization, since everything else had failed before. Summers needs to check the history books. Next thing, he'll be proposing other ways ("other ways") of constructing students -- ala Fordism, the evil twin of Taylorism.

My daughter has been using Aleks for 2 years. On Aleks it is easy to "master" a topic. Typically the student needs only to answer a problem type correctly three times to achieve "mastery." It is also easy to forfeit that putative mastery through typographical errors or simply idiosyncrasies of the assessment. This forces the student to revisit topics, resulting in better retention. There is no downside to having student demonstrate mastery more than once. Clumsiness of the assessment process isn't a bug. It's a feature.

My daughter, by the way, is 10 years old and is 90% complete with the Aleks 9th grade algebra. Very few available school environments would have provided her with the flexibility to be 4 grade levels ahead. For her at least, more sophisticated AI style assessment would provide no additional pedagogical benefit.

Before we think about video substituting for a teacher, we need to figure out why a book doesn't substitute for a teacher--more precisely, why the mass lecture, with little interaction, survived the invention of the printing press. You can read the best book on the subject--not only the best one by a living author, but the best ever written. Compare that to listening to a lecture by, with luck, the best professor in your university. You can read the book at your own pace, rereading things you don't follow, skimming the obvious. You can read it at whatever time works best for you.

I agree with David. Recalling my college years I would estimate the input of lectures as closing to zero. The most important parts of education were book reading and seminars, where we mainly discussed problems to solve (math and physics). The interaction at seminars was much more productive in understanding any important topic than lectures and books.The latter can not actually provide the full diversity of approaches and common mistakes in tackling problems. And these discussions were invividual.

I agree that there is still a long way to go, but at least some progress is being made. Math is also probably one of the easiest to translate to an "AI" type of format, but there is still a lot to be accomplished.

The core of Aleks is pretty good, but the user interface could definitely use some work. It benefits from the fact that if a student works with it for very long, they learn the style of writing it's expecting and make less typo-style mistakes. It can be very frustrating at first for some subjects, because when you get out of simple decimal answers and into fractions, geometry, calculus and so on the answers get more complex and the student has to write them in a way the computer can evaluate the answer.

Compared to what's possible, even with today's level of technology, Aleks.com comes off pretty short. It's still probably the farthest along the path also in widespread use.

An easy improvement would be to detect digit switches and other simple typos and treat them less negatively than a full wrong answer.

Harder, but still doable today would be to have a database of common mistakes and what they result in, then focus the explanation on how to avoid that particular mistake.

We could spend our time writing code to make the perfect analytics, or to catch mistakes that are really typos, and there is no doubt that this will happen, but these are not the ways for people reinventing education to spend their time in the short term. They are easy to discuss topics that lead us astray from the primary changes that are now available:

Everybody can learn more quickly at once in the online world. This leaves ample room for a complete reinvention of our entire pedagogical approach from what we are teaching (our goals) to how we are teaching it (not simply the delivery mechanism, but with a balance between factual details and examination of the beauty of each subject).

The few groups who recognize this last point, and execute solutions well, will succeed out of the pile of the many who try.

Even with its flaws, ALEKS is still the best computerized math instruction program out there. I started teaching 6 years ago and was initially shocked at the weak arithmetic skills of high school Algebra and Geometry students (e.g. times tables not memorized, counting on their fingers for addition and subtraction, and fractions were completely out of the question.) Drilling math facts in grade school is practically considered abuse these days and it shows. Instead children in primary grades are "exposed" to all kinds of abstract algebra concepts instead of simply mastering the skills that they're actually ready to learn. For example, in my district the Pythagorean Theorem is introduced in 2nd grade to children who haven't mastered adding whole numbers! I've questioned this a number of times and essentially the answer is some version of "Let's teach it to them now so that when we teach it to them later they'll already know it." Of course, the rationale behind that is that the "studies show" that children who mastered Algebra in grade school went on to get PhD's in math, so it must be good for everyone! Yes, believe me, that is how stupid education experts are. Needless to say, the opportunity cost of this strategy is huge and the result is reasonably intelligent high schoolers who are innumerate and hate math as a result. They're completely dependent on calculators and don't have the number sense to do a reality check on the "answer" the calculator gives them.

As a former computer programmer I knew that they needed computerized drill to catch up. What a computer can bring to the table for drill as well as learning new material is that it can give instant feedback on an individual level. A classroom teacher of 40 students at several different levels (remember, tracking is now a dirty word) simply cannot provide that to every student every moment.

I checked out 4 other software packages that claimed to teach math and they were terrible for the following reasons:
1. Multiple Choice - when a student who hates math sees a multiple choice question he says to himself "the answer is here somewhere...maybe I'll get lucky!" Better students will say "Let me work the problem first and then choose." But the clever ones will use the process of elimination as much as possible and avoid actually working the problem. Sure it exercises their deductive powers, but they still need to practice some mathematics!
2. Questions came from a question bank instead of a question generator, which creates new questions within some defined parameters. The question banks were fairly limited as well, so it wouldn't take long for a student to start getting repeat questions and after 4 repeats he may finally guess right! The ALEKS question generator isn't perfect, but it's a hell of a lot better than a bank.
3. Fancy graphics - Big deal. I'm not impressed. This is the same problem with commercial computer math games too.

I've been using ALEKS in the classroom since February 2011. Usually a student will immediately ask for my help when they get a question wrong. But with 42 students it takes a little while to get to each one. While waiting for me, some students will read the explanation and get it. Some will ask another student who either helps him or together they read the explanation and work it out. But what I see the most is that they often simply figure it out on their own from getting it wrong over and over. ALEKS only occasionally identifies the error ("the decimal is in the wrong place" or "check your positive and negative signs"). More AI would definitely be nice, but it's surprising to me how often the kid simply figures out for himself what he did wrong. Then when I get to them they kind of take pride in telling me they don't need my help.
The 3 times correct in a row mastery is a feature, not a bug. What's wrong with starting the counter over when you make a "little" mistake? It either makes you more work harder at precision or it gives you the practice that you clearly need on that topic.
Again, I love the question generator because students can be working on the exact same topic at the same time, but they will have different questions! Initially, they'd lean over and try to cheat, but ended up helping each other figure out their respective problems.
I am much better at explaining the math than ALEKS or any other software I've seen. And I'm better than most videos I've seen, including Khan Academy. But when a kid figures something out for themselves simply from analyzing their own mistakes, that's better than any explanation from anyone. Getting instant feedback let's them do that.

All of this dreaming of new and better learning systems is largely irrelevant because lots of students want a college degree but they don't want to put any effort into learning. Thus, my professor gig is pretty safe: students would much rather pay my princely salary (and live at, let's face it, a resort away from home) and have me spoon-feed them their "education" than sit down in front of a heartless machine and force themselves to learn it on their own.

He has created a data informed drill-and-practice component focused on helping coaches (teachers, students, tutors, others) intervene and assist precisely when and where needed (as well as providing links in the drill to short blackboard explanations of the concepts in question). Even better, his team has added badges and other incentives out of the (addictive) computer game genre.

Blogging software: Powered by Movable Type 4.2.1.
Pictures courtesy of the authors.
All opinions expressed on EconLog reflect those of the author or individual commenters, and do
not necessarily represent the views or positions of the Library of
Economics and Liberty (Econlib) website or its owner, Liberty Fund,
Inc.

The cuneiform inscription in the Liberty Fund logo is the
earliest-known written appearance of the word
"freedom" (amagi), or "liberty." It
is taken from a clay document written about 2300 B.C. in the Sumerian city-state of Lagash.